{
  "paper_id": "c8947d25b4fad1c54ead37183eead261",
  "paper_title": "BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation",
  "paper_authors": [
    "Zhijian Liu",
    "Haotian Tang",
    "Alexander Amini",
    "Xinyu Yang",
    "Huizi Mao",
    "Daniela L. Rus",
    "Song Han"
  ],
  "publication_date": "",
  "publication_venue": "2023 IEEE International Conference on Robotics and Automation (ICRA)",
  "abstract": "Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with camera features. However, the camera-to-LiDAR projection throws away the semantic density of camera features, hindering the effectiveness of such methods, especially for semantic-oriented tasks (such as 3D scene segmentation). In this paper, we propose BEVFusion, an efficient and generic multi-task multi-sensor fusion framework. It unifies multi-modal features in the shared bird's-eye view (BEV) representation space, which nicely preserves both geometric and semantic information. To achieve this, we diagnose and lift the key efficiency bottlenecks in the view transformation with optimized BEV pooling, reducing latency by more than 40×. BEVFusion is fundamentally task-agnostic and seamlessly supports different 3D perception tasks with almost no architectural changes. It establishes the new state of the art on the nuScenes benchmark, achieving 1.3% higher mAP and NDS on 3D object detection and 13.6% higher mIoU on BEV map segmentation, with 1.9× lower computation cost. Code to reproduce our results is available at https://github.com/mit-han-lab/bevfusion.",
  "keywords": [
    "multi-sensor fusion",
    "bird's-eye view",
    "3D object detection",
    "BEV map segmentation",
    "autonomous driving",
    "BEV pooling",
    "multi-task"
  ],
  "paper_institutions": [],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:58.986434Z",
  "template_tag": "simple",
  "summary": "### 摘要\nBEVFusion 针对自动驾驶中多传感器融合的有效性问题，提出将多模态特征统一到共享的鸟瞰图（BEV）表示空间中进行融合，以同时保留几何结构与语义密度信息。该方法通过优化 BEV pooling 操作，将视图变换延迟降低超过 40 倍，并采用全卷积 BEV 编码器缓解局部错位，任务无关的设计可无缝支持 3D 目标检测和 BEV 地图分割等多种三维感知任务。在 nuScenes 和 Waymo 数据集上，BEVFusion 在 3D 检测上分别提升 mAP/NDS 1.3% 和 mIoU 13.6%，同时计算开销降低 1.9 倍，延迟降低 1.3 倍。核心机制是将相机特征通过预测深度分布投射到 BEV 空间，并利用预计算和区间约减优化 BEV pooling，实现了高效且精确的融合，为多任务多传感器融合提供了简单而强大的基线。\n\n### 关键要点\n- BEVFusion 选择共享的鸟瞰图（BEV）表示作为多传感器融合空间，避免了相机视图的几何损失和 LiDAR 视图的语义损失。\n- 相机到 BEV 变换采用类似 LSS 的深度分布预测，将每个像素特征散射成 D 个离散点，并乘以深度概率。\n- 原始 BEV pooling 在典型配置下耗时超过 500ms，占模型运行时的 80% 以上，成为效率瓶颈。\n- 通过预计算网格关联索引和排序，将网格关联延迟从 17ms 降至 4ms。\n- 利用区间约减和专用 GPU 核，直接对每个 BEV 网格并行计算区间和，将特征聚合延迟从 500ms 降至 2ms。\n- 优化后的相机到 BEV 变换整体提速超过 40 倍，从 500ms 以上降至 12ms，仅占端到端运行时的 10%。\n- BEVFusion 在 nuScenes 检测基准上达到 SOTA，测试集 mAP 70.2、NDS 72.9，相比 TransFusion 提升 1.3%，计算量减少 1.9 倍。\n- 在 BEV 地图分割任务上，多模态 BEVFusion 比 LiDAR-only 方法提升 13.6% mIoU，比现有融合方法（如 PointPainting、MVP）提升超过 13%。\n- 在 Waymo 数据集上，BEVFusion 以较少的输入帧数（3帧 vs 5帧）超过 DeepFusion-Ens，且无需模型集成。\n- 在雨夜等恶劣条件下，BEVFusion 相比单模态模型大幅提升检测与分割性能，夜间分割 mIoU 提升 12.8。\n- 对小物体、远处物体和稀疏 LiDAR 场景，BEVFusion 的提升更为显著，体现了相机语义密度的价值。\n- BEVFusion 采用 Swin-T 图像骨干和 VoxelNet 点云骨干，通过任务特定头支持检测和分割，且代码已开源。"
}