{
  "paper_id": "c15724973a096c148cd0847d1fd2904c",
  "paper_title": "RaLiBEV: Radar and LiDAR BEV Fusion Learning for Anchor Box Free Object Detection Systems",
  "paper_authors": [
    "Yanlong Yang",
    "Jianan Liu",
    "Tao Huang",
    "Qing-Long Han",
    "Gang Ma",
    "Bing Zhu"
  ],
  "publication_date": "",
  "publication_venue": "IEEE Transactions on Circuits and Systems for Video Technology",
  "abstract": "In autonomous driving, LiDAR and radar are crucial for environmental perception. LiDAR offers precise 3D spatial sensing information but struggles in adverse weather like fog. Conversely, radar signals can penetrate rain or mist due to their specific wavelength but are prone to noise disturbances. Recent state-of-the-art works reveal that the fusion of radar and LiDAR can lead to robust detection in adverse weather. Current approaches typically fuse features from various data sources using basic convolutional/transformer network architectures and employ straightforward label assignment strategies for object detection. However, these methods have two main limitations: they fail to adequately capture feature interactions and lack consistent regression constraints. In this paper, we propose a bird's-eye view fusion learning-based anchor box-free object detection system. Our approach introduces a novel interactive transformer module for enhanced feature fusion and an advanced label assignment strategy for more consistent regression, addressing key limitations in existing methods. Specifically, experiments show that, our approach's average precision ranks 1st and significantly outperforms the state-of-the-art method by 13.1% and 19.0% at Intersection of Union (IoU) of 0.8 under \"Clear+Foggy\" training conditions for \"Clear\" and \"Foggy\" testing, respectively. Our code repository is available at: <https://github.com/yyxr75/RaLiBEV>.\n",
  "keywords": [
    "Autonomous driving",
    "anchor box free object detection",
    "LiDAR",
    "point cloud",
    "radar",
    "range-azimuth heatmap",
    "label assignment",
    "bird eye's view fusion",
    "interactive transformer",
    "deep learning"
  ],
  "paper_institutions": [
    "James Cook University",
    "Beijing Vanjee Technology",
    "Swinburne University of Technology",
    "Beihang University"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "deepseek-v4-flash",
  "prompt_template": "simple",
  "extracted_at": "2026-08-18T10:31:07.541641Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n本文针对自动驾驶中雷达与LiDAR融合感知在恶劣天气下的鲁棒性问题展开研究。现有基于锚框的雷达-LiDAR融合检测方法依赖预定义锚框，且常用卷积/Transformer融合与简单标签分配策略难以充分捕捉特征交互并缺乏一致的回归约束。为此，提出RaLiBEV，一种基于鸟瞰图（BEV）的锚框无关目标检测系统：LiDAR点云经PointPillars柱化得到BEV表示，雷达距离-方位热图经离散BEV采样，两分支独立提取特征后通过密集查询图交互式Transformer融合模块（DQMITBF）进行对称交互融合，再由YOLOv4骨干和多尺度无锚检测头解码2D旋转框；同时提出高斯区域一致热图与IoU代价正样本分配策略（GACHIPS），使分类与回归由同一正样本控制，缓解训练与推理不一致。核心机制可表示为高斯分布监督信号 G(x|μ,Σ)=1/(2π^(d/2)|Σ|^(1/2))·exp(-1/2·(x-μ)^T Σ^{-1}(x-μ))，用于生成热图标签并选择候选锚点。实验采用Oxford Radar RobotCar（ORR）数据集，包含8862帧（7071训练/1791测试），通过模拟雾天评估；在“Clear+Foggy”训练下，RaLiBEV在IoU=0.8时对“Clear”和“Foggy”测试的平均精度分别比最先进方法提升13.1%和19.0%，在IoU=0.5/0.65/0.8下均达到最高AP。进一步方向包括采集真实恶劣天气数据、引入稀疏可变形查询降低计算量、动态标签分配、多类别扩展和闭环测试；实际应用需注意模拟雾与真实雾的差距以及LiDAR在雾天噪声增大、雷达角分辨率低等约束。\n\n### 关键要点\n\n- RaLiBEV将LiDAR点云通过PointPillars柱化为320×320×9的BEV表示，并将雷达距离-方位热图转换为320×320×1的BEV输入，两分支经3层卷积块后各得到320×320×16的特征。\n- 提出的DQMITBF使用与特征图同尺寸的可学习密集BEV查询，分别以LiDAR和雷达特征作为键/值进行对称交互融合，融合输出为320×320×32的特征图。\n- 提出的GACHIPS标签分配策略在GAHIPS基础上，让选中的正样本同时控制分类损失和回归损失，从而统一训练与推理阶段的最优点选择。\n- 在ORR数据集上，RaLiBEV采用8862帧数据，其中7071帧用于训练、1791帧用于测试，雾天数据通过MVDNet的DEF方法以0.05雾概率模拟生成。\n- 消融实验显示，GACHIPS在IoU=0.5/0.65/0.8下分别达到97.4%、96.7%和93.9%的AP，优于GAHIPS的96.8%、96.7%和92.0%。\n- 融合模块消融表明，完整DQMITBF在IoU=0.8下AP为94.8%，而去掉BEV Query后降至84.5%，去掉Transformer仅简单拼接时降至82.0%，验证了对称交互与密集查询的作用。\n- 在“Clear+Foggy”训练、“Foggy”测试下，RaLiBEV with DQMITBF在IoU=0.5/0.65/0.8的AP为97.9%、97.8%、94.1%，比ST-MVDNet++分别提升4.4%、7.8%和19.0%。\n- 与仅用LiDAR的RaLiBEV LiDAR-Only相比，雷达+LiDAR融合在“Clear+Foggy”训练、“Foggy”测试IoU=0.8下AP从74.4%提升到93.7%（GACHIPS）或94.1%（DQMITBF），证明多模态融合在雾天中的增益。\n- 在仅用“Clear”数据训练后测试“Foggy”时，GACHIPS的AP@IoU=0.8为73.8%，DQMITBF为70.6%，均明显低于包含雾天训练数据的93.7%/94.1%，说明训练数据需覆盖多样天气条件。\n- 可视化特征图显示，晴天下BEV查询对LiDAR特征的注意力权重较大，雾天则对雷达特征的权重增大，说明DQMITBF能根据天气自适应调整模态关注。\n- 当前方法在模拟雾天ORR数据集上表现优异，但真实恶劣天气更复杂，需采集和标注真实数据；未来可采用稀疏可变形Transformer查询降低密集BEV查询的计算开销。"
}