{
  "paper_id": "ea1a1cf82fd142a33cf98683697d64ab",
  "paper_title": "Uplifting Range-View-based 3D Semantic Segmentation in Real-Time with Multi-Sensor Fusion",
  "paper_authors": [
    "Shiqi Tan",
    "Hamidreza Fazlali",
    "Yixuan Xu",
    "Yuan Ren",
    "Bingbing Liu"
  ],
  "publication_date": "",
  "publication_venue": "2024 IEEE International Conference on Robotics and Automation (ICRA)",
  "abstract": "Range-View(RV)-based 3D point cloud segmentation is widely adopted due to its compact data form. However, RV-based methods fall short in providing robust segmentation for the occluded points and suffer from distortion of projected RGB images due to the sparse nature of 3D point clouds. To alleviate these problems, we propose a new LiDAR and Camera Range-view-based 3D point cloud semantic segmentation method (LaCRange). Specifically, a distortion-compensating knowledge distillation (DCKD) strategy is designed to remedy the adverse effect of RV projection of RGB images. Moreover, a context-based feature fusion module is introduced for robust and preservative sensor fusion. Finally, in order to address the limited resolution of RV and its insufficiency of 3D topology, a new point refinement scheme is devised for proper aggregation of features in 2D and augmentation of point features in 3D. We evaluated the proposed method on large-scale autonomous driving datasets i.e. SemanticKITTI and nuScenes. In addition to being real-time, the proposed method achieves state-of-the-art results on nuScenes benchmark.",
  "keywords": [
    "3D语义分割",
    "距离视图(Range View)",
    "多传感器融合",
    "知识蒸馏",
    "LiDAR点云",
    "自动驾驶",
    "实时分割"
  ],
  "paper_institutions": [
    "Huawei Noah's Ark Lab"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:36:43.633207Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n距离视图(Range-View, RV)的3D点云语义分割因数据表示紧凑、延迟低而被广泛应用于自动驾驶感知系统，但存在两个关键问题：对遮挡点无法提供稳健的分割，且RGB图像投影到RV后因点云稀疏性导致外观信息严重失真。先前工作中，RGBAL采用RV投影进行早期/中期融合但丢失大量外观信息，PMF利用透视投影保留外观信息却因需处理多路相机图像而计算开销巨大。为此，本文提出LiDAR与相机RV融合的实时分割方法LaCRange，其核心机制包括：(1)失真补偿知识蒸馏(DCKD)，在原始无失真RGB图像上预训练的教师编码器通过蒸馏损失 L_d = (1/K)Σ_{k=1}^{K}||T_k - S_k|| 引导处理RV-RGB图像的轻量学生编码器，使其恢复连贯的纹理与外观信息；(2)基于上下文的特征融合模块(CFF)，通过空间/通道维度的平均与最大池化生成全局上下文，并经模态查询(MLU，可用CBAM或交叉注意力实现)检索各模态特有信息以补充初始融合特征；(3)可移植的点精化方案，包含RV平面上基于语义概率与距离-反射率双分支动态置信软选择(δ = [1-exp(-λ(1-Φ))]δ^{R²} + [exp(-λ(1-Φ))]δ^S)的SR²FA聚合模块，以及利用粗到细多尺度柱体体素化注入3D邻域上下文的3D-NAFA增强模块，用于修正2D到3D反投影错误。方法基于PyTorch实现，教师-学生编码器默认采用ResNet18(学生通道数减半)，教师分别在nuImages和KITTI-STEP上预训练，并在大规模自动驾驶数据集SemanticKITTI(64线LiDAR、90°前视相机)与nuScenes(32线LiDAR、6个360°环绕相机)上验证，选用mIoU作为评价指标。结果显示，LaCRange在SemanticKITTI验证集上取得64.1 mIoU(超过PMF的63.9)，在nuScenes测试集上取得75.3 mIoU，与PMF-ResNet34持平但速度显著更快(50.1 ms对150.5 ms)，因为仅需处理单张RV图像即可覆盖360°视场；消融实验证明3D-NAFA贡献+4.5%、CFF贡献+1.0%、DCKD贡献+0.7%的mIoU提升，验证了各创新点的有效性。未来可进一步探索RV方法在遮挡点与稀疏点云上的性能提升及融合策略与计算成本的平衡。在真实自动驾驶场景中，方法须满足实时性约束并应对多相机覆盖、恶劣天气与光照变化，而点精化模块可作为即插即用组件附加于其他RV方法以普遍提升性能。\n\n### 关键要点\n\n- 问题背景：RV方法因紧凑表示而高效，但对遮挡点分割不稳健，且RGB图像经RV投影后因3D点云稀疏性产生外观信息失真，同时kNN后处理导致投影点与全点云分割性能存在差距。\n- DCKD策略：在原始RGB图像上预训练的教师编码器仅在训练阶段通过多尺度特征对齐(蒸馏损失 L_d = (1/K)Σ||T_k - S_k||)引导学生编码器，学生虽接收失真的RV-RGB图像仍能恢复连贯纹理信息，且不引入额外推理延迟。\n- CFF融合模块：相机与LiDAR特征先经ResBlock拼接完成初始融合，再通过空间/通道池化获得全局上下文，由模态查询(MLU)从各模态检索融合前的独特信息补充初始融合特征，实现保留性传感器融合。\n- SR²FA模块：在RV平面上基于语义概率绝对差与距离-反射率绝对差两次选取近邻，经PFE与Softmax加权聚合后，通过动态置信度软选择公式 δ = [1-exp(-λ(1-Φ))]δ^{R²} + [exp(-λ(1-Φ))]δ^S 融合两分支特征。\n- 3D-NAFA模块：对每个3D点考虑其在RV上投影位置的K个近邻，利用多尺度粗到细柱体体素化为点特征注入3D邻域上下文，可修正2D到3D反投影后物体边界处的标签错误。\n- 实验设置：在SemanticKITTI(64线LiDAR、90°前视相机)和nuScenes(32线LiDAR、6个360°相机、1000个多城市多天气场景)上评估，PyTorch实现，教师-学生编码器基于ResNet18，教师分别在nuImages和KITTI-STEP上预训练，评价指标为mIoU。\n- 主要结果：SemanticKITTI验证集上取得64.1 mIoU，超过PMF的63.9，且在bicycle、bicyclist、pole、traffic-sign等对RV失真敏感的小目标类别上表现相当或更优；nuScenes测试集上取得75.3 mIoU，与PMF-ResNet34持平但速度快约3倍(50.1 ms对150.5 ms)。\n- 消融证据：3D-NAFA带来+4.5% mIoU提升(最大增益)，CFF带来+1.0%，DCKD带来+0.7%；SR²FA在SemanticKITTI上仅+0.1%但在nuScenes验证集上+0.6%，因32线LiDAR点云更稀疏而更受益于邻域信息聚合。\n- MLU策略对比：CBAM式查询+0.7% mIoU(29.1 ms)，交叉注意力式+0.5%(45.5 ms)，二者组合(2 CBAM + 2 Cross Attention)+1.0%(31.1 ms)，兼顾精度与内存/速度效率。\n- 体素网格设计：较粗的体素网格因更大邻域提供交互信息纠正错误标签而获得更好mIoU，多尺度网格优于单尺度且推理时间仅略增(如[30,3,2]+[60,6,4]组合达64.0 mIoU/43.8 ms)。\n- 通用性验证：点精化模块微调60个epoch后附加到SqueezeSegV3和SalsaNext上分别提升+4.4%和+5.3% mIoU，证明其作为即插即用方案在不同RV分割方法间的普适有效性。\n- 适用约束：方法面向自动驾驶实时语义分割需求；SemanticKITTI评估仅限有RGB对应的有色点(前视90°相机限制)；信息保留与计算成本的权衡是相机-LiDAR融合的关键挑战，PMF类多相机方案在实时场景下不可行。"
}