{
  "paper_id": "9b6255ec5ad7ac7a31fba1a672796d95",
  "paper_title": "M2CNet: LiDAR 3D Semantic Segmentation Based on Multi-Level Multi-View Cross-Attention Fusion for Autonomous Vehicles",
  "paper_authors": [
    "Mu Zhou",
    "Rui Xiang",
    "Nan Du",
    "Wei He",
    "Zhao Jiang"
  ],
  "publication_date": "2025",
  "publication_venue": "IEEE Transactions on Vehicular Technology",
  "abstract": "Light Detection and Ranging (LiDAR) point cloud semantic segmentation is an essential task for autonomous vehicles, enabling precise environment perception for safe navigation. Existing point-based and voxel-based methods face limitations related to computational costs and quantization errors, especially dealing with large-scale point clouds. In contrast, 2D projection methods, including Range View and Bird's Eye View, have gained popularity for their balance of real-time performance and accuracy. However, most projection approaches overlook critical issues arising from projection, including 1) inter-class occlusion, 2) boundary-blurring, and 3) inadequate multi-scale feature extraction. To address these issues, we propose a Multi-level Multi-view Cross-attention Fusion Network (M2CNet), which fully leverages complementary information across different views. First, we employ parallel encoder-decoder networks to extract semantic information within a multi-task segmentation framework. Given the high correlation between views, we incorporate a Multi-view Feature Cross Fusion Module (MFCFM) to facilitate cross-view feature association, capturing potential complementarities to mitigate information loss from single-view projections. Then, we place a Multi-cascaded Enhanced Semantic Context Atrous Spatial Pyramid Pooling (MESCASPP) module between the encoder and decoder to capture multi-scale contextual information, enhancing pixel relationships effectively. Finally, the Kernel Point Convolution (KPCConv) is employed to refine point-wise features at the output level. The proposed method is evaluated on urban datasets SemanticKITTI and nuScenes (achieving mIoU scores of 65.7% and 78.9% respectively), the off-road dataset RELIS-3D (44.6%), and a real-world test set. Experimental results demonstrate that our method surpasses representative methods in both effectiveness and scene adaptability.",
  "keywords": [
    "Autonomous driving",
    "deep learning",
    "multi-view fusion",
    "point cloud",
    "semantic segmentation"
  ],
  "paper_institutions": [
    "Chongqing University of Posts and Telecommunications",
    "Chongqing University",
    "Zhejiang University of Technology"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:42:37.592566Z",
  "template_tag": "simple",
  "summary": "### 摘要\n针对自动驾驶中LiDAR点云语义分割任务，现有基于点（如PointNet、KPConv）与基于体素（如VoxelNet、Cylinder3D）的方法分别面临计算开销大、难以处理大规模点云和量化误差的问题，而流行的2D投影方法（距离视图RV与鸟瞰图BEV）虽兼顾实时性与精度，却普遍存在类间遮挡、边界模糊及多尺度特征提取不足三大缺陷；已有研究（如AMVNet、MPF、GFNet等多视图融合方法）多在输入或输出阶段融合，忽视了中层特征交互与深度几何对齐。为此本文提出多层级多视图交叉注意力融合网络M2CNet：采用基于SalsaNext与ResNet34的并行编码器-解码器双流结构，分别处理RV流（球面投影、5通道编码、RCCSA注意力）与BEV流（极坐标投影、9通道网格编码）；设计多视图特征交叉融合模块MFCFM，通过点级坐标矩阵与像素级索引矩阵构建跨视图关联矩阵实现特征网格对齐，并利用窗口与移位窗口多头交叉注意力完成深度融合，其核心机制为 head_i(Q_i,K_i,V_i)=Softmax(Q_iK_i^T/√d_k)V_i；在编码器与解码器之间嵌入MESCASPP模块，以四条膨胀率分别为3、6、12、18的级联3×3膨胀卷积分支结合ESCM全局-局部注意力捕获多尺度上下文；输出级采用KPConv核点卷积后处理以缓解逆投影误差；总损失为 Loss=αLoss_F+βLoss_R+λLoss_B（权重2、1、1），各项由加权交叉熵与Lovasz-Softmax组成。实验在城市场景数据集SemanticKITTI（43,551帧、19类）、nuScenes（40,157帧、16类）、越野数据集RELLIS-3D（13,556帧、14类）及自采2,000帧校园实测集（Velodyne VLP-16）上开展，选择这些数据集是为了覆盖结构化城市道路、大规模多场景、非结构化越野及真实稀疏点云等多样条件以验证场景适应性；实现基于PyTorch与两块NVIDIA A40 GPU，RV尺寸64×2048或32×1024、BEV尺寸480×320，训练150轮、批大小4、SGD优化器（初始学习率0.001余弦退火）。以mIoU与FPS为指标，M2CNet在SemanticKITTI测试集达65.7%、nuScenes验证集78.9%、RELLIS-3D测试集44.6%，优于AMVNet（+0.4%）与GFNet（+0.3%）等多视图方法，在40–50米远距上较SalsaNext对person、bicycle、pole、car、road分别提升+16.9%、+8.7%、+14.2%、+19.0%、+11.1%，推理速度11–18 Hz满足LiDAR约10 Hz采集频率，这些结果支撑了跨视图融合（MFCFM消融+3.0%）与多尺度注意力（MESCASPP较标准ASPP提升+3.5%）两项创新的有效性。未来方向包括网络轻量化以提升实时性、更高效的融合策略、类间相似性建模与多帧时序关联机制；实际部署约束在于远距离点云稀疏、相似类别（如可行驶面/人行道/地形）混淆及遮挡导致的点缺失仍是挑战。\n\n### 关键要点\n- M2CNet是面向自动驾驶LiDAR点云语义分割的双流多层级多视图交叉注意力融合网络，旨在克服单视图投影的类间遮挡、边界模糊与多尺度特征提取不足问题。\n- RV流基于SalsaNext骨干采用球面投影并编码(x,y,z,r,d)五通道特征，配合RCCSA残差坐标通道空间注意力增强方向位置感知与长程空间依赖；BEV流基于ResNet34采用极坐标投影，每网格编码9维特征以缓解笛卡尔坐标下的长尾分布。\n- MFCFM利用点级坐标矩阵Q_{I→P}与像素级索引矩阵Q_{P→Î}构建跨视图关联矩阵Q_{I→Î}，经翻转、插值与grid_sample重采样实现RV与BEV特征网格对齐。\n- MFCFM采用窗口（W-MHCA）与移位窗口（SW-MHCA）多头交叉注意力双阶段融合，注意力计算为head_i(Q_i,K_i,V_i)=Softmax(Q_iK_i^T/√d_k)V_i，将注意力限制在非重叠局部窗口以兼顾效率。\n- MESCASPP由四条并行分支组成，每条级联两个3×3膨胀卷积（膨胀率3、6、12、18），并嵌入ESCM注意力（F_out=F_in⊙F_0⊙F_1）以自适应加权多尺度特征、捕获密集像素上下文。\n- 输出级通过R2P/B2P逆投影与双线性插值将2D解码特征恢复为3D点特征，再用KPConv核点卷积细化逐点特征以缓解投影信息损失导致的重投影误差。\n- 训练采用加权交叉熵（α_c=1/√f_c）与Lovasz-Softmax联合损失，总损失Loss=αLoss_F+βLoss_R+λLoss_B，权重设为2、1、1，消融显示α=2时在SemanticKITTI验证集取得最佳63.0% mIoU。\n- 实验基于PyTorch、两块NVIDIA A40 GPU（各48 GB），RV尺寸64×2048（SemanticKITTI/RELLIS-3D）或32×1024（nuScenes）、BEV尺寸480×320，训练150轮、批大小4、SGD（动量0.9、权重衰减0.0001、初始学习率0.001余弦退火），并采用随机丢点、缩放、翻转、旋转等数据增强。\n- 在SemanticKITTI测试集、nuScenes验证集与RELLIS-3D测试集上分别取得65.7%、78.9%、44.6%的mIoU，优于AMVNet与GFNet等多视图方法，但较Cylinder3D在SemanticKITTI上低2.1%（投影信息损失所致）。\n- 距离鲁棒性方面，在40–50米范围较SalsaNext对person、bicycle、pole分别提升+16.9%、+8.7%、+14.2%，对car与road分别提升+19.0%与+11.1%，验证了远距离稀疏目标下的感知能力。\n- 推理效率为11–18 Hz，显著快于点式与体素方法（如Cylinder3D为7–9 Hz），虽低于单视图方法（如SalsaNext 28–41 Hz）但满足LiDAR约10 Hz的采集频率，兼顾精度与效率。\n- 局限与约束：相似类别（driveable/terrain/sidewalk）混淆与遮挡致点缺失仍具挑战，16线LiDAR实测稀疏点云下依赖多视图融合保持鲁棒性；未来将探索网络轻量化、更高效融合策略、类间相似性建模与多帧时序关联机制。"
}