{
  "paper_id": "4bac1eb608904a0396d57333e0ba65dd",
  "paper_title": "Multiview Fusion Driven 3-D Point Cloud Semantic Segmentation Based on Hierarchical Transformer",
  "paper_authors": [
    "Wang Xu",
    "Xu Li",
    "Peizhou Ni",
    "Xingxing Guang",
    "Hang Luo",
    "Xijun Zhao"
  ],
  "publication_date": "",
  "publication_venue": "IEEE Sensors Journal",
  "abstract": "Three-dimensional semantic segmentation is a key task of environment understanding in various outdoor scenes. Due to the sparsity and varying density of point clouds, it becomes challenging to obtain fine-grained segmentation results. Previous point-based and voxel-based methods suffer from the expensive computational cost. Recent 2-D projection-based methods, including range-view (RV), bird-eye-view (BEV), and multiview fusion methods, can run in real time, but the information loss during the projection leads to low accuracy. Also, we find that the occlusion and interlacing problems exist in single projection-based methods and most multiview fusion networks only focus on the output-level fusion. Considering the above issues, we propose a multilevel multiview fusion network using attention modules and hierarchical transformer, which ensures the effectiveness and efficiency mainly by the following three aspects: 1) the spatial-channel attention module (SCAM) integrates contextual information between points and learns differences of each channel's features; 2) the proposed geometry-based multiprojection fusion module (GMFM) achieves the geometric feature alignment between RV and BEV and fuses the features of the two views at both feature level and output level; and 3) we introduce KPConv to replace KNN, which can reduce the information loss during the postprocessing. Experiments are conducted on both structured and unstructured datasets, including urban dataset SemanticKITTI and off-road dataset Rellis3D. Our results achieve a better performance compared to other projection-based methods and are comparable with the state-of-the-art Cylinder3D.",
  "keywords": [
    "3-D point cloud",
    "multihead attention",
    "multiview fusion",
    "semantic segmentation"
  ],
  "paper_institutions": [
    "Southeast University",
    "China North Artificial Intelligence and Innovation Research Institute",
    "Collective Intelligence and Collaboration Laboratory (CIC)"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:38:49.769616Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n三维点云语义分割是无人驾驶环境感知的关键任务，但点云的稀疏性和密度不均使得获得细粒度分割结果极具挑战。基于点的方法和基于体素的方法计算成本高昂，而基于二维投影的方法（距离视图RV、鸟瞰视图BEV及多视图融合）虽能实时运行，投影过程中的信息损失却导致精度偏低；此外单投影方法存在遮挡与交错问题，多数多视图融合网络仅关注输出级融合而忽略特征级的对齐。为此，本文提出一种基于注意力模块与分层Transformer的多级多视图融合网络：空间-通道注意力模块（SCAM）整合点间上下文信息并学习各通道特征差异；基于几何的多投影融合模块（GMFM）以原始点云为中介计算几何变换矩阵 M_{B→R} 实现RV与BEV的特征对齐，再通过多头交叉注意力完成深度特征融合，其核心机制为 F_MHCA = Norm(F_RB + MHCA(F_RB, F_RV, F_RV)) 与 F_RVfused = Norm(F_MHCA + FFN(F_MHCA))；同时引入KPConv替代KNN以减少后处理反投影中的信息损失。实验选用结构化城市数据集SemanticKITTI与非结构化越野数据集RELLIS-3D以及自采100帧非结构化场景测试集，以同时验证方法在结构化与非结构化场景下的有效性和鲁棒性；实现上RV分支采用SalsaNext编码器-解码器主干、BEV分支采用极坐标划分与ResNet-34编码器，在四块NVIDIA RTX 3090 GPU上以批大小16训练50个epoch（约48小时），使用SGD优化器（初始学习率0.02）。以mIoU为评估指标，方法在SemanticKITTI上取得67.5% mIoU、12 FPS，较SalsaNext提升8.0%、较GFNet提升2.1%，与最先进的Cylinder3D（67.8%）相当且推理快107.1ms；在RELLIS-3D上取得44.5% mIoU、14 FPS，超过SalsaNext与GFNet，略低于Cylinder3D（1.1%）但快78.3ms；消融实验证明SCAM、GMFM与KPConv分别带来+1.2、+1.8、+0.8 mIoU提升，验证了各创新点的有效性。未来可进一步探索更高效的特征聚合方式与更广泛的场景适应性；实际部署中需在实时性与精度间权衡，投影固有的信息损失仍是与体素方法存在小幅精度差距的主要约束，尤其适用于自动驾驶等对推理速度要求严格的户外环境感知场景。\n\n### 关键要点\n\n- 提出多级多视图融合网络，在特征级和输出级同时融合RV与BEV，缓解单投影造成的信息损失，在投影类方法中达到先进性能\n- SCAM由空间注意力模块（SAM）和通道注意力模块（CAM）组成，整合点间上下文信息并学习通道特征差异，提升远程稀疏点的特征表达能力\n- GMFM通过以原始点云为中介计算几何变换矩阵实现RV与BEV特征对齐，再用多头交叉注意力（MHCA）完成两视图的深度特征融合\n- 输出级融合采用G2P模块（网格采样+KPConv）替代KNN进行反投影，减少后处理信息损失，最终经point head（MLP+最大池化+全连接层）得到点级预测\n- RV分支以SalsaNext编码器-解码器为主干，输入2048×64的五通道球面投影图；BEV分支采用极坐标划分、MLP与最大池化编码网格特征，并以ResNet-34为编码器\n- 实验涵盖结构化城市数据集SemanticKITTI、非结构化越野数据集RELLIS-3D及自采100帧非结构化场景测试集（Velodyne 32线LiDAR与单目相机同步采集）\n- 训练配置：4块NVIDIA RTX 3090 GPU、50个epoch、批大小16、SGD优化器（动量0.9、权重衰减0.001、初始学习率0.02），训练约48小时\n- SemanticKITTI上mIoU为67.5%、12 FPS，优于SalsaNext（+8.0%）和GFNet（+2.1%），与Cylinder3D（67.8%）相当且推理快107.1ms\n- RELLIS-3D上mIoU为44.5%、14 FPS，超过SalsaNext（+2.0%）和GFNet（+0.9%），略低于Cylinder3D（-1.1%）但推理快78.3ms\n- 消融实验：仅输出级融合的ensemble为62.4 mIoU，逐步加入SCAM、GMFM、KPConv后提升至66.2 mIoU，且在40m以外远处目标上优于单视图与ensemble方法\n- 损失函数为交叉熵与Lovász组合，对融合输出、BEV输出、RV输出三部分加权（α=1.0、β=0.5、γ=0.5）联合优化\n- 约束：投影过程仍不可避免信息损失，导致与Cylinder3D等体素方法存在小幅精度差距；远处稀疏点和垂直延展物体的分割仍是难点",
  "available_templates": [
    "deep_read",
    "simple"
  ]
}