{
  "paper_id": "dc588a572189d4be968e2f5f04f96a2c",
  "paper_title": "Deep Learning on 3D Semantic Segmentation: A Detailed Review",
  "paper_authors": [
    "Thodoris Betsas",
    "Andreas Georgopoulos",
    "Anastasios Doulamis",
    "Pierre Grussenmeyer"
  ],
  "publication_date": "2024",
  "publication_venue": "Remote Sensing",
  "abstract": "In this paper an exhaustive review and comprehensive analysis of recent and former deep learning methods in 3D Semantic Segmentation (3DSS) is presented. In the related literature, the taxonomy scheme used for the classification of the 3DSS deep learning methods is ambiguous. Based on the taxonomy schemes of 9 existing review papers, a new taxonomy scheme of the 3DSS deep learning methods is proposed, aiming to standardize it and improve the comparability and clarity across related studies. Furthermore, an extensive overview of the available 3DSS indoor and outdoor datasets is provided along with their links. The core part of the review is the detailed presentation of recent and former 3DSS deep learning methods and their classification using the proposed taxonomy scheme along with their GitHub repositories. Additionally, a brief but informative analysis of the evaluation metrics and loss functions used in 3DSS is included. Finally, a fruitful discussion of the examined 3DSS methods and datasets, is presented to foster new research directions and applications in the field of 3DSS. Supplementary, to this review a GitHub repository is provided (https://github.com/thobet/Deep-Learning-on-3D-Semantic-Segmentation-a-Detailed-Review) including a quick classification of over 400 3DSS methods, using the proposed taxonomy scheme.",
  "keywords": [
    "3D Semantic Segmentation",
    "Point Clouds",
    "Deep Learning",
    "Review"
  ],
  "paper_institutions": [
    "National Technical University of Athens (NTUA), School of Rural Surveying and Geoinformatics Engineering, Lab. of Photogrammetry",
    "Université de Strasbourg, INSA Strasbourg, CNRS, ICube Laboratory UMR 7357, Photogrammetry and Geomatics Group"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:25:59.636194Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n本文针对三维语义分割(3DSS)领域深度学习方法分类体系混乱、跨研究可比性差的问题进行了详尽综述。过去十年间大量综述论文提出了各不相同的3DSS方法分类命名,导致术语混淆与方法归类不一致;作者在分析9篇已有综述论文分类方案的基础上,提出了一套统一的3DSS深度学习分类体系,将方法划分为基于点、基于降维、基于离散化、基于图和混合方法五大类及各自子类,旨在标准化分类并提升研究间的清晰度与可比性。文中系统梳理了40余个室内外基准数据集(如S3DIS、ScanNet、Matterport3D、SemanticKITTI、Semantic3D、nuScenes、SensatUrban等),室内数据多由RGB-D相机采集、室外数据多由LiDAR采集,这些数据集因覆盖不同传感器类型、场景规模与类别数量而被选用,用以验证各类方法在真实与合成环境下的泛化能力。评估指标以交并比为核心,其计算机制为 IoU = TP/(TP+FP+FN),mIoU = (1/#c)Σ IoU,辅以总体精度(OA)、平均精度、精确率、召回率与F1分数;损失函数部分涵盖类别交叉熵、加权交叉熵、地理感知各向异性损失与Lovasz-SoftMax等。综述核心是对各类代表性方法(PointNet/PointNet++、RandLA-Net、KPConv、Point Transformer系列、SqueezeSeg系列、RangeNet++、SalsaNext、OctNet、SPLATNet、MinkowskiNet、DGCNN、DeepGCN、PolarNet、Cylinder3D、2DPASS、PTv3等)的详细机制分析与性能对比,结果显示方法持续演进,如PointNet在S3DIS上mIoU为47.6%、PointNet++为54.5%、RandLA-Net为70.0%、Point Transformer为73.5%、PTv3达80.81%,验证了局部特征聚合、注意力机制、序列化与点-体粒融合等创新的有效性。讨论部分指出采样策略效率、邻域定义、局部/全局特征平衡、标签重投影、类别不平衡、数据集地理与模态多样性不足及监督学习的数据饥渴等开放问题,并建议探索半监督/弱监督/零样本学习、多模态融合、时空数据集与面向应用的数据集构建等方向,为自动驾驶、城市测绘、文化遗产保护等实际场景中算法选型与部署(如实时性、轻量化、大规模约束)提供了重要参考,同时配套GitHub仓库提供了400余种3DSS方法的快速分类资源。\n\n### 关键要点\n\n- 本文基于9篇已有综述论文的分类方案分析,提出了统一的3DSS深度学习方法分类体系,包括基于点、基于降维、基于离散化、基于图和混合方法五大类别及各自的子类划分。\n- 基于点的方法直接以原始点云为输入提取特征,无需构建中间表示、保留详细几何信息,但邻域定义(如kNN、Ball Query)与采样方法(如FPS)计算耗时,是大规模点云应用的主要瓶颈。\n- 降维方法(如SqueezeSeg系列、RangeNet++、SalsaNext、GFNet)将点云投影为2D图像后利用成熟的2D分割技术,推理速度快,但投影造成信息损失与标签重投影问题,常用kNN或KPConv后处理缓解。\n- 离散化方法(如OctNet、SPLATNet、MinkowskiNet)将点云转换为体素或高维格子表示以应用3D卷积,稀疏结构(八叉树、哈希表、稀疏张量)可显著降低计算与内存开销,但量化带来信息损失。\n- 图方法(如DGCNN、DeepGCN)通过图的节点和边提取特征,能保留局部几何并捕获复杂形状,但图构建耗时且受梯度消失限制,模型通常较浅,DeepGCN借助残差/密集连接与膨胀卷积使GCN得以加深。\n- 混合方法(如PVCNN、SPVConv、Cylinder3D、2DPASS、PTv3)结合多类技术优势,例如点-体素双分支同时捕获细粒度细节与内存局部性,PTv3通过点云序列化与Patch注意力机制利用规模化原理取得领先性能。\n- 文中汇总了40余个室内外基准数据集及其链接,室内数据集(如ScanNet、Matterport3D、SUN RGB-D)多由RGB-D相机采集,室外数据集(如SemanticKITTI、nuScenes、SensatUrban、Toronto-3D)多由LiDAR或摄影测量获取,并给出类别数、点数、传感器类型等特性。\n- 评估指标以IoU/mIoU最为常用,辅以OA、mAcc、精确率、召回率和F1分数,均基于混淆矩阵(TP、TN、FP、FN)计算;损失函数涵盖类别交叉熵、加权交叉熵、地理感知各向异性损失和Lovasz-SoftMax等,用于处理类别不平衡与细节恢复。\n- 性能对比显示方法持续演进:PointNet在S3DIS上mIoU为47.6%,PointNet++为54.5%,RandLA-Net为70.0%,Point Transformer为73.5%,PTv3达80.81%;在SemanticKITTI上从SqueezeSeg的30.8%提升至PTv3的75.5%。\n- 开放挑战包括点采样与邻域定义效率、局部/全局特征提取平衡、类别不平衡与类内几何差异、数据集地理多样性与多模态性不足,以及监督学习依赖大量精细标注的数据饥渴问题。\n- 未来方向建议探索半监督、弱监督、少样本/零样本与无监督学习以缓解标注成本,推进图像与点云的多模态融合、时空数据集构建、面向特定应用(建筑、文化遗产、气候变化)的数据集,以及新的体素形状与投影方式设计。\n- 配套GitHub仓库(https://github.com/thobet/Deep-Learning-on-3D-Semantic-Segmentation-a-Detailed-Review)提供了400余种3DSS方法的快速分类,同时涵盖传统与深度学习方法,是领域研究的重要资源。"
}