{
  "paper_id": "4ee20cb88a2c4dd8332202ae101fae77",
  "paper_title": "RangeSAM: On the Potential of Visual Foundation Models for Range-View represented LiDAR segmentation",
  "paper_authors": [
    "Paul Julius Kühn",
    "Duc Anh Nguyen",
    "Arjan Kuijper",
    "Saptarshi Neil Sinha"
  ],
  "publication_date": "",
  "publication_venue": "unknown",
  "abstract": "LiDAR point cloud segmentation is central to autonomous driving and 3D scene understanding. While voxel- and point-based methods dominate recent research due to their compatibility with deep architectures and ability to capture fine geometry, they often incur high computational cost, irregular memory access, and limited runtime efficiency due to scaling issues. In contrast, range-view methods, though relatively underexplored, can leverage mature 2D semantic segmentation techniques for fast and accurate predictions. Motivated by the rapid progress in Visual Foundation Models (VFMs) for captioning, zero-shot recognition, and multimodal tasks, we investigate whether SAM2, the current state-of-the-art VFM for segmentation tasks, can serve as a strong backbone for LiDAR point cloud segmentation in the range view representations. We present RangeSAM, to our knowledge, the first range-view framework that adapts SAM2 to 3D segmentation, coupling efficient 2D feature extraction with projection/back-projection to operate on point clouds. To optimize SAM2 for range-view representations, we implement several architectural modifications to the encoder: (1) a novel Stem module that emphasizes horizontal spatial dependencies inherent in LiDAR range images, (2) a customized configuration of Hiera Blocks tailored to the geometric properties of spherical projections, and (3) an adapted Window Attention mechanism in the encoder backbone specifically designed to capture the unique spatial patterns and discontinuities present in range-view pseudo-images. Our approach achieves competitive performance on SemanticKITTI while benefiting from the speed, scalability, and deployment simplicity of 2D-centric pipelines. This work highlights the viability of VFMs as general-purpose backbones for point cloud segmentation and opens a path toward unified, foundation-model-driven LiDAR segmentation.",
  "keywords": [
    "LiDAR point cloud segmentation",
    "range view",
    "SAM2",
    "visual foundation models",
    "SemanticKITTI",
    "autonomous driving",
    "semantic segmentation"
  ],
  "paper_institutions": [
    "Fraunhofer IGD"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:31:08.561522Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\nLiDAR点云语义分割是自动驾驶与3D场景理解的核心任务，而主流的体素与点方法虽性能强劲，却在大规模户外数据上面临高计算成本、不规则内存访问与运行效率受限的问题；相比之下，range-view方法将3D点云投影为稠密2D表示，可复用成熟的2D分割技术（如SqueezeSeg系列、RangeNet++、RangeViT、RangeFormer）实现更快推理，同时视觉基础模型（SAM/SAM2、DINO、CLIP）在零样本识别与分割上进展迅速。为此，本文提出RangeSAM——据作者所知首个将SAM2适配到range-view LiDAR分割的框架：通过球面投影 θ=arctan(y/x)、φ=arcsin(z/r)、r=√(x²+y²+z²) 将无序点云栅格化为64×2048的距离图像，再经定制Stem模块、(4,128)嵌入矩阵、非对称窗口注意力（8×64与16×128）的Hiera编码器与改进RFB解码器进行分割，最后用k=7的k-NN多数投票将标签传播回全分辨率点云，并以加权交叉熵、Dice、边界与IoU构成的复合损失（权重均为1）训练。实验选用SemanticKITTI（19,130训练扫描、19类）与nuScenes（28,130训练帧、16类，转换为SemanticKITTI格式）以验证城市驾驶场景下的方法可行性，在8块A100（有效batch 16）上以AdamW差异化学习率训练并在nuScenes上预训练60个epoch。以mIoU为指标，模型在SemanticKITTI验证集达60.9%、测试集达61.5%，其中SAM2-tiny（约63M参数）优于更大变体，range-view数据增强带来约10%的mIoU提升，而Cityscapes 2D迁移学习反而降低性能，说明多3D数据集训练优于2D预训练。未来方向包括优化RFB计算瓶颈以实现实时部署、扩充数据以改善摩托车、自行车、行人等长尾类别（IoU 29–47%）的表现；现实约束在于自动驾驶系统要求实时性，而当前RFB构成主要计算瓶颈，阻碍实时应用。\n\n### 关键要点\n\n- RangeSAM是首个将SAM2视觉基础模型适配到range-view LiDAR点云语义分割的框架，结合高效2D特征提取与投影/反投影操作直接处理点云。\n- 编码器基于预训练Hiera骨干（SAM2-tiny，四个阶段分别含1、2、7、2个Hiera块），全局注意力位于第5、7、9、11块以捕获长程空间依赖。\n- 针对距离图像固有的水平结构，采用非对称注意力窗口设计：第1、4阶段为8×64，第2、3阶段为16×128，经验上优于常规方形注意力窗口。\n- Stem模块通过线性变换、LayerNorm与GELU将(B,6,H,W)输入映射为(B,96,H,W)，并以新颖的(4,128)嵌入矩阵替换SAM2位置编码以增强水平空间敏感性。\n- 解码器采用改进的Receptive Field Block（以LayerNorm和GELU替代BatchNorm和ReLU），将多尺度特征通道标准化为256后拼接并逐步降维至类别数，辅以辅助分类头改善训练中的梯度流。\n- 后处理使用k=7的k-NN插值与多数投票，将预测标签从处理点传播到全分辨率点云，在效率与噪声鲁棒性之间取得平衡。\n- 训练目标为复合损失：加权交叉熵、Dice、边界与Jaccard/IoU损失的加权和，各权重λi均设为1，以应对类别不平衡并提升边界精度。\n- 数据集为SemanticKITTI（64线LiDAR，19类，19,130训练/4,071验证/20,351测试扫描）与nuScenes（32线LiDAR，16类，28,130训练/6,019验证帧，已统一转换为SemanticKITTI格式）。\n- 训练配置：8块A100 GPU（40GB），每GPU batch 2（有效batch 16），5 epoch线性warm-up加余弦退火，AdamW差异化学习率（Hiera骨干lr=0.0004、weight decay 0.001，其余lr=0.001），并在nuScenes上预训练60 epoch。\n- 结果：SemanticKITTI验证集mIoU 60.9%、测试集61.5%；SAM2-tiny（63.3M参数）优于Hiera-small（70.2M参数、60.5 mIoU），表明该任务中模型容量与性能不必然相关。\n- 消融研究显示range-view数据增强（mixing、union、shifting、copy-paste）使mIoU从55.0提升至61.5（约+10%），而Cityscapes 2D迁移学习使性能降至59.7，可能因SAM2的大规模图像预训练造成域失配。\n- 局限与约束：无卷积的Hiera骨干难以处理点云稀疏性、需RFB补偿，而RFB构成主要计算瓶颈、阻碍实时部署；摩托车、自行车、行人等稀有小目标IoU仅29–47%，需通过数据扩充与设计改进缩小差距。"
}