{
  "paper_id": "0150fd9fc438b07005cbc431ff964408",
  "paper_title": "SemanticVoxels: Sequential Fusion for 3D Pedestrian Detection using LiDAR Point Cloud and Semantic Segmentation",
  "paper_authors": [
    "Juncong Fei",
    "Wenbo Chen",
    "Philipp Heidenreich",
    "Sascha Wirges",
    "Christoph Stiller"
  ],
  "publication_date": "",
  "publication_venue": "2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)",
  "abstract": "3D pedestrian detection is a challenging task in automated driving because pedestrians are relatively small, frequently occluded and easily confused with narrow vertical objects. LiDAR and camera are two commonly used sensor modalities for this task, which should provide complementary information. Unexpectedly, LiDAR-only detection methods tend to outperform multisensor fusion methods in public benchmarks. Recently, PointPainting has been presented to eliminate this performance drop by effectively fusing the output of a semantic segmentation network instead of the raw image information. In this paper, we propose a generalization of PointPainting to be able to apply fusion at different levels. After the semantic augmentation of the point cloud, we encode raw point data in pillars to get geometric features and semantic point data in voxels to get semantic features and fuse them in an effective way. Experimental results on the KITTI test set show that SemanticVoxels achieves state-of-the-art performance in both 3D and birds eye view pedestrian detection benchmarks. In particular, our approach demonstrates its strength in detecting challenging pedestrian cases and outperforms current state-of-the-art approaches.",
  "keywords": [],
  "paper_institutions": [],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:28.553195Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文针对自动驾驶中3D行人检测的难点，如行人体积小、易遮挡、易与窄垂直物体混淆等问题，提出了一种名为SemanticVoxels的序列融合方法。该方法推广了PointPainting，能够在不同层级进行融合：利用图像语义分割网络的输出对点云进行语义增强，再分别通过柱体（pillar）编码原始点云得到几何特征，通过体素（voxel）编码语义点数据得到语义特征，并在骨干网络的不同层级（早期、中期、晚期）融合这些特征。在KITTI数据集上，采用DeepLabv3+作为分割网络，检测网络遵循PointPillars设置，使用Adam优化器训练160轮。在KITTI测试集上，该方法在3D和BEV行人检测任务上均达到最先进水平，尤其在困难级别上表现突出，mAP分别达到44.20和52.05。在验证集上，早期融合相比PointPillars将3D mAP提升3.20，BEV mAP提升2.82，验证了语义信息的有效性。未来工作包括研究对分割质量的依赖、鲁棒性及实车测试。该方法适用于自动驾驶中的远距离遮挡行人检测等现实场景，其核心机制是利用语义分割的紧凑输出高效融合多模态信息。\n\n### 关键要点\n- SemanticVoxels推广了PointPainting，支持在原始数据级、中间特征级和最终特征级进行融合，而PointPainting仅限原始数据级。\n- 几何特征通过PointPillars风格编码，将点云分割为柱体，每个柱体用简化PointNet编码为64维特征。\n- 语义特征通过沿z轴进一步体素化，计算每个体素内语义概率的均值，再用1×1卷积聚合，输出8通道BEV表示。\n- 融合方案包括早期融合（输入前）、中期融合（第一块输出后）和晚期融合（最终特征图），均采用拼接操作。\n- 在KITTI测试集上，SemanticVoxels的3D mAP为44.20，BEV mAP为52.05，均优于对比方法，尤其在困难级别提升明显。\n- 相比PointPillars，早期融合在验证集上3D mAP提升3.20，BEV mAP提升2.82，表明语义特征对行人检测有效。\n- 语义信息来自DeepLabv3+在Cityscapes预训练并在KITTI语义分割数据微调的网络，输出四类概率（行人、骑车人、汽车、背景）。\n- 点云增强后维度从4变为8，原始点编码维度为9，检测网络使用与PointPillars相同的锚点尺寸（0.6,0.8,1.73米）。\n- 损失函数包含回归SmoothL1、方向正弦损失和焦点损失，总损失权重为β1=2.0、β2=0.2、β3=1.0。\n- 定性分析显示，SemanticVoxels能消除对窄垂直物体（如杆子、路牌）的误检，并成功检测严重遮挡的行人。",
  "available_templates": [
    "deep_read",
    "simple"
  ]
}