{
  "paper_id": "4c65d16786ef063fb525dbebace3f896",
  "paper_title": "Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors",
  "paper_authors": [
    "Svetlana Seliunina",
    "Artem Otelepko",
    "Raphael Memmesheimer",
    "Sven Behnke"
  ],
  "publication_date": "",
  "publication_venue": "2025 IEEE/SICE International Symposium on System Integration (SII)",
  "abstract": "Robots need to perceive persons in their surroundings for safety and to interact with them. In this paper, we present a person segmentation and action classification approach that operates on 3D scans of hemisphere field of view LiDAR sensors. We recorded a data set with an Ouster OSDome-64 sensor consisting of scenes where persons perform three different actions and annotated it. We propose a method based on a MaskDINO model to detect and segment persons and to recognize their actions from combined spherical projected multi-channel representations of the LiDAR data with an additional positional encoding. Our approach demonstrates good performance for the person segmentation task and further performs well for the estimation of the person action states walking, waving, and sitting. An ablation study provides insights about the individual channel contributions for the person segmentation task. The trained models, code and dataset are made publicly available.",
  "keywords": [
    "LiDAR",
    "person segmentation",
    "action classification",
    "MaskDINO",
    "hemisphere field of view",
    "privacy-preserving"
  ],
  "paper_institutions": [],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:19.343961Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文针对半球视场LiDAR传感器在机器人感知中的需求，提出了一种基于MaskDINO模型的人员分割与动作分类方法。该方法将Ouster OSDome-64 LiDAR的多通道数据（距离、信号、反射率、近红外）投影为二维图像，并加入由点云坐标计算的位置编码，形成七通道输入，从而利用二维检测分割模型实现像素级人员分割。作者构建了一个包含442帧、标注了行走、挥手、坐下三种动作的新数据集，采用半自动标注（SegmentAnything）并进行人工修正。实验表明，引入位置编码和训练骨干网络可提升F1分数至0.97（分割）和0.94（动作分类加权平均）。消融研究显示距离和反射率通道贡献最大，且模型在具有更多光束的OSDome-128传感器上以11Hz实时运行，验证了其泛化能力和实际应用潜力。\n\n### 关键要点\n- 提出基于半球视场LiDAR的人员分割与动作分类方法，可直接用于机器人安全与人机交互。\n- 数据集采用Ouster OSDome-64传感器，包含442帧，标注了行走、挥手、坐下三种动作及像素级人员掩膜。\n- 将距离、信号、反射率、NIR四个通道投影至512×64的图像，并融合位置编码构成七通道输入。\n- 位置编码通过Ouster SDK从坐标恢复，包含方位角、仰角等几何信息，显著提升分割精度。\n- 方法基于MaskDINO，采用SwinL骨干，通过卷积编码层适应多通道输入，训练约35分钟/5000次迭代。\n- 训练时采用冻结或微调骨干网络，学习率1e-4时效果最佳，位置编码使分割F1从0.93提升至0.95。\n- 动作分类初始化自分割模型，三种动作的加权平均F1达到0.94（带位置编码），优于直接使用预训练模型。\n- 消融实验显示，去除距离通道时F1下降至0.66，去除反射率通道F1降至0.84，而NIR和信号通道贡献较小。\n- 模型在带有不同通道组合和位置编码时，性能差异表明卷积编码器可能需要改进以更好融合多通道信息。\n- 在线实验中，方法在Ouster OSDome-128（128束）上以11Hz运行，未做调整即泛化，适用于实时追踪等应用。\n- 数据集、代码和模型已开源，便于社区复现和进一步研究。"
}