{
  "paper_id": "ccd260c5e40ba944f94431eee1e0c70e",
  "paper_title": "Cross-Modal Analysis of Human Detection for Robotics: An Industrial Case Study",
  "paper_authors": [
    "Timm Linder",
    "Narunas Vaskevicius",
    "Robert Schirmer",
    "Kai O. Arras"
  ],
  "publication_date": "",
  "publication_venue": "2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
  "abstract": "Advances in sensing and learning algorithms have led to increasingly mature solutions for human detection by robots, particularly in selected use-cases such as pedestrian detection for self-driving cars or close-range person detection in consumer settings. Despite this progress, the simple question which sensor-algorithm combination is best suited for a person detection task at hand? remains hard to answer. In this paper, we tackle this issue by conducting a systematic cross-modal analysis of sensor-algorithm combinations typically used in robotics. We compare the performance of state-of-the-art person detectors for 2D range data, 3D lidar, and RGB-D data as well as selected combinations thereof in a challenging industrial use-case. We further address the related problems of data scarcity in the industrial target domain, and that recent research on human detection in 3D point clouds has mostly focused on autonomous driving scenarios. To leverage these methodological advances for robotics applications, we utilize a simple, yet effective multi-sensor transfer learning strategy by extending a strong image-based RGB-D detector to provide cross-modal supervision for lidar detectors in the form of weak 3D bounding box labels. Our results show a large variance among the different approaches in terms of detection performance, generalization, frame rates and computational requirements. As our use-case contains difficulties representative for a wide range of service robot applications, we believe that these results point to relevant open challenges for further research and provide valuable support to practitioners for the design of their robot system.",
  "keywords": [],
  "paper_institutions": [],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:57.509512Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文针对机器人领域中“哪种传感器-算法组合最适合特定的人员检测任务”这一难题，开展了跨模态的系统性分析。研究背景是现有人员检测技术在自动驾驶和消费场景中已较成熟，但工业场景面临数据稀缺和领域差距问题，且3D点云中的人员检测研究多聚焦于自动驾驶。本文在具有挑战性的工业用例（工厂仓库）中，比较了基于2D激光、3D激光和RGB-D传感器的多种最先进人员检测器及其组合的性能。为解决数据稀缺，提出了一种简单的多传感器迁移学习策略：利用扩展的RGB-D YOLO检测器（可回归3D有向边界框）为激光雷达检测器提供弱监督标签。实验在自建的工业数据集上进行，包括两个代表性序列（动态场景和存储室场景），采用AP和peak-F1作为评估指标。结果显示各方法在检测性能、泛化能力、帧率和计算需求上差异很大；RGB-D YOLO表现最佳，迁移学习显著提升了3D激光雷达检测器在目标域上的性能。研究发现，2D激光方法在最不拥挤的序列上可超越部分3D方法，而3D激光检测器存在较大领域差距但可通过迁移学习缓解。结果为机器人系统设计提供了实践指导，并指出了未来研究方向，如更深入的融合策略、专用数据集构建等。核心机制在于利用合成数据训练的RGB-D教师网络生成弱3D框标签，通过外参校准直接迁移到激光雷达点云，避免了手动标注。\n### 关键要点\n- 本文首次系统比较了2D激光、3D激光和RGB-D三种模态下多种人物检测器在工业环境中的性能，覆盖深度学习方法与经典机器学习基线。\n- 结果表明，传感器-算法组合的性能差异显著，RGB-D YOLO在所有测试序列中取得最高AP和peak-F1，其3D定位能力完全从合成数据学习。\n- 2D激光检测器中，深度学习方法（如DR-SPAAM）在多数场景优于经典方法，但在存储室场景中，经典方法Leigh et al.的性能接近深度学习方法，适合资源受限的机器人。\n- 3D激光检测器在汽车数据集（KITTI）上训练后存在明显的领域差距，尤其在拥挤室内场景中性能下降，但通过本文提出的迁移学习策略可大幅改善。\n- 本文扩展了RGB-D YOLO，使其不仅能预测人体质心，还能回归3D有向边界框，且该扩展仅使用合成数据训练，无需真实3D标注。\n- 迁移学习过程中，利用RGB-D检测器生成的弱3D框标签作为监督信号训练3D激光检测器，避免了手动标注，且该方法在AP上比训练于KITTI的模型提升了高达11.5%。\n- 实验数据集包含55小时以上记录，但评估选用两个共3分钟的代表性序列，涵盖高密度动态场景和狭窄存储室场景，具有典型服务机器人应用的困难度。\n- 评估时以3D质心关联为基准，距离阈值0.5米，仅评估Kinect v2视场内的检测，以保证不同模态比较的公平性。\n- 检测融合实验表明，简单最近邻融合可提高召回率但会降低精度，更复杂的投票或跟踪融合策略值得探索。\n- 隐私和成本考虑方面，3D激光雷达在需要隐私的应用中比RGB-D相机更有优势，但性能略低；低成本2D激光加经典方法适合预算敏感的简单场景。\n- 未来工作包括引入JackRabbit数据集、下一代传感器（如Azure Kinect）和更深度的检测融合研究，以提升泛化并降低部署差距。"
}