{
  "paper_id": "bf652419a060d00c0d8f4b458034c78d",
  "paper_title": "HPERL: 3D Human Pose Estimation from RGB and LiDAR",
  "paper_authors": [
    "Michael Fürst",
    "Shriya T. P. Gupta",
    "René Schuster",
    "Oliver Wasenmüller",
    "Didier Stricker"
  ],
  "publication_date": "",
  "publication_venue": "2020 25th International Conference on Pattern Recognition (ICPR)",
  "abstract": "In-the-wild human pose estimation has a huge potential for various fields, ranging from animation and action recognition to intention recognition and prediction for autonomous driving. The current state-of-the-art is focused only on RGB and RGB-D approaches for predicting the 3D human pose. However, not using precise LiDAR depth information limits the performance and leads to very inaccurate absolute pose estimation. With LiDAR sensors becoming more affordable and common on robots and autonomous vehicle setups, we propose an end-to-end architecture using RGB and LiDAR to predict the absolute 3D human pose with unprecedented precision. Additionally, we introduce a weakly-supervised approach to generate 3D predictions using 2D pose annotations from PedX. This allows for many new opportunities in the field of 3D human pose estimation.",
  "keywords": [
    "sensor fusion",
    "3D human pose estimation",
    "LiDAR",
    "RGB",
    "autonomous vehicles",
    "perception"
  ],
  "paper_institutions": [
    "DFKI - German Research Center for Artificial Intelligence",
    "Birla Institute of Technology and Science (BITS) Pilani",
    "University of Applied Sciences Mannheim",
    "University of Kaiserslautern - TUK"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:42.985717Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文针对野外场景（如自动驾驶）中的3D人体姿态估计问题，提出了一种融合RGB图像和LiDAR点云的端到端多人体姿态估计架构HPERL。现有方法主要依赖RGB或RGB-D，缺乏精确的深度信息，导致绝对位置估计不准确。HPERL采用两阶段结构：第一阶段基于AVOD的RPN，将LiDAR转换为六通道BEV表示，与RGB特征通过RoI Align提取并拼接；第二阶段借鉴LCR-Net，利用8个锚点姿态进行分类和回归，同时预测2D和3D姿态。由于PedX数据集仅提供2D姿态标注，该方法采用弱监督训练，将3D预测投影到2D计算损失，并引入CDE和XYE指标评估3D绝对定位精度。在PedX上的实验表明，相比RGB基线，HPERL将2D MPJPE从87.76px降至45.66px，PCKh@0.5从65.02%提升至70.08%，CDE从4.88m降至0.95m，XYE从1.44m降至0.39m，验证了LiDAR融合对绝对3D定位的显著提升。消融实验显示LiDAR输入是最大贡献因素，而预训练、融合方式等需针对性调整。未来应构建更多含3D标注的野外数据集以扩大评估，该方法可应用于自动驾驶中的行人意图识别等需要高绝对精度场景，但仍受限于数据稀缺。\n\n### 关键要点\n- HPERL是首个融合RGB和LiDAR进行3D人体姿态估计的端到端多人体架构，解决了仅使用RGB时的深度模糊问题。\n- 网络结构采用两阶段：第一阶段基于AVOD的RPN生成3D区域提议，第二阶段借鉴LCR-Net进行姿态分类和回归。\n- LiDAR点云被转换为六通道鸟瞰图表示，并与RGB特征通过RoI Align提取后拼接，而不是简单平均，以保留信息。\n- 锚点姿态选用8个完整身体姿态，通过3D边界框和地面平面生成，并在推理时拟合到预测的RoI中。\n- 损失函数包含RPN损失、分类损失、2D姿态回归损失和投影3D姿态回归损失，总损失为四者之和。\n- 弱监督训练仅使用2D姿态标注，通过将3D预测投影到图像平面计算3D损失，无需3D姿态真值。\n- 在PedX数据集上，HPERL相比RGB基线将2D MPJPE降低1.9倍，PCKh@0.5提升4.3个百分点。\n- 新建的CDE和XYE指标分别衡量深度和XY平面误差，HPERL将CDE降低5.1倍，XYE降低3.7倍。\n- 最佳配置下HPERL的XYE为0.39m，接近KITTI上最先进行人检测器的精度（0.11-0.22m）。\n- 消融实验显示，即使配置较差的HPERL也优于RGB基线，表明LiDAR信息的加入是性能提升的关键。\n- 数据增强采用左右翻转，对性能提升有限，因为姿态分布大致对称。\n- 该方法适用于自动驾驶中的行人意图识别等场景，但受限于野外3D姿态标注数据的缺乏，限制了大规模评估。"
}