{
  "paper_id": "d3f45b6e42a0a58a5791f39967a5d396",
  "paper_title": "LPFormer: LiDAR Pose Estimation Transformer with Multi-Task Network",
  "paper_authors": [
    "Dongqiangzi Ye",
    "Yufei Xie",
    "Weijia Chen",
    "Zixiang Zhou",
    "Lingting Ge",
    "Hassan Foroosh"
  ],
  "publication_date": "",
  "publication_venue": "2024 IEEE International Conference on Robotics and Automation (ICRA)",
  "abstract": "Due to the difficulty of acquiring large-scale 3D human keypoint annotation, previous methods for 3D human pose estimation (HPE) have often relied on 2D image features and sequential 2D annotations. Furthermore, the training of these networks typically assumes the prediction of a human bounding box and the accurate alignment of 3D point clouds with 2D images, making direct application in real-world scenarios challenging. In this paper, we present the $1^{st}$ framework for end-to-end 3D human pose estimation, named LPFormer, which uses only LiDAR as its input along with its corresponding 3D annotations. LPFormer consists of two stages: firstly, it identifies the human bounding box and extracts multi-level feature representations, and secondly, it utilizes a transformer-based network to predict human keypoints based on these features. Our method demonstrates that 3D HPE can be seamlessly integrated into a strong LiDAR perception network and benefit from the features extracted by the network. Experimental results on the Waymo Open Dataset demonstrate the state-of-the-art performance, and improvements even compared to previous multi-modal solutions.",
  "keywords": [],
  "paper_institutions": [
    "TuSimple",
    "University of Central Florida"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:36.056435Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n本文提出LPFormer，这是首个仅依赖LiDAR点云和3D标注的端到端3D人体姿态估计框架。针对现有方法依赖2D图像特征和弱监督、且需要精确的相机-LiDAR标定，在真实场景中误差易放大的问题，LPFormer采用两阶段设计：第一阶段利用强大的多任务LiDAR感知网络（LidarMultiNet）预测3D目标框并提取多层级特征（点级、体素级、框级）；第二阶段引入轻量级的关键点Transformer（KPTR），通过注意力机制融合这些特征并回归3D关键点及其可见性。实验在Waymo Open Dataset上进行，测试集上PEM为0.1524、MPJPE为0.0594，排名第一；验证集上MPJPE达6.16cm，优于所有先前基于相机和多模态的方法，且无需图像特征或2D标注。核心机制包括将框特征压缩后与点特征和体素特征拼接，利用可学习的关键点查询进行多级自注意力，并将点级关键点分割作为辅助任务。未来工作包括融合LiDAR与相机数据及探索2D弱监督。\n\n### 关键要点\n- LPFormer是首个仅用LiDAR输入和3D标注实现的端到端3D人体姿态估计框架。\n- 方法分为两阶段：第一阶段使用LidarMultiNet预测3D边界框并提取多级特征，第二阶段使用Transformer预测关键点。\n- 关键点Transformer（KPTR）融合了点级、体素级和框级特征，利用自注意力机制提高关键点回归精度。\n- 使用局部坐标变换和逆变换，将预测的关键点从框内局部坐标系转换回全局LiDAR坐标系。\n- 关键点回归分为XY轴和Z轴两个分支，采用平滑L1损失，且仅对可见关键点计算损失。\n- 引入了辅助任务：点级关键点分割，通过为每个关键点分配最近的K个点生成伪标签，并用交叉熵损失监督。\n- 训练时使用GT边界框代替预测框，简化训练并提升性能；验证了多种框匹配策略的效果。\n- 在Waymo Open Dataset v1.4.2上训练和验证，使用了14类关键点（如鼻、肩、肘等）。\n- 评估指标包括MPJPE（平均关节位置误差）和PEM（姿态估计指标），PEM同时考虑检测质量。\n- 测试集上达到PEM 0.1524、MPJPE 0.0594，在官方榜单排名第一。\n- 验证集上MPJPE为6.16cm，优于所有先前方法，包括多模态和弱监督方法。\n- 消融实验显示，加入点级分割辅助任务、Transformer结构、框特征以及同时使用点特征和体素特征都能有效提升性能。"
}