{
  "paper_id": "a677e6285cbafea2354f9beeca8cccee",
  "paper_title": "Towards Practical Human Motion Prediction with LiDAR Point Clouds",
  "paper_authors": [
    "Xiao Han",
    "Yiming Ren",
    "Yichen Yao",
    "Yujing Sun",
    "Yuexin Ma"
  ],
  "publication_date": "2024-10-28",
  "publication_venue": "Proceedings of the 32nd ACM International Conference on Multimedia (MM'24), October 28-November 1, 2024, Melbourne, VIC, Australia",
  "abstract": "Human motion prediction is crucial for human-centric multimedia understanding and interacting. Current methods typically rely on ground truth human poses as observed input, which is not practical for real-world scenarios where only raw visual sensor data is available. To implement these methods in practice, a prephrase of pose estimation is essential. However, such two-stage approaches often lead to performance degradation due to the accumulation of errors. Moreover, reducing raw visual data to sparse keypoint representations significantly diminishes the density of information, resulting in the loss of fine-grained features. In this paper, we propose LiDAR-HMP, the first single-LiDAR-based 3D human motion prediction approach, which receives the raw LiDAR point cloud as input and forecasts future 3D human poses directly. Building upon our novel structure-aware body feature descriptor, LiDAR-HMP adaptively maps the observed motion manifold to future poses and effectively models the spatial-temporal correlations of human motions for further refinement of prediction results. Extensive experiments show that our method achieves state-of-the-art performance on two public benchmarks and demonstrates remarkable robustness and efficacy in real-world deployments.",
  "keywords": [
    "Human motion prediction",
    "Multimedia understanding",
    "LiDAR point cloud"
  ],
  "paper_institutions": [
    "ShanghaiTech University",
    "The University of Hong Kong"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:42.687823Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文提出LiDAR-HMP，首个基于单LiDAR点云的3D人体运动预测方法，直接以原始LiDAR点云为输入预测未来3D人体姿态，解决传统方法依赖ground truth骨架或两阶段级联误差累积及信息丢失的问题。现有方法常依赖多视角相机或IMU获取精确姿态，实际部署困难；两阶段方法（先姿态估计再预测）会累积误差并损失细粒度特征。LiDAR-HMP通过结构感知身体特征描述符（融合全局特征与按解剖语义划分的部位局部特征，得到H=[H_glo⊕H_part]）、自适应运动潜空间映射（用可学习的运动查询从历史帧中提取信息）以及时空相关性精化模块（交替使用空间Transformer和时间Transformer建模关节间与帧间依赖）实现端到端预测。在LIPD和LiDARHuman26M两个公开数据集上，短时预测平均MPJPE分别提升17.42mm和7.86mm，长时预测提升11.62mm和9.12mm；在遮挡、噪声、远距离（17米）等场景下仍保持稳健，并支持多人多样预测（最多4种未来运动）且实时运行约40fps。进一步研究方向包括提高远距离稀疏点云下的精度、扩展更多传感器融合以及更复杂的交互场景。\n\n### 关键要点\n- LiDAR-HMP是首个直接以单LiDAR原始点云为输入进行3D人体运动预测的方法，避免了传统两阶段方法的误差累积和细粒度信息丢失。\n- 该方法提出结构感知身体特征描述符，将点云按解剖部位（如大腿、小腿、手臂）分割，并融合全局与局部特征，生成表征人体结构的描述符H=[H_glo⊕H_part]。\n- 自适应运动潜空间映射采用可学习的运动查询，通过多头交叉注意力从历史帧特征中提取信息，映射到未来运动潜空间，优于静态填充和线性映射方法。\n- 时空相关性精化模块交替使用空间Transformer（STFormer）和时间Transformer（TTFormer），显式建模关节间的空间依赖与帧间时间依赖，提升预测精度。\n- 网络含双预测头：关节回归头和点云回归头，联合使用L2损失和Chamfer距离损失进行监督训练。\n- 实验在LIPD和LiDARHuman26M上评估，短时（400ms）平均MPJPE为76.80mm和74.55mm，长时（1000ms）为124.49mm和109.55mm，大幅优于现有方法。\n- 与两阶段方法（LiDARCap+SPGCN等）相比，短时预测误差减少约17mm（LIPD）和7.86mm（LiDARHuman26M），长时减少约11.62mm和9.12mm。\n- 与扩散模型MDM比较，扩散方法在点云噪声和变异性下性能有限，而LiDAR-HMP能保留细微运动特征。\n- 消融实验表明，结构感知特征、时空精化模块和点预测辅助头均带来显著提升，如加入局部特征平均改善2.65mm（短时）和2.04mm（长时）。\n- 在模拟80%遮挡或噪声干扰下，MPJPE仅小幅增加，显示出强鲁棒性；在17米距离内稳定预测，适合自动驾驶等远距离应用。\n- 多样预测扩展使用4个可学习查询和“赢者通吃”训练策略，在长时预测中显著降低MinMPJPE，例如LIPD上1000ms从136.10mm降至117.68mm。\n- 实际部署测试在白天和夜晚均能实时运行（约40fps），验证了其在AR、辅助机器人等真实场景中的实用性。"
}