{
  "paper_id": "222cbb6b605bfa78f56b993c92b9b287",
  "paper_title": "Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation",
  "paper_authors": [
    "Wenkang Shan",
    "Zhenhua Liu",
    "Xinfeng Zhang",
    "Zhao Wang",
    "Kai Han",
    "Shanshe Wang",
    "Siwei Ma",
    "Wen Gao"
  ],
  "publication_date": "",
  "publication_venue": "2023 IEEE/CVF International Conference on Computer Vision (ICCV)",
  "abstract": "In this paper, a novel Diffusion-based 3D Pose estimation (D3DP) method with Joint-wise reProjection-based Multi-hypothesis Aggregation (JPMA) is proposed for probabilistic 3D human pose estimation. On the one hand, D3DP generates multiple possible 3D pose hypotheses for a single 2D observation. It gradually diffuses the ground truth 3D poses to a random distribution, and learns a denoiser conditioned on 2D keypoints to recover the uncontaminated 3D poses. The proposed D3DP is compatible with existing 3D pose estimators and supports users to balance efficiency and accuracy during inference through two customizable parameters. On the other hand, JPMA is proposed to assemble multiple hypotheses generated by D3DP into a single 3D pose for practical use. It reprojects 3D pose hypotheses to the 2D camera plane, selects the best hypothesis joint-by-joint based on the reprojection errors, and combines the selected joints into the final pose. The proposed JPMA conducts aggregation at the joint level and makes use of the 2D prior information, both of which have been overlooked by previous approaches. Extensive experiments on Human3.6M and MPI-INF-3DHP datasets show that our method outperforms the state-of-the-art deterministic and probabilistic approaches by 1.5% and 8.9%, respectively. Code is available at https://github.com/paTRICK-swk/D3DP.",
  "keywords": [
    "3D human pose estimation",
    "diffusion model",
    "multi-hypothesis aggregation",
    "probabilistic pose estimation",
    "reprojection"
  ],
  "paper_institutions": [
    "National Engineering Research Center of Visual Technology, Peking University",
    "University of Chinese Academy of Sciences",
    "Huawei Noah's Ark Lab",
    "Peng Cheng Laboratory"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:24.635141Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n本文针对单目3D人体姿态估计中的深度模糊和多解性问题，提出了一种基于扩散模型的概率方法D3DP（Diffusion-based 3D Pose estimation）以及联合级重投影多假设聚合方法JPMA（Joint-wise reProjection-based Multi-hypothesis Aggregation）。D3DP利用扩散模型在训练时逐步向真值3D姿态加入噪声，并学习一个以2D关键点为条件的去噪器来重构原始姿态；推理时从高斯噪声采样多个初始姿态，通过去噪器生成多个可行的3D姿态假设，并通过DDIM迭代细化，支持自定义假设数量H和迭代次数K以在效率与精度间平衡。D3DP的兼容性体现在可以直接采用现有确定性姿态估计器（如MixSTE）作为骨干网络，仅需微小的修改（拼接2D关键点和添加时间步嵌入）即可转化为概率版本。JPMA将多个3D姿态假设重投影到2D相机平面，针对每个关节选择重投影误差最小的假设，再组合成最终姿态，从而在关节级别进行细粒度聚合，并利用了2D先验信息，克服了以往方法在姿态级别平均导致性能下降的问题。在Human3.6M和MPI-INF-3DHP数据集上，D3DP+JPMA在MPJPE等指标上分别比最优的确定性和概率性方法提升1.5%和8.9%。消融实验表明，关节级聚合的上界性能远高于姿态级，且JPMA优于平均和基于MLP的方法。进一步探索方向包括提升JPMA的性能以缩小与上界的差距、扩展到其他姿态估计任务以及应用更复杂的融合策略。现实场景包括人机交互、元宇宙和自动驾驶等，局限性在于需要相机内参，且计算开销随H和K线性增长。核心机制为扩散过程公式 q(y_t|y_0)=√(ᾱ_t) y_0 + ε√(1-ᾱ_t) 及重投影选择公式 ỹ_0^(i) = ỹ_{h',0}^(i), h'=argmin_h ||P(ỹ_{h,0}^(i)) - x^(i)||_2。\n\n### 关键要点\n\n- D3DP利用扩散模型生成多个3D姿态假设，通过条件去噪器以2D关键点作为条件，能够反映3D姿态估计中的深度不确定性。\n- D3DP与现有确定性3D姿态估计器兼容，可直接将MixSTE等作为骨干，仅需修改输入拼接和时间步嵌入，实现从确定性到概率方法的转换。\n- D3DP支持推理时自定义假设数H和迭代数K，用户可根据计算资源权衡精度和效率，H和K越大性能越好但计算量线性增加。\n- 提出JPMA方法，将3D假设重投影到2D平面，按关节选择重投影误差最小的假设，然后组合成最终姿态，实现关节级聚合。\n- 实验证明关节级聚合的上界性能（J-Best）远高于姿态级（P-Best），在Human3.6M上J-Best比P-Best低约4mm，凸显了关节级聚合的优势。\n- 在Human3.6M上，D3DP在单假设（H=1）时MPJPE达到40.0mm，优于确定性方法MixSTE（41.0mm）和DUE（40.6mm），表明扩散训练有助于泛化。\n- 在多假设场景下（H=20, K=10），JPMA的J-Agg性能（39.5mm）优于P-Agg（39.9mm），且与P-Best（39.5mm）持平，表明JPMA有效利用2D先验。\n- 在MPI-INF-3DHP上，D3DP在单假设时MPJPE为30.2mm，比MixSTE（35.4mm）降低5.2mm，J-Agg（29.7mm）进一步优于P-Agg（30.0mm），显示良好的泛化能力。\n- 消融实验显示，联合扩散模型带来的性能提升为1mm（41.0→40.0mm），而JPMA相比平均聚合带来0.4mm提升，且关节级方法普遍优于姿态级方法。\n- 定性分析表明，在2D关键点误检测、遮挡和深度模糊情况下，D3DP生成高方差的假设以反映不确定性；对简单姿势则趋于确定性输出。\n- JPMA需要已知或估计的相机内参（针孔相机4参数或畸变相机9参数），重投影过程无网络参数，计算开销可忽略。\n- D3DP在推理时通过调整H和K可线性增加MACs和降低FPS，如H=20, K=10时MACs为228.2G，FPS为29，适合需要平衡速度与精度的应用场景。"
}