{
  "paper_id": "27074c3380bf86df103eccbbf60fd712",
  "paper_title": "Expressive Body Capture: 3D Hands, Face, and Body from a Single Image",
  "paper_authors": [
    "Georgios Pavlakos",
    "Vasileios Choutas",
    "Nima Ghorbani",
    "Timo Bolkart",
    "Ahmed A. A. Osman",
    "Dimitrios Tzionas",
    "Michael J. Black"
  ],
  "publication_date": "",
  "publication_venue": "2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",
  "abstract": "To facilitate the analysis of human actions, interactions and emotions, we compute a 3D model of human body pose, hand pose, and facial expression from a single monocular image. To achieve this, we use thousands of 3D scans to train a new, unified, 3D model of the human body, SMPL-X, that extends SMPL with fully articulated hands and an expressive face. Learning to regress the parameters of SMPL-X directly from images is challenging without paired images and 3D ground truth. Consequently, we follow the approach of SMPLify, which estimates 2D features and then optimizes model parameters to fit the features. We improve on SMPLify in several significant ways: (1) we detect 2D features corresponding to the face, hands, and feet and fit the full SMPL-X model to these; (2) we train a new neural network pose prior using a large MoCap dataset; (3) we define a new interpenetration penalty that is both fast and accurate; (4) we automatically detect gender and the appropriate body models (male, female, or neutral); (5) our PyTorch implementation achieves a speedup of more than 8x over Chumpy. We use the new method, SMPLify-X, to fit SMPL-X to both controlled images and images in the wild. We evaluate 3D accuracy on a new curated dataset comprising 100 images with pseudo ground-truth. This is a step towards automatic expressive human capture from monocular RGB data. The models, code, and data are available for research purposes at https://smpl-x.is.tue.mpg.de.",
  "keywords": [
    "SMPL-X",
    "SMPLify-X",
    "3D人体姿态估计",
    "参数化人体模型",
    "手部姿态估计",
    "面部表情捕捉",
    "单目RGB图像",
    "变分自编码器姿态先验",
    "碰撞惩罚",
    "性别分类"
  ],
  "paper_institutions": [
    "MPI for Intelligent Systems, Tübingen, DE",
    "University of Pennsylvania, PA, USA"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:35:09.029888Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文解决从单张单目RGB图像同时重建人体、双手和面部完整3D表面的问题,以支持对人类动作、交互和情感的分析。以往研究要么只关注身体主要关节(如SMPL、SMPLify),要么将身体、手、脸分开建模(如Frank将三个异构模型拼接导致不真实伪影),缺乏统一的表达性模型与图像-3D真值配对数据。为此,作者提出统一参数化模型SMPL-X:以SMPL为基础,融合FLAME头部模型与MANO手部模型,并注册拟合到5586个精选3D扫描(3800个A-pose对齐训练形状空间、1786个多姿态对齐训练姿态空间)以捕捉身体、脸、手形状间的自然相关性;该模型具有10475个顶点、54个关节(含颈、下颌、眼球和手指关节),共119个参数。由于缺乏配对训练数据,直接回归不可行,故采用SMPLify式优化拟合方法SMPLify-X:先用OpenPose自底向上检测身体、手、脚和面部2D关键点,再通过最小化目标函数 E(β,θ,ψ)=E_J+λ_θb·E_θb+λ_θf·E_θf+λ_mh·E_mh+λ_α·E_α+λ_β·E_β+λ_E·E_E+λ_C·E_C 自顶向下拟合模型,其中数据项为基于Geman-McClure鲁棒函数的重投影误差,并引入基于变分自编码器的VPoser姿态先验(在CMU、Human3.6M、PosePrior约100万姿态上训练)、基于BVH与锥形距离场的快速精确自穿透惩罚、基于ResNet18的自动性别检测(选择男性/女性/中性模型)以及PyTorch+L-BFGS实现(比Chumpy快8倍以上)。评估采用新整理的EHF数据集(100张带伪真值图像),使用比3D关节误差更严格的顶点到顶点(v2v)指标:SMPL-X取得52.9mm,优于\"SMPL\"(57.6mm)和\"SMPL+H\"(54.2mm);消融实验显示性别中立模型误差升至58.0mm、以GMM替代VPoser升至56.4mm、去掉碰撞项升至53.5mm;与仅手部方法相比误差为19.8mm对26.5mm。方法还成功应用于LSP、MPII等in-the-wild图像,并在仅用1张图像条件下达到与使用500+相机的Frank相当的表现力。未来工作计划整理in-the-wild SMPL-X拟合数据集并训练直接从RGB图像回归SMPL-X参数的回归器;实际约束包括需要已知或近似的相机焦距、2D检测噪声、遮挡导致部分关节无约束以及深度歧义等失败情形。\n\n### 关键要点\n- SMPL-X是新的统一3D人体模型,联合建模身体、双手和面部,具有10475个顶点和54个关节(含颈、下颌、眼球、手指),总参数119个,基于SMPL并保留其可微、高效、兼容图形软件等优点。\n- 模型通过将SMPL与FLAME头部模型和MANO手部模型结合,并注册拟合到5586个精选3D扫描而学习,捕捉身体、脸、手形状间的自然相关性,避免了Frank拼接模型的伪影;手部采用MANO的低维PCA姿态空间(24参数)。\n- SMPLify-X遵循SMPLify范式:先用OpenPose自底向上检测身体、手、脚、面部2D关键点,再以优化方式自顶向下拟合SMPL-X,因为缺乏图像与3D真值配对数据而无法直接回归模型参数。\n- VPoser是基于变分自编码器的身体姿态先验,在CMU、Human3.6M训练集和PosePrior三个公开MoCap数据集约100万姿态上训练,学习32维潜在空间,惩罚不可能姿态同时允许合理姿态,以应对2D特征到3D姿态的歧义性。\n- 新的自穿透(碰撞)惩罚项基于包围体层次结构(BVH)检测碰撞三角形并计算局部锥形3D距离场,按侵入深度双向惩罚,通过自定义CUDA内核与PyTorch算子实现高并行,比SMPLify的胶囊近似更准确高效且保持可微。\n- 训练了基于ResNet18的深度性别分类器,使用LSP、LSP-extended、MPII、MS-COCO和LIP共50216个训练样本(经Amazon Mechanical Turk标注),以0.9为接受阈值(验证集上62.38%正确、7.54%错误),自动选择男性、女性或性别中立模型。\n- PyTorch实现配合L-BFGS优化器和强Wolfe线搜索,比原Chumpy实现快8倍以上;采用多阶段退火方案,先以高正则化优化全局身体姿态,再逐步增加手部(γ_h: 0→0.1→2)和面部关键点(γ_f: 0→2)的影响以捕捉表现力。\n- 评估采用新整理的EHF数据集:从SMPL+H数据集出发,由专家筛选100帧可视为伪真值的图像,支持比3D关节误差更严格的顶点到顶点(v2v)误差度量,因为关节误差无法反映表面误差和沿骨骼的旋转。\n- 在EHF上,SMPL-X取得52.9mm的v2v误差,优于\"SMPL\"(仅身体关键点57.6mm)和\"SMPL+H\"(身体+手54.2mm),证明更丰富的建模能力带来更低误差;标准3D关节误差(62.6mm)不足以反映模型表现力差异。\n- 消融实验表明:性别中立模型误差升至58.0mm,以SMPLify的GMM替代VPoser升至56.4mm,去掉碰撞项升至53.5mm且允许物理上不合理的姿态,验证了各组件的贡献。\n- 与使用超过500个相机的Frank模型相比,SMPLify-X仅用1张RGB图像和2D关节即可达到相当的表现力,且因包含姿态混合形状而无肘部等关节处的蒙皮伪影;与仅手部方法相比,在噪声检测下更鲁棒,手部3D关节误差为19.8mm对26.5mm。\n- 方法可自然应用于LSP、LSP-extended和MPII等in-the-wild图像;失败情形包括深度歧义导致的躯干姿态错误、遮挡关节使某些身体部位缺乏约束;未来工作将整理in-the-wild SMPL-X拟合数据集并学习直接从RGB图像回归SMPL-X参数的回归器。",
  "available_templates": [
    "deep_read",
    "simple"
  ]
}