{
  "paper_id": "b2b175602e049d50983771a63ad8b2f6",
  "paper_title": "HySUP: Seam-Aware Hand-Body Fusion for Accurate Whole-Body 3-D Reconstruction from a Single Image",
  "paper_authors": [
    "Le Cong Thuong",
    "Thu Phuong Le",
    "Minh Hang Dang",
    "Son Tung Ha",
    "Thanh Ha Le"
  ],
  "publication_date": "",
  "publication_venue": "2025 International Conference on Multimedia Analysis and Pattern Recognition (MAPR)",
  "abstract": "Expressive human pose and shape estimation (EHPS)—which jointly captures body, hand, and facial motion—is vital for emerging AR/VR and telepresence applications. State-of-the-art holistic regressors accurately recover overall body pose yet still yield fused or imprecise fingers, whereas hand-specific networks resolve fine articulation but sacrifice global coherence. We benchmark two leading holistic models (SMPLest-X, SMPLest-X) and two hand-specialized models (HaMeR, WiLoR) on the EHF dataset and confirm this tradeoff. Leveraging their complementary strengths, we introduce HySUP—a lightweight pipeline that replaces a SMPLest-X hand region with HaMeR output through a two-stage seam-aware alignment: (i) a PCA-guided Procrustes transform for scale and pose normalization, followed by (ii) a rigid-as-possible refinement that enforces vertex- and normal-level consistency along the wrist seam. Without retraining and with only 150 ms of overhead, HySUP reduces hand PA-MPVPE and PA-MPJPE on EHF by 3.30 mm (27.11%) and 3.55 mm (29.48%), respectively, while preserving full-body accuracy. These results highlight a simple path forward: \"plug-and-play\" fusion of specialist parts can outperform ever-larger holistic models at a fraction of the computational cost. Code and more results can be found on our project page.",
  "keywords": [
    "Expressive human pose estimation",
    "SMPL-X",
    "3-D reconstruction",
    "optimization",
    "hybrid models"
  ],
  "paper_institutions": [
    "Faculty of Information Technology, University of Engineering and Technology, Hanoi, Vietnam",
    "Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, United States"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:37:16.574535Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n表达性人体姿态与形状估计（EHPS）旨在从单张RGB图像中联合重建身体、手部与面部的三维运动，是AR/VR、远程呈现、XR内容创作与人机交互等现实应用的关键技术，但整体式基础模型（如SMPLer-X、SMPLest-X）虽能准确恢复躯干与四肢，手指输出却常出现粘连或错位，而手部专用网络（如HaMeR、WiLoR）虽提供毫米级手指细节却牺牲全局一致性。本文在EHF数据集上系统基准评测两类模型并证实该权衡后，提出HySUP——一个轻量级、无需重训练、每帧仅增加约150毫秒开销的两阶段接缝感知融合流水线：以SMPLest-X为身体专家、HaMeR为手部专家，第一阶段利用MANO模板腕部锚点通过PCA引导的Procrustes变换估计初始相似变换（尺度s0、旋转R0=UbUhᵀ、平移t0）完成尺度与姿态归一化，第二阶段进行刚性优化，在腕部接缝处强制顶点与法线级一致性，其核心损失为 L(s,R,t)=Σᵢ‖sR·V_h^(i)+t−V_b^(i)‖²+λΣᵢ(1−⟨R·N_h^(i), N_b^(i)⟩)，采用L-BFGS-B优化器求解（收敛条件为目标变化低于10⁻⁴或达5000次迭代）。实验在EHF基准上进行（该数据集提供多视角扫描拟合的毫米级精度SMPL-X真值网格、纯评测用途不参与任何训练，故能公正验证融合方法本身的贡献），硬件为Intel Core i9-10900 CPU与NVIDIA RTX 3090 GPU，指标采用Procrustes对齐后的PA-MPVPE与PA-MPJPE：HySUP将手部PA-MPVPE降低3.30mm（27.11%）、PA-MPJPE降低3.55mm（29.48%），同时保持全身精度，验证了\"即插即用\"部件融合能以极小计算代价优于更大规模整体模型的创新路径。消融实验进一步表明，引入法线损失（λ≥10⁻²）可将法线误差从20.52°/54.98°降至约0.26°，PCA初始化使迭代次数减少约四倍。未来该范式可推广至面部、鞋类、服装等部件融合及手语理解等应用；其局限在于增益受底层部件网络质量制约，且需在低延迟场景中权衡约150ms的额外开销。\n\n### 关键要点\n\n- 问题与动机：整体式模型（SMPLest-X）手指常粘连或错位，手部专用模型（HaMeR、WiLoR）细节精确但缺乏与全身坐标系的全局一致性，HySUP旨在无需重训练、低延迟地融合两者互补优势。\n- 基准发现：在EHF数据集上，HaMeR手部PA-MPVPE为8.87mm，比次优专用模型WiLoR（11.95mm）低约3.08mm（≈26%），且较弱的WiLoR仍优于最强整体模型SMPLest-X（12.17mm），证实专用手部网络在手指几何上始终占优。\n- 全身基线：SMPLest-X在整体模型中全身精度最佳（PA-MPVPE 36.33mm、PA-MPJPE 39.89mm），比SMPLer-X分别改善约6.3%与6.8%，故被选为身体专家。\n- 方法架构：HySUP（Hybrid Sub-module Unification Pipeline）以SMPLest-X为身体专家、HaMeR为手部专家，通过两阶段接缝感知对齐将HaMeR的精细手部网格替换SMPLest-X的粗糙手部区域。\n- 第一阶段（PCA引导Procrustes）：基于MANO模板腕部锚点顶点（与SMPL-X腕环一一对应的闭合环）计算质心与最大范围得t0、s0，PCA得正交基Ub、Uh，旋转R0=UbUhᵀ，构成初始相似变换。\n- 第二阶段（接缝优化）：以顶点距离损失加λ加权法线一致性损失为目标函数，使用L-BFGS-B优化器（仅有软平滑约束与s>0线性界），目标变化低于10⁻⁴或达5000次迭代时判定收敛。\n- 核心结果：在EHF上HySUP将手部PA-MPVPE降低3.30mm（27.11%）、PA-MPJPE降低3.55mm（29.48%），同时保持全身精度，每帧仅增加约150ms且完全无需重训练。\n- 消融—法线损失：仅用顶点损失（λ=0）致法线误差高达20.52°（右手）/54.98°（左手）；λ=10⁻²即降至约0.26°且不明显影响顶点精度，λ在10⁻²至10间结果稳定，平衡权重λ≈1最优。\n- 消融—阶段作用：PCA引导Procrustes初始化虽不显著提升最终精度，但使迭代次数约减四倍（右手134.81→31.68，左手109.30→26.44），自身开销不足1ms，大幅提升整体效率。\n- 数据与硬件：所有评测在EHF数据集（高分辨率RGB帧、多视角扫描拟合的毫米级SMPL-X真值、纯基准不参与训练）上进行，实验平台为Intel Core i9-10900 CPU（2.80GHz）与NVIDIA GeForce RTX 3090 GPU。\n- 局限与展望：HySUP增益受底层部件网络质量制约，若未来整体模型完全解决手部关节问题其优势将缩小；该\"即插即用\"融合范式可推广至面部、鞋类、服装等领域，并使手语理解等应用更可行。"
}