{
  "paper_id": "875cb5c163a59fb04d3d1f6d62c8868c",
  "paper_title": "Sequential 3D Human Pose and Shape Estimation from Point Clouds",
  "paper_authors": [
    "Kangkan Wang",
    "Jin Xie",
    "Guofeng Zhang",
    "Lei Liu",
    "Jian Yang"
  ],
  "publication_date": "",
  "publication_venue": "2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",
  "abstract": "This work addresses the problem of 3D human pose and shape estimation from a sequence of point clouds. Existing sequential 3D human shape estimation methods mainly focus on the template model fitting from a sequence of depth images or the parametric model regression from a sequence of RGB images. In this paper, we propose a novel sequential 3D human pose and shape estimation framework from a sequence of point clouds. Specifically, the proposed framework can regress 3D coordinates of mesh vertices at different resolutions from the latent features of point clouds. Based on the estimated 3D coordinates and features at the low resolution, we develop a spatial-temporal mesh attention convolution (MAC) to predict the 3D coordinates of mesh vertices at the high resolution. By assigning specific attentional weights to different neighboring points in the spatial and temporal domains, our spatial-temporal MAC can capture structured spatial and temporal features of point clouds. We further generalize our framework to the real data of human bodies with a weakly supervised fine-tuning method. The experimental results on SURREAL, Human3.6M, DFAUST and the real detailed data demonstrate that the proposed approach can accurately recover the 3D body model sequence from a sequence of point clouds.",
  "keywords": [
    "3D human pose",
    "shape estimation",
    "point clouds",
    "spatial-temporal mesh attention convolution",
    "weakly supervised fine-tuning",
    "SMPL"
  ],
  "paper_institutions": [
    "Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education",
    "Jiangsu Key Lab of Image and Video Understanding for Social Security, School of Computer Science and Engineering, Nanjing University of Science and Technology, China",
    "State Key Laboratory of CAD&CG, Zhejiang University, China",
    "ZJU-SenseTime Joint Lab of 3D Vision"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:31.256273Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文针对从点云序列进行三维人体姿态和形状估计的问题。现有方法主要依赖于深度图像的模板模型拟合或RGB图像的参数模型回归，而直接利用点云几何信息的方法较少。我们提出了一种新颖的从点云序列估计三维人体姿态和形状的框架，通过PointNet++提取每帧点云的潜在特征，再基于空间-时间网格注意力卷积（MAC）由粗到细地回归不同分辨率的网格顶点坐标。空间MAC动态分配顶点一环邻域的注意力权重，时间MAC融合连续帧对应顶点的特征，从而有效捕捉点云的结构化时空特征。该框架无需构建点对应关系，避免了误差累积。为了适应真实人体数据，我们还提出了弱监督微调方法，利用粗略预测的3D模型作为监督信号，结合3D对应损失、拉普拉斯损失和边损失，使模型能够注册到真实点云。在SURREAL、Human3.6M、DFAUST和真实数据上的实验表明，该方法能够准确恢复3D人体模型序列，重建误差显著低于现有方法。进一步探索方向包括处理极端姿态和宽松衣物，以及构建大规模真实对齐数据集。真实应用场景包括机器人、增强现实和虚拟现实，但受限于深度传感器噪声和自遮挡。核心机制是通过注意力权重动态聚合局部时空特征，实现从粗到细的网格回归。\n\n### 关键要点\n- 首次提出从点云序列估计3D人体姿态和形状，直接回归不同分辨率的网格顶点坐标，避免构建点对应关系。\n- 设计空间网格注意力卷积（SMAC），对每个顶点的一环邻域动态分配注意力权重，基于坐标和特征差异学习局部结构。\n- 设计时间网格注意力卷积（TMAC），对连续帧的对应顶点进行注意力加权，融合时空特征，缓解遮挡和形状模糊。\n- 采用四分辨率粗到细回归，通过上采样矩阵和MAC逐步细化，最终输出1723个顶点的网格。\n- 使用SMPL模型作为人体模型，但直接预测顶点坐标，而非参数，从而获得更高拟合精度。\n- 在SURREAL、Human3.6M和DFAUST数据集上进行评估，MAVE误差分别低至18.2mm、21.4mm和19.7mm，优于现有方法。\n- 与纯模型拟合、Bogo等人、Wei等人以及Kanazawa等人等方法相比，重建误差显著降低，证明直接回归的有效性。\n- 消融实验表明，移除SMAC或TMAC会显著增加误差，验证了时空注意力卷积的重要性。\n- 弱监督微调方法利用预先粗略预测的3D模型作为监督，通过拉普拉斯损失和边损失保持表面平滑和边缘一致，适用于真实穿衣人体。\n- 实验发现采样点数从2500增加到7500时精度仅略有提升，但训练时间明显增加，因此选择2500点平衡精度和效率。\n- 在真实数据上测试，方法能处理自遮挡和任意变形，但极端姿态和宽松衣物（如长裙）仍可能失败。\n- 每个测试样本的平均运行时间约24.7ms（NVIDIA 2080 Ti GPU），表明方法具有实时应用潜力。"
}