{
  "paper_id": "92ade0acf1082c49994fd03a877cff91",
  "paper_title": "Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments",
  "paper_authors": [
    "Catalin Ionescu",
    "Dragos Papava",
    "Vlad Olaru",
    "Cristian Sminchisescu"
  ],
  "publication_date": "",
  "publication_venue": "IEEE Transactions on Pattern Analysis and Machine Intelligence",
  "abstract": "We introduce a new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, for training realistic human sensing systems and for evaluating the next generation of human pose estimation models and algorithms. Besides increasing the size of the datasets in the current state-of-the-art by several orders of magnitude, we also aim to complement such datasets with a diverse set of motions and poses encountered as part of typical human activities (taking photos, talking on the phone, posing, greeting, eating, etc.), with additional synchronized image, human motion capture, and time of flight (depth) data, and with accurate 3D body scans of all the subject actors involved. We also provide controlled mixed reality evaluation scenarios where 3D human models are animated using motion capture and inserted using correct 3D geometry, in complex real environments, viewed with moving cameras, and under occlusion. Finally, we provide a set of large-scale statistical models and detailed evaluation baselines for the dataset illustrating its diversity and the scope for improvement by future work in the research community. Our experiments show that our best large-scale model can leverage our full training set to obtain a 20% improvement in performance compared to a training set of the scale of the largest existing public dataset for this problem. Yet the potential for improvement by leveraging higher capacity, more complex models with our large dataset, is substantially faster and should stimulate future research. The dataset together with code for the associated large-scale learning models, features, visualization tools, as well as the evaluation server, is available online at <http://vision.imar.ro/human3.6m>.",
  "keywords": [
    "3D human pose estimation",
    "human motion capture data",
    "articulated body modeling",
    "optimization",
    "large-scale learning",
    "structured prediction",
    "Fourier kernel approximations"
  ],
  "paper_institutions": [
    "University of Bonn",
    "Politehnica University of Bucharest",
    "Technical University of Karlsruhe"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:43.386999Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文针对现有3D人体姿态估计数据集规模小、动作多样性不足的问题，提出了Human3.6M大规模数据集，包含360万个精确3D人体姿态，由11位演员在4个视角下录制，覆盖15种典型日常活动场景，并提供同步的2D图像、深度（TOF）数据、3D运动捕获及演员全身3D扫描。为验证方法，作者构建了多种基准，包括实验室环境和混合现实场景（具有真实背景、移动相机和遮挡）。在方法上，提出了基于傅里叶核近似的大规模非线性模型（如核岭回归、结构化预测），使得在数百万样本上训练成为可能，其中最佳模型LinKDE相比小规模训练集可降低约20%测试误差。实验采用MPJPE、MPJAE和MPJLE等指标，验证了大规模数据带来的性能提升，并展示了模型在混合现实数据上的效果。进一步研究方向包括改进图像描述符、增强人体关节相关性建模以及更高容量模型的设计，而实际应用场景涉及自然环境下的人体感知，需应对遮挡、复杂背景和姿态多样性等挑战。核心机制是使用随机傅里叶特征将核方法转化为线性问题，从而支持大规模训练。\n\n### 关键要点\n- Human3.6M数据集包含约360万个3D人体姿态，由11名演员（5女6男）在4个视角下录制，覆盖15种日常活动场景。\n- 数据集同步提供RGB图像、深度（TOF）数据、3D运动捕获、人物剪影和边界框，以及演员的3D身体扫描。\n- 训练集包含5名受试者（2女3男），验证集2人（1女1男），测试集4人（2女2男），共约146万训练姿态、64.6万验证姿态、146.7万测试姿态。\n- 文中提出三种误差度量：MPJPE（毫米）、MPJAE（角度）、MPJLE（感知误差，基于容差），支持不同评估需求。\n- 使用金字塔SIFT网格描述符（3层：2x2、4x4、8x8，9个方向），并比较了基于背景分割（BS）和边界框（BB）的特征提取方式，实验显示BB略优于BS。\n- 大规模模型采用傅里叶核近似，将核岭回归（KRR）转化为线性问题（LinKRR），并将结构化预测基于核依赖估计（KDE）转化为线性形式（LinKDE），支持全数据集训练。\n- 在训练集规模实验中，LinKDE利用全数据集（约210万样本）相比仅用40K样本的KRR，测试误差降低约20%，证明大规模数据收益明显。\n- 在活动特定模型（ASM）和通用模型（GM）设置下，LinKDE在大多数场景中优于kNN、KRR和LinKRR，例如在“Eating”场景中MPJPE达99.52mm（BB）。\n- 混合现实测试数据包含5个动态背景和7,466帧，其中1,270帧有不同程度遮挡，模型在混合现实上的MPJAE最低可达约15度（ASM），但仍有很大改进空间。\n- 数据集中存在自然出现的视觉歧义（前视缩短、单肢体歧义和肢体间歧义），通过自动检测统计了各场景的发生次数，并提供了自我遮挡分析工具。\n- 代码、特征、可视化工具和评估服务器均开放，便于学术界复现和比较。"
}