{
  "paper_id": "6faff01276a5cdc8f210a3cbfa491ac5",
  "paper_title": "3D Human Keypoints Estimation from Point Clouds in the Wild without Human Labels",
  "paper_authors": [
    "Zhenzhen Weng",
    "Alexander S. Gorban",
    "Jingwei Ji",
    "Mahyar Najibi",
    "Yin Zhou",
    "Dragomir Anguelov"
  ],
  "publication_date": "",
  "publication_venue": "2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",
  "abstract": "Training a 3D human keypoint detector from point clouds in a supervised manner requires large volumes of high quality labels. While it is relatively easy to capture large amounts of human point clouds, annotating 3D keypoints is expensive, subjective, error prone and especially difficult for long-tail cases (pedestrians with rare poses, scooterists, etc.). In this work, we propose GC-KPL -Geometry Consistency inspired Key Point Leaning, an approach for learning 3D human joint locations from point clouds without human labels. We achieve this by our novel unsupervised loss formulations that account for the structure and movement of the human body. We show that by training on a large training set from Waymo Open Dataset without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach. Further, the backbone benefits from the unsupervised training and is useful in downstream few-shot learning of keypoints, where fine-tuning on only 10 percent of the labeled training data gives comparable performance to fine-tuning on the entire set. We demonstrated that GC-KPL outperforms by a large margin over SoTA when trained on entire dataset and efficiently leverages large volumes of unlabeled data.",
  "keywords": [
    "3D human keypoint estimation",
    "point clouds",
    "unsupervised learning",
    "self-supervised learning",
    "autonomous driving",
    "Waymo Open Dataset"
  ],
  "paper_institutions": [
    "Stanford University",
    "Waymo"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:32.868375Z",
  "template_tag": "simple",
  "summary": "### 摘要\nGC-KPL 提出了一种从野外点云中无人工标签学习 3D 人体关键点的方法，解决传统监督方法对高质量 3D 标注依赖严重、标注昂贵且难覆盖长尾姿态的问题。该方法基于人体骨骼与表面点运动的刚性一致性，设计了三类无监督损失：流动损失、点到肢体损失和对称损失，并辅以关节到部位损失与分割损失进行正则化。训练分两阶段：第一阶段在由 SMPL 模型随机姿态生成的合成数据上初始化 transformer 骨干和分割模型；第二阶段在 Waymo Open Dataset 约 20 万个未标注样本上优化无监督损失，无需任何人工 3D 关键点标注。在指标上，使用平均每关节位置误差（MPJPE）进行评测，经过第二阶段无监督训练后 MPJPE 从仅合成数据的 17.7 cm 降至 13.92 cm；在下游微调任务中，仅用 10% 标注数据微调即可达到与全量监督训练相当的性能（10.10 cm vs 12.21 cm），且显著优于随机初始化基线。核心机制在于将点云投影到预测骨骼的局部圆柱坐标系，通过流动损失保持相邻帧间表面点在肢体局部坐标中的径向和轴向一致性，结合点到肢体距离与对称性约束，使得预测骨架贴合人体表面。该方法还可作为自监督表示学习，利用大规模无标签数据提升下游任务表现，未来可改进流动估计器、使用更强的分割模型或融合多模态输入以进一步提升精度。\n### 关键要点\n- GC-KPL 首次实现完全无人工标签的野外点云 3D 人体关键点学习，无需 RGB 或 2D 弱监督。\n- 提出三类新颖的无监督损失：流动损失、点到肢体损失和对称损失，利用人体结构刚性与运动一致性约束关键点位置。\n- 训练分两阶段：先在合成 SMPL 数据（1,000 序列，每序列 16 帧）上监督初始化，再在大规模未标注的 Waymo Open Dataset（约 200,000 样本）上进行无监督精炼。\n- 合成数据阶段应用多种数据增强（随机降采样、遮挡、添加背景点、噪声等）以模拟真实噪声和遮挡，增强泛化能力。\n- 模型使用 transformer 骨干（同 HUM3DIL），总参数量 352,787，输入最多 1024 个 LiDAR 点。\n- 在无监督阶段，分割分支使用 Stage I 预训练模型推理得到硬标签，并保留分割损失作为正则化。\n- 在 Waymo Open Dataset 的验证集上，经过 Stage II 无监督训练后 MPJPE 从 17.7 cm 降至 13.92 cm，显著优于仅合成数据。\n- 下游微调实验中，GC-KPL 预训练骨干在 1%、10%、100% 标注数据下均优于随机初始化，分别获得 2.37、2.96、2.11 cm 的 MPJPE 增益。\n- 与域适应方法（梯度反转）相比，GC-KPL 在 WOD 验证集上 MPJPE 更低（10.1 cm vs 11.35 cm）。\n- 消融实验表明，去掉流动、点到肢体或对称损失中的任意一个都会导致训练不稳定和性能大幅下降。\n- 使用 KMeans 替代合成数据真值进行初始化时，最终性能与有监督初始化相当，说明方法无需任何人工标签。\n- 该方法可拓展为自监督表示学习，从大规模传感器数据中学习有效特征，适用于自动驾驶中行人姿态理解。"
}