{
  "paper_id": "92c4096cb579cc7c31418e395e8ca908",
  "paper_title": "VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Data",
  "paper_authors": [
    "Jian Shi",
    "Peter Wonka"
  ],
  "publication_date": "2023",
  "publication_venue": "CVPR 2023",
  "abstract": "We present VoxelKP, a novel fully sparse network architecture tailored for human keypoint estimation in LiDAR data. The key challenge is that objects are distributed sparsely in 3D space, while human keypoint detection requires detailed local information wherever humans are present. We propose four novel ideas in this paper. First, we propose sparse selective kernels to capture multi-scale context. Second, we introduce sparse box-attention to focus on learning spatial correlations between keypoints within each human instance. Third, we incorporate a spatial encoding to leverage absolute 3D coordinates when projecting 3D voxels to a 2D grid encoding a bird's eye view. Finally, we propose hybrid feature learning to combine the processing of per-voxel features with sparse convolution. We evaluate our method on the Waymo dataset and achieve an improvement of 27% on the MPJPE metric compared to the state-of-the-art, HUM3DIL, trained on the same data, and 12% against the state-of-the-art, GC-KPL, pretrained on a 25× larger dataset. To the best of our knowledge, VoxelKP is the first single-staged, fully sparse network that is specifically designed for addressing the challenging task of 3D keypoint estimation from LiDAR data, achieving state-of-the-art performances. Our code is available at <https://github.com/shijianjian/VoxelKP>.",
  "keywords": [
    "LiDAR",
    "human keypoint estimation",
    "3D pose estimation",
    "sparse convolution",
    "voxel-based network",
    "bird's eye view",
    "self-attention"
  ],
  "paper_institutions": [
    "King Abdullah University of Science and Technology"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:33.157842Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文提出VoxelKP，一种专为LiDAR点云中的三维人体关键点估计设计的全稀疏网络架构。现有目标检测方法关注稀疏分布的对象，但关键点估计需要在人类所在区域获得密集局部信息，两者存在上下文尺度差异。相关工作如HUM3DIL和GC-KPL主要依赖额外数据或半监督策略。VoxelKP通过四个关键组件解决该问题：稀疏选择性核（SSK）模块以数据驱动方式聚合多尺度特征；稀疏盒注意力在局部盒内建模体素间依赖；空间感知多尺度BEV融合通过高度编码保留三维空间信息，并采用缩放偏移避免特征重叠；混合特征学习结合逐体素MLP分支与稀疏卷积分支，分别捕捉细粒度细节与邻域结构。网络主干包含stem和四个阶段，用稀疏卷积降低特征图尺寸，后两阶段应用盒注意力，并将特征投影到通道增强的BEV表示。在Waymo v1.4.2数据集上训练，仅使用8,125个带关键点标注的人类样本，采用全局与局部增强、GT采样、AdamW优化器和OneCycle调度，损失包含热图聚焦损失、L1回归损失和骨骼正则化损失；核心机制为SSK模块通过对多尺度稀疏卷积特征进行稀疏全局平均池化，生成通道注意力权重并加权融合，公式为\\bar{F}_s={1/|S_P|∑_{p∈S_P} f_p}。实验显示MPJPE为8.87cm，相比HUM3DIL（12.21cm）改善27%，相比使用200k额外真实样本预训练的GC-KPL（10.10cm）提升12%；各组件消融均有效。未来方向包括更大规模数据、利用时序信息和采用逆运动学约束；实际应用需考虑遮挡、训练数据稀缺和计算效率。\n\n### 关键要点\n- VoxelKP是首个单阶段、全稀疏的网络，专门针对LiDAR数据中3D人体关键点估计任务。\n- 问题挑战在于3D空间中对象稀疏，但关键点估计需要在人类所在位置获取密集局部信息，现有检测器缺乏细粒度空间上下文。\n- 稀疏选择性核模块使用3×3×3和5×5×5的子流形稀疏卷积提取多尺度特征，并通过稀疏全局平均池化与softmax通道注意力选择最优核。\n- 稀疏盒注意力将稀疏体素空间划分为不重叠的盒，在各盒内应用自注意力，从而捕捉局部体素间依赖，优于无注意力和分层注意力。\n- 空间感知多尺度BEV融合采用高度编码卷积（核为(1,1,h)）保留3D高度信息，并用缩放偏移避免多尺度特征投影时位置重叠。\n- 混合特征学习通过每阶段侧MLP分支（线性层+批归一化+ReLU）与稀疏卷积分支逐元素相加，融合逐体素细节和邻域几何信息。\n- 训练使用Waymo v1.4.2数据集，仅8,125个带关键点标注的样本，合并行人和骑行者类别，去除无标注框内点，并用GT采样和数据增强缓解标注稀缺。\n- 损失包括热图聚焦损失、L1回归损失和骨骼长度Huber损失；骨骼正则化约束预测关键点符合人体骨骼长度先验。\n- 在MPJPE指标上，VoxelKP达到8.87cm，比同数据训练的HUM3DIL（12.21cm）改进27%，比使用20万真实样本预训练的GC-KPL（10.10cm）提升12%。\n- 全评估显示所有关键点OKS@AP为0.7300，PEM为0.1695，其中肩部和髋部误差较低，腕部误差最高；真实场景中需处理遮挡和标注稀缺，未来可结合时序和逆运动学。"
}