{
  "paper_id": "5bb28276af79c3c15eea77309d202b8e",
  "paper_title": "LidPose: Real-Time 3D Human Pose Estimation in Sparse Lidar Point Clouds with Non-Repetitive Circular Scanning Pattern",
  "paper_authors": [
    "Lóránt Kovács",
    "Balázs M. Bódis",
    "Csaba Benedek"
  ],
  "publication_date": "2024-05-26",
  "publication_venue": "Sensors",
  "abstract": "In this paper, we propose a novel, vision-transformer-based end-to-end pose estimation method, LidPose, for real-time human skeleton estimation in non-repetitive circular scanning (NRCS) lidar point clouds. Building on the ViTPose architecture, we introduce novel adaptations to address the unique properties of NRCS lidars, namely, the sparsity and unusual rosetta-like scanning pattern. The proposed method addresses a common issue of NRCS lidar-based perception, namely, the sparsity of the measurement, which needs balancing between the spatial and temporal resolution of the recorded data for efficient analysis of various phenomena. LidPose utilizes foreground and background segmentation techniques for the NRCS lidar sensor to select a region of interest (RoI), making LidPose a complete end-to-end approach to moving pedestrian detection and skeleton fitting from raw NRCS lidar measurement sequences captured by a static sensor for surveillance scenarios. To evaluate the method, we have created a novel, real-world, multi-modal dataset, containing camera images and lidar point clouds from a Livox Avia sensor, with annotated 2D and 3D human skeleton ground truth.",
  "keywords": [
    "lidar-only 3D human pose estimation",
    "real-time surveillance",
    "point cloud",
    "rosetta pattern non-repetitive circular scanning",
    "NRCS lidar"
  ],
  "paper_institutions": [
    "HUN-REN Institute for Computer Science and Control (SZTAKI)",
    "Faculty of Information Technology and Bionics, Pázmány Péter Catholic University"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:56.016161Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文提出了一种基于视觉 Transformer 的端到端姿态估计方法 LidPose，用于在非重复圆形扫描（NRCS）激光雷达点云中实现实时人体骨骼估计。该方法针对 NRCS 激光雷达固有的稀疏性和不均匀的 rosetta 扫描模式，对 ViTPose 架构进行了改造，引入新的 patch-embedding 模块以灵活处理不同通道数，并增加了 Transformer 块数量以提升泛化能力。LidPose 利用基于混合高斯模型的背景分割技术从原始激光雷达序列中提取运动行人区域，并通过将点云投影到二维距离图像生成五通道（XYZ、深度、强度）输入，支持 2D、2D+ 和 3D 三种姿态预测变体。训练损失采用联合热图误差与深度误差的加权和（权重分别为 10 和 1，深度损失选用 SSIM）。为验证方法，作者构建了包含 Livox Avia 传感器采集的真实多模态数据集，包含 9500 个骨骼和 161,000 个关节标注，并在不同室内外场景下进行训练、验证和测试。实验表明，最佳配置（XYZ+I 输入，SSIM 深度损失）在 2D 任务中 ADE 达 13.2 像素，在 3D 任务中 ADE 为 0.158 米，PCK 和 AUC-PCK 均优于基线，且推理速度达 52 FPS，满足实时监控需求。未来工作将把方法迁移到移动平台，并探索更鲁棒的点云分割方法。\n### 关键要点\n- LidPose 是一种仅依赖 NRCS 激光雷达点云的实时三维人体姿态估计方法，端到端处理行人检测与骨骼拟合。\n- 方法基于 ViTPose 架构，创新性地设计了适应性 patch-embedding 以处理不同输入通道（XYZ、深度、强度）组合。\n- 输入数据采用二维距离图像表示，将点云投影为五通道图像，缓解了稀疏性和非均匀密度问题。\n- 提出了三种模型变体：LidPose-2D（预测二维关节）、LidPose-2D+（利用现有深度扩展）、LidPose-3D（回归所有关节深度）。\n- 新数据集包含真实多模态数据（Livox Avia 激光雷达与相机），标注了 2D 和 3D 人体骨架，共 9520 个骨骼。\n- 实验测试了五种输入配置，最佳 2D 模型采用 XYZ+I（三维坐标+强度），最佳 3D 模型同样采用 XYZ+I 且深度损失为 SSIM。\n- 3D 模型的深度损失比较显示，L1 损失导致网络无法正确学习深度，而 L2 和 SSIM 表现良好，SSIM 更优。\n- 在 3D 评估中，LidPose-3D 显著优于简单的 2D+ 投影基线，后者因 NRCS 点云稀疏性导致大量关节缺乏深度信息。\n- 模型在边缘区域的精度下降，反映了 NRCS 扫描模式导致点云密度不均匀的固有约束。\n- 系统运行速度达 52 FPS，实际帧率受限于激光雷达采样率（10 FPS），满足实时监控需求。"
}