{
  "paper_id": "b588420bb25c7485b3a0770abe877df6",
  "paper_title": "MOVIN: Real-time Motion Capture using a Single LiDAR",
  "paper_authors": [
    "Deok-Kyeong Jang",
    "Dongseok Yang",
    "Deok-Yun Jang",
    "Byeoli Choi",
    "Taeil Jin",
    "Sung-Hee Lee"
  ],
  "publication_date": "",
  "publication_venue": "Computer Graphics Forum",
  "abstract": "Recent advancements in technology have brought forth new forms of interactive applications, such as the social metaverse, where end users interact with each other through their virtual avatars. In such applications, precise full-body tracking is essential for an immersive experience and a sense of embodiment with the virtual avatar. However, current motion capture systems are not easily accessible to end users due to their high cost, the requirement for special skills to operate them, or the discomfort associated with wearable devices. In this paper, we present MOVIN, the data-driven generative method for real-time motion capture with global tracking, using a single LiDAR sensor. Our autoregressive conditional variational autoencoder (CVAE) model learns the distribution of pose variations conditioned on the given 3D point cloud from LiDAR. As a central factor for high-accuracy motion capture, we propose a novel feature encoder to learn the correlation between the historical 3D point cloud data and global, local pose features, resulting in effective learning of the pose prior. Global pose features include root translation, rotation, and foot contacts, while local features comprise joint positions and rotations. Subsequently, a pose generator takes into account the sampled latent variable along with the features from the previous frame to generate a plausible current pose. Our framework accurately predicts the performer's 3D global information and local joint details while effectively considering temporally coherent movements across frames. We demonstrate the effectiveness of our architecture through quantitative and qualitative evaluations, comparing it against state-of-the-art methods. Additionally, we implement a real-time application to showcase our method in real-world scenarios. MOVIN dataset is available at <https://movin3d.github.io/movin_pg2023/>.",
  "keywords": [
    "Motion capture",
    "LiDAR",
    "Conditional variational autoencoder",
    "Full-body pose estimation",
    "Real-time"
  ],
  "paper_institutions": [
    "Korea Advanced Institute of Science and Technology (KAIST)",
    "Gwangju Institute of Science and Technology (GIST)"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:46:42.376218Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文提出MOVIN框架，首次使用单个LiDAR传感器实现带全局平移的实时全身动作捕捉。针对现有动作捕捉系统成本高、操作复杂或穿戴不适等问题，MOVIN采用自回归条件变分自编码器（CVAE）建模基于LiDAR点云的条件姿态分布。其核心是新颖的特征编码器，利用PointNet++提取点云历史特征，并分别通过图卷积网络（GCN）和多层感知机（MLP）嵌入局部关节特征和全局根部位姿特征，从而有效学习姿态先验。姿态生成器采用混合专家（MoE）解码器，结合采样潜变量和上一帧特征生成当前帧姿态。实验中构建了包含10名受试者、同步LiDAR点云和光学动捕数据的MOVIN数据集，涵盖静态动作和移动动作。在留出测试集上与VIBE和MotionBERT对比，MOVIN在关节位置平均误差（MJPE）上降低约4.41 cm，旋转误差（MJRE）降低约7.93°，但速度误差略高。消融研究验证了点云历史输入、自回归机制和256点输入的有效性。实时演示在Unity3D中实现，能捕捉不同体型和多种动作。未来工作包括扩展数据集以泛化未见动作、提高传感器帧率、以及多LiDAR融合解决严重遮挡。\n\n### 关键要点\n- 提出首个基于单LiDAR的实时全身动作捕捉框架，支持全局平移跟踪，无需穿戴设备。\n- 采用自回归条件变分自编码器（CVAE），结合历史点云和上一帧姿态生成当前姿态。\n- 设计点云特征编码器（PointNet++）融合历史1秒内多帧点云，增强时间一致性。\n- 局部姿态通过骨架感知图卷积网络嵌入，全局姿态（根位置、旋转、速度、脚接触）通过MLP嵌入。\n- 姿态生成器采用混合专家（MoE）解码器，由8个专家网络和门控网络组成，提升姿态质量。\n- 构建MOVIN数据集，包括10名受试者，同步LiDAR点云和光学动捕数据，共约15万帧训练数据。\n- 训练损失包含重建损失（位置、旋转、速度误差）和KL散度，自回归训练长度8帧并使用调度采样。\n- 相比VIBE和MotionBERT，MOVIN关节位置误差降低约4.41 cm，旋转误差降低约7.93°，但速度误差略高。\n- 消融实验表明，去除历史点云或自回归机制会降低时间连续性和增加抖动，增加点到512点未显著提升性能。\n- 实时演示在Unity3D中实现，能捕捉不同体型和多种动作，包括坐地、弓步、跑步等。\n- 后处理利用预测脚接触标签和逆运动学修正脚部滑动。\n- 当前限制包括对未见动作泛化不足、LiDAR帧率低、严重自遮挡时性能下降。"
}