{
  "paper_id": "89dfedadb2d7a0c8de0b2ca049b6a50a",
  "paper_title": "MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image",
  "paper_authors": [
    "Xingyu Chen",
    "Yufeng Liu",
    "Yajiao Dong",
    "Xiong Zhang",
    "Chongyang Ma",
    "Yanmin Xiong",
    "Yuan Zhang",
    "Xiaoyan Guo"
  ],
  "publication_date": "2022",
  "publication_venue": "CVPR 2022",
  "abstract": "In this work, we propose a framework for single-view hand mesh reconstruction, which can simultaneously achieve high reconstruction accuracy, fast inference speed, and temporal coherence. Specifically, for 2D encoding, we propose lightweight yet effective stacked structures. Regarding 3D decoding, we provide an efficient graph operator, namely depth-separable spiral convolution. Moreover, we present a novel feature lifting module for bridging the gap between 2D and 3D representations. This module begins with a map-based position regression (MapReg) block to integrate the merits of both heatmap encoding and position regression paradigms for improved 2D accuracy and temporal coherence. Furthermore, MapReg is followed by pose pooling and pose-to-vertex lifting approaches, which transform 2D pose encodings to semantic features of 3D vertices. Overall, our hand reconstruction framework, called MobRecon, comprises affordable computational costs and miniature model size, which reaches a high inference speed of 83FPS on Apple A14 CPU. Extensive experiments on popular datasets such as FreiHAND, RHD, and HO3Dv2 demonstrate that our MobRecon achieves superior performance on reconstruction accuracy and temporal coherence. Our code is publicly available at https://github.com/SeanChenxy/HandMesh.",
  "keywords": [
    "手部网格重建",
    "移动友好",
    "时序一致性",
    "深度可分离螺旋卷积",
    "特征提升模块",
    "单目图像",
    "轻量网络"
  ],
  "paper_institutions": [
    "Y-tech, Kuaishou Technology",
    "YY Live, Baidu Inc.",
    "SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University, China"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:26:30.488443Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n单视图手部网格重建是AR/VR与行为理解等应用的核心技术，但现有方法（基于MANO参数模型、体素表示、隐式函数、UV图及顶点回归等路线）大多以计算密集型网络追求重建精度，忽视了移动设备上同样关键的推理效率与时序一致性，而传统时序方法依赖序列模型、计算昂贵甚至需要未来帧信息，难以在线部署。为此，本文提出移动友好的MobRecon框架：2D编码采用受沙漏网络启发的轻量堆叠结构（DenseStack与GhostStack）；2D到3D映射由特征提升模块完成，其中MapReg（基于图的位置回归）融合热图编码与位置回归两种范式的优点——通过跳跃连接融合低/高分辨率特征保证精度、利用全局感受野提升时序一致性，随后姿态池化与姿态到顶点提升（PVL）借助可学习提升矩阵将2D姿态编码转换为3D顶点语义特征，核心机制为 F_mini_v = M^l · F^p（其中 M^l ∈ R^(V^mini×N) 为可学习提升矩阵，将N=21个关节特征线性组合为V^mini=49个顶点特征，把特征映射复杂度从O(V^mini·C^e²)降至O(N·V^mini·C^e)）；3D解码提出深度可分离螺旋卷积DSConv，将图卷积复杂度从SpiralConv++的O(SD²)降至O(SD+D²)。鉴于现有数据集存在手部姿态与视角的长尾分布问题，作者构建了包含1520个均匀分布姿态与216个视角的合成数据集作为训练补充；模型以128×128输入分辨率、Adam优化器（批大小32、38轮、初始学习率10^-3）训练，并在FreiHAND（130,240训练样本）、RHD（41,258训练样本）与HO3Dv2（66,034训练样本、含严重物体遮挡）上评估，这些基准分别验证了重建精度、泛化能力与时序一致性。采用MPJPE/MPVPE、PA-MPJPE/MPVPE、AUC、F-Score与Acc（加速度）等指标，MobRecon整体仅123M Mult-Adds与5M参数，在Apple A14 CPU上达83FPS；DenseStack/GhostStack版本在FreiHAND上取得7.55/8.89mm PAMPJPE，放大版ResNet50模型（224×224输入）达5.7mm PAMPJPE创当时最优，HO3Dv2上DenseStack取得9.2mm PJ优于ObMan、HO3D与I2UV-HandNet，时序一致性（2D/3D Acc为1.98/4.75）亦优于对比方法，验证了轻量编码、特征提升与DSConv等创新的有效性。未来工作计划研究双手交互的高效重建方法；实际部署需注意移动设备有限的内存与计算预算约束，且DSConv会增加内存访问成本、需工程优化以进一步提升推理速度。\n\n### 关键要点\n\n- 提出MobRecon移动友好手部网格重建流水线，整体仅123M乘加运算与5M参数，在Apple A14 CPU上达到83FPS推理速度，同时兼顾重建精度与时序一致性。\n- 2D编码设计两种轻量堆叠结构：DenseStack（373.0M Mult-Adds、6.6M参数）与GhostStack（96.2M Mult-Adds、5.0M参数），远低于ResNet18堆叠网络的2391.3M Mult-Adds与25.2M参数。\n- MapReg融合热图与位置回归两种范式：通过跳跃连接融合低/高分辨率特征保证2D精度，利用全局感受野描述关节铰接关系以提升时序一致性，在RHD上2D AUC达0.870、2D Acc降至2.04。\n- PVL设计可学习提升矩阵 M^l ∈ R^(49×21)，将21个关节特征线性组合为49个顶点特征，参数仅1029个，把2D到3D特征映射复杂度从O(V^mini·C^e²)降至O(N·V^mini·C^e)，显著减小模型规模。\n- DSConv借鉴深度可分离卷积思想处理图结构网格数据，由深度操作与逐点操作组成，3D解码器仅需19.5M Mult-Adds与0.1M参数，复杂度从SpiralConv++的O(SD²)降至O(SD+D²)，性能持平或更优。\n- 构建包含1520个均匀分布手部姿态与216个视角的合成数据集，弥补现有数据集的长尾分布问题，可用于预训练与微调阶段。\n- 训练设置：输入分辨率128×128，Adam优化器，批大小32，训练38轮，初始学习率10^-3并在第30轮除以10，超参数S=9、C^e=256。\n- 在FreiHAND上，DenseStack/GhostStack版本分别取得7.55/8.89mm PAMPJPE；放大版ResNet50模型（224×224输入）达5.7mm PAMPJPE与0.784 F@5，创当时最优成绩。\n- 在HO3Dv2（含严重物体遮挡的挑战性数据集）上，DenseStack版本取得9.2mm PJ与0.538 F@5，优于ObMan、HO3D与I2UV-HandNet等方法，验证了遮挡场景下的泛化能力。\n- 时序一致性方面，MapReg结合3D/2D一致性损失使2D/3D Acc分别降至1.98/4.75，无需序列模块、时序优化或后处理即可实现稳定预测。\n- 局限与约束：DSConv增加内存访问成本，需工程优化以进一步提升推理速度；移动设备内存与计算预算有限；未来计划研究双手交互的高效重建方法。"
}