{
  "paper_id": "045196e8be19ffeb312166f5d5ed12e1",
  "paper_title": "LiDAR as a Geometric Prior: Enhancing Camera Pose Tracking Through High-Fidelity View Synthesis",
  "paper_authors": [
    "Rafael Muñoz-Salinas",
    "Jianheng Liu",
    "Francisco J. Romero-Ramírez",
    "Manuel J. Marín-Jiménez",
    "Fu Zhang"
  ],
  "publication_date": "2025-08-07",
  "publication_venue": "Applied Sciences (MDPI), 2025, 15, 8743",
  "abstract": "This paper presents a robust framework for monocular camera pose estimation by leveraging high-fidelity, pre-built 3D LiDAR maps. The core of our approach is a render-and-match pipeline that synthesizes photorealistic views from a dense LiDAR point cloud. By detecting and matching keypoints between these synthetic images and the live camera feed, we establish reliable 3D–2D correspondences for accurate pose estimation. We evaluate two distinct strategies: an Online Rendering and Tracking method that renders views on the fly, and an Offline Keypoint-Map Tracking method that precomputes a keypoint map for known trajectories, optimizing for computational efficiency. Comprehensive experiments demonstrate that our framework significantly outperforms several state-of-the-art visual SLAM systems in both accuracy and tracking consistency. By anchoring localization to the stable geometric information from the LiDAR map, our method overcomes the reliance on photometric consistency that often causes failures in purely image-based systems, proving particularly effective in challenging real-world environments.",
  "keywords": [
    "camera pose estimation",
    "LiDAR mapping",
    "3D reconstruction",
    "visual SLAM"
  ],
  "paper_institutions": [
    "Departamento de Informática e Inteligencia Artificial, Universidad de Córdoba, Spain",
    "Mechatronics and Robotic Systems (MaRS) Laboratory, Department of Mechanical Engineering, University of Hong Kong (HKU), Hong Kong, China",
    "Departamento de Teoría de la Señal y Comunicaciones y Sistemas Telemáticos y Computación, Universidad Rey Juan Carlos, Spain"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:34:56.672418Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n本文解决单目相机位姿估计中纯视觉方法依赖光度一致性、在挑战性真实环境中易失效的问题。相关研究涵盖视觉SLAM系统（ORB-SLAM2/3、UcoSLAM、OpenVSLAM）、LiDAR-视觉-惯性融合里程计（FAST-LIVO2）以及神经渲染技术（NeRF、M2Mapping、3D高斯泼溅）。提出的解决方案是“渲染-匹配”管线：利用FastLIVO2生成带颜色的稠密LiDAR点云，从中合成逼真视图，在合成图像与实时相机画面间检测并匹配关键点以建立可靠的3D-2D对应，通过最小化重投影误差 E(R,t)=Σ‖uʳ−π(R,t,X)‖² 的PnP问题（嵌入RANSAC求解）估计位姿，并以IMU积分预测作为每帧先验。框架包含两种策略：在线渲染跟踪逐帧实时渲染合成视图；离线关键点地图跟踪针对已知轨迹预计算稳定3D关键点地图，可选用M2Mapping提升渲染保真度。实验基于FastLivo2数据集的九个室外序列（Livox Avia LiDAR与RGB相机10 Hz、IMU 100 Hz、距离最长300米），以FastLIVO2位姿为真值，点云经2mm体素降采样与统计离群点滤除；硬件为Intel Xeon 4316 CPU与NVIDIA RTX 3090 GPU，采用AKAZE与ORB检测器并多线程并行化。评估指标为绝对轨迹误差（ATE）与跟踪帧比例（Trck）：AKAZE较ORB平均降低ATE约37%，在线方法ATE为0.031–0.107米、跟踪率78.4%–94.8%，跟踪环路达30 FPS（AKAZE）至80 FPS（ORB）、点云渲染16 FPS；在仅纳入跟踪率超75%系统的鲁棒对比中，所提方法于九个数据集中的五个取得最低ATE，并在HKUCulturalCenter序列上以0.063米成为唯一有效方案，验证了LiDAR几何先验相对视觉SLAM在精度与一致性上的优势。未来方向包括在线更新地图以应对动态物体、探索更先进渲染技术及适配热成像/事件相机。实际约束在于离线方法对轨迹偏离敏感、静态地图难以应对动态场景与突发遮挡、IMU无法预测的突变运动可能导致跟踪失败；该框架适用于自主导航、增强现实与3D场景重建等已知环境精确定位场景。\n\n### 关键要点\n- 核心机制为“渲染-匹配”管线：从预建稠密LiDAR点云渲染合成RGBD视图，与实时相机图像进行关键点检测与匹配，建立3D-2D对应后以PnP（RANSAC内嵌）最小化重投影误差求解位姿。\n- IMU测量被积分用于预测每帧相机位姿，作为视觉匹配的强先验，可约束特征搜索空间并提升鲁棒性。\n- 提出两种策略：在线渲染跟踪逐帧渲染合成视图，适合任意轨迹；离线关键点地图跟踪针对已知轨迹预计算关键点地图，将计算负担移出运行时，适合重复性任务的机器人应用。\n- 离线地图生成包含时序关键点过滤（仅保留连续帧中稳定可检测的关键点）与基于关键帧的地图结构，在线阶段仅检索最近关键帧并投影匹配候选3D点。\n- 实验采用FastLivo2数据集九个序列，由Livox Avia LiDAR与RGB相机（10 Hz）及IMU（100 Hz）同步采集，覆盖最长300米的多样室外环境，FastLIVO2位姿作为真值。\n- 点云经两阶段过滤：2mm体素网格降采样与统计离群点移除；渲染优化采用10cm体素网格分层视锥剔除，实现16 FPS的实时点云渲染。\n- 硬件为Intel Xeon Silver 4316 CPU（Ubuntu 20.04）与NVIDIA RTX 3090 GPU（24 GB），关键点检测三线程并行，跟踪环路约30 FPS（AKAZE，33.3ms延迟）至80 FPS（ORB，12.5ms延迟）。\n- AKAZE在全部九个数据集上一致优于ORB：平均ATE降低约37%，RedSculpture上误差降低近49%（0.107米对0.210米），HKUCulturalCenter上跟踪率高出12%以上。\n- IMU消融实验显示移除IMU后AKAZE仅轻微退化（如CBDBuilding ATE从0.048米增至0.057米），而ORB显著恶化（HITGraffitiWall从0.069米增至0.186米），表明IMU有益但配合强特征检测器时非严格必需。\n- M2Mapping渲染需约2小时/序列离线训练加约1小时渲染，不适合实时应用，但用于离线地图可小幅提升AKAZE跟踪鲁棒性与精度（如HITGraffitiWall ATE从0.050米降至0.025米）。\n- 与ORB-SLAM2/3、OpenVSLAM、UcoSLAM对比：在跟踪率超75%的鲁棒对比中，所提方法于九个数据集中的五个取得最低ATE，且在HKUCulturalCenter序列上是唯一达到阈值的方案（ATE 0.063米）。\n- 主要局限：离线方法对偏离预定轨迹高度敏感；依赖静态LiDAR地图，未显式建模动态物体，剧烈场景变化或突发遮挡会降低匹配可靠性；IMU无法预测的突变运动可能导致跟踪失败。"
}