{
  "paper_id": "7bf62cd0ccd57ab42513b5d92e786761",
  "paper_title": "SDV-LOAM: Semi-Direct Visual-LiDAR Odometry and Mapping",
  "paper_authors": [
    "Zikang Yuan",
    "Qingjie Wang",
    "Ken Cheng",
    "Tianyu Hao",
    "Xin Yang"
  ],
  "publication_date": "2023-03-29",
  "publication_venue": "IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)",
  "abstract": "Visual-LiDAR odometry and mapping (V-LOAM), which fuses complementary information of a camera and a LiDAR, is an attractive solution for accurate and robust pose estimation and mapping. However, existing systems could suffer nontrivial tracking errors arising from 1) association between 3D LiDAR points and sparse 2D features (i.e., 3D-2D depth association) and 2) obvious drifts in the vertical direction in the 6-degree of freedom (DOF) sweep-to-map optimization. In this paper, we present SDV-LOAM which incorporates a semi-direct visual odometry and an adaptive sweep-to-map LiDAR odometry to effectively avoid the above-mentioned errors and in turn achieve high tracking accuracy. The visual module of our SDV-LOAM directly extracts high-gradient pixels where 3D LiDAR points project on for tracking. To avoid the problem of large scale difference between matching frames in the VO, we design a novel point matching with propagation method to propagate points of a host frame to an intermediate keyframe which is closer to the current frame to reduce scale differences. To reduce the pose estimation drifts in the vertical direction, our LiDAR module employs an adaptive sweep-to-map optimization method which automatically chooses to optimize 3 horizontal DOF or 6 full DOF pose according to the richness of geometric constraints in the vertical direction. In addition, we propose a novel sweep reconstruction method which can increase the input frequency of LiDAR point clouds to the same frequency as the camera images, and in turn yield a high frequency output of the LiDAR odometry in theory. Experimental results demonstrate that our SDV-LOAM ranks 8th on the KITTI odometry benchmark which outperforms most LiDAR/visual-LiDAR odometry systems. In addition, our visual module outperforms state-of-the-art visual odometry and our adaptive sweep-to-map optimization can improve the performance of several existing open-sourced LiDAR odometry systems. Moreover, we demonstrate our SDV-LOAM on a custom-built hardware platform in large-scale environments which achieves both a high accuracy and output frequency. We have released the source code of our SDV-LOAM for the development of the community.",
  "keywords": [
    "Mapping",
    "pose estimation",
    "visual-LiDAR sensor system"
  ],
  "paper_institutions": [
    "Institute of Artificial Intelligence, Huazhong University of Science and Technology",
    "School of Electronic Information and Communications, Huazhong University of Science and Technology"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:40:34.945710Z",
  "template_tag": "simple",
  "summary": "### 摘要\n视觉-LiDAR里程计与建图（V-LOAM）通过融合相机与LiDAR的互补信息实现精确鲁棒的位姿估计与建图，但现有系统（如紧耦合代表V-LOAM及松耦合方法DEMO、LIMO、DVL-SLAM）存在两大误差来源：3D LiDAR点与稀疏2D特征关联时的深度关联误差，以及6自由度sweep-to-map优化中垂直方向的明显漂移（多数LiDAR系统如A-LOAM、LeGO-LOAM、Fast-LOAM、ISC-LOAM、MULLS因地面平面约束弱而受此困扰）。本文提出SDV-LOAM，由半直接深度增强视觉里程计与自适应sweep-to-map LiDAR里程计组成：视觉模块直接在3D LiDAR点投影处提取均匀分布的高梯度像素进行跟踪以消除深度关联误差，并设计带传播的点匹配方法将宿主帧点传播至更接近当前帧的中间关键帧以减小尺度差异，同时在纹理不足时自适应添加非深度点补充；LiDAR模块根据垂直方向几何约束丰富程度自适应选择优化3水平自由度或6全自由度位姿；此外提出sweep重建方法按相机图像时间戳重叠重建点云，将LiDAR输入频率提升至与相机相同。实验选用KITTI、KITTI-360（真实道路驾驶场景，涵盖城市/乡村/高速环境，验证真实场景精度与鲁棒性）、KITTI-CARLA（CARLA仿真数据，验证仿真环境泛化性）三个公开数据集及自建硬件平台（FL3-U3-13E4M-C相机60Hz与Velodyne VLP-16E LiDAR 10Hz，验证实时系统的频率提升能力），采用KITTI相对平移误差（RTE）作为指标，在Intel Core i7-11700 CPU上运行。结果表明：SDV-LOAM在KITTI训练集、KITTI-360、KITTI-CARLA上分别达0.47%、0.46%、0.038%的RTE，在KITTI测试集达0.60%排名第8（2023.01前），验证了系统整体精度；视觉模块平均RTE为0.72%，优于DEMO（低37.9%）、LIMO（低15.3%）、Huang等（低23.4%）和DVL-SLAM（低26.5%），验证了半直接方法的有效性；自适应sweep-to-map优化使六个开源LiDAR里程计精度普遍提升，验证了垂直约束自适应机制的价值。核心机制为点对面残差优化：r^{p^l} = ω_p·n^T(p^w − q)，其中p^w = R^w_{t_p}·p^l + t^w_{t_p}通过slerp插值旋转与线性插值平移获得，当垂直几何约束不足（地面表面占垂直表面比例M/N超过0.8）时仅优化3水平自由度变量[r_y, t_x, t_z]^T。未来方向包括在视觉BA中将LiDAR深度作为软约束。该系统适用于无人驾驶与自动导航等实际场景，但受计算资源限制实际输出频率约20Hz（理论60Hz），且在弱纹理场景（如KITTI-CARLA多数序列）中视觉模块会失效，外参标定误差与车辆振动也会影响变换积分法的轨迹一致性。\n\n### 关键要点\n- SDV-LOAM由半直接深度增强视觉里程计与自适应sweep-to-map LiDAR里程计组成，通过sweep重建模块连接，解决3D-2D深度关联误差与垂直方向漂移两大问题。\n- 视觉模块直接在LiDAR点投影处提取均匀分布的高梯度深度点进行跟踪，结合特征法的重投影误差最小化精度与直接法消除深度关联误差的优势。\n- 带传播的点匹配方法将宿主帧的点传播到更接近当前帧的中间关键帧再匹配，帧间隔为7关键帧时匹配误差从10.40像素降至6.77像素。\n- 点云预处理模块通过地面分割与RANSAC拟合计算地面点占比（阈值0.8）及不规则结构点占比（阈值0.5），自适应选择仅用深度点（模式1）或添加非深度点（模式2）进行跟踪。\n- LiDAR模块的自适应sweep-to-map优化根据垂直方向几何约束丰富程度（地面表面占垂直表面比例M/N是否超过0.8）自动选择优化3水平自由度或6全自由度位姿。\n- sweep重建方法按相机图像时间戳将10Hz原始点云重叠重建为与相机频率一致的重建sweep，使每个视觉模块输出位姿均可被LiDAR里程计优化，避免变换积分法因外参误差导致的轨迹不一致。\n- 在KITTI训练集、KITTI-360、KITTI-CARLA上分别达到0.47%、0.46%、0.038%相对平移误差，KITTI测试集0.60%排名第8（2023.01前），在LiDAR-only/单目视觉-LiDAR系统中排名第6，在开源系统中排名第2。\n- 视觉模块在KITTI序列00-10平均RTE为0.72%，比DEMO低37.9%、比LIMO低15.3%、比Huang等低23.4%、比DVL-SLAM低26.5%，运行时间0.06秒/帧，可实时运行于纯CPU平台。\n- 自适应sweep-to-map优化应用于A-LOAM、LeGO-LOAM、Fast-LOAM、ISC-LOAM、MULLS、CT-ICP六个开源系统后，在KITTI、KITTI-360、KITTI-CARLA几乎所有序列上精度显著提升。\n- 实验在Intel Core i7-11700 CPU与16GB RAM的消费级笔记本上进行，评估数据集涵盖真实道路（KITTI/KITTI-360，最高车速85km/h）与仿真（KITTI-CARLA）场景。\n- 自建硬件平台（FLIR FL3-U3-13E4M-C相机60Hz + Velodyne VLP-16 LiDAR 10Hz）的大规模户外实验验证了系统精度与sweep重建对输出频率的提升，但受计算资源限制实际输出约20Hz。\n- 系统约束包括：KITTI-CARLA仅测试序列01、02、07（其余弱纹理场景VO失败）、ISC-LOAM因序列过长无法在KITTI-360/KITTI-CARLA上完成全局图优化、自采数据无真值仅能定性评估；未来工作包括在视觉BA中将LiDAR深度作为软约束。"
}