{
  "paper_id": "602f3f26d0c37d6c6cf8367f8bdc0394",
  "paper_title": "R³LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package",
  "paper_authors": [
    "Jiarong Lin",
    "Fu Zhang"
  ],
  "publication_date": "",
  "publication_venue": "2022 International Conference on Robotics and Automation (ICRA)",
  "abstract": "In this paper, we propose a novel LiDAR-Inertial-Visual sensor fusion framework termed R³LIVE, which takes advantage of measurement of LiDAR, inertial, and visual sensors to achieve robust and accurate state estimation. R³LIVE consists of two subsystems, a LiDAR-Inertial odometry (LIO) and a Visual-Inertial odometry (VIO). The LIO subsystem (FAST-LIO) utilizes the measurements from LiDAR and inertial sensors and builds the geometric structure (i.e., the positions of 3D points) of the map. The VIO subsystem uses the data of Visual-Inertial sensors and renders the map's texture (i.e., the color of 3D points). More specifically, the VIO subsystem fuses the visual data directly and effectively by minimizing the frame-to-map photometric error. The proposed system R³LIVE is developed based on our previous work R²LIVE, with a completely different VIO architecture design. The overall system is able to reconstruct the precise, dense, 3D, RGB-colored maps of the surrounding environment in real-time. Our experiments show that the resultant system achieves higher robustness and accuracy in state estimation than its current counterparts. To share our findings and make contributions to the community, we open source R³LIVE on our Github.",
  "keywords": [
    "SLAM",
    "LiDAR-Inertial-Visual fusion",
    "state estimation",
    "RGB-colored mapping",
    "photometric error",
    "error-state iterated Kalman filter"
  ],
  "paper_institutions": [],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:33:35.261009Z",
  "template_tag": "simple",
  "summary": "### 摘要\n针对LiDAR-SLAM系统在几何特征不足场景（尤其是视场受限的固态LiDAR）下容易失效的问题，以及现有LiDAR-惯性-视觉融合方案（如松耦合的V-LOAM、DV-LOAM，紧耦合的LIC-Fusion、LVI-SAM及作者前期工作R²LIVE）在鲁棒性、实时性与精度上的局限，本文提出紧耦合传感器融合框架R³LIVE，由构建地图几何结构的LiDAR-惯性里程计（LIO，基于FAST-LIO，通过误差状态迭代卡尔曼滤波ESIKF最小化点到平面残差）与渲染地图纹理的视觉-惯性里程计（VIO）组成；VIO采用全新架构，先以Lucas-Kanade光流跟踪地图点并最小化PnP重投影误差，再以核心机制——帧到地图光度误差 o(x,p,c)=c−γ（c为地图点存储的RGB颜色，γ为当前图像观测颜色；颜色作为地图点固有属性对相机平移与旋转均不变）——在ESIKF框架内迭代精化状态，最后通过贝叶斯更新融合观测颜色完成纹理渲染，无需显著视觉特征提取，从而在无纹理环境中更加鲁棒。实验基于自研手持设备（DJI Manifold-2c平台、Intel i7-8550u CPU、FLIR Blackfly全局快门相机、Livox AVIA LiDAR，另装D-GPS RTK与ArUco标记板提供真值）采集数据：狭窄\"T\"形通道（同时LiDAR退化与视觉无纹理）用于验证鲁棒性，香港科技大学校园四条1191–1524米轨迹（大尺度室内外复杂环境）与海港序列（含行人干扰与开阔海面导致LiDAR测量稀少）用于验证大尺度建图精度与定量评估。结果表明：退化场景中漂移仅1.62°旋转与4.57厘米平移且估计协方差全程有界；长达1.5公里轨迹上平移漂移仅0.16米、旋转3.9°；以相对旋转/平移误差（RRE/RTE）度量，R³LIVE优于LVI-SAM、R²LIVE、VINS-Mono与FAST-LIO2，验证了其架构创新的有效性；VIO每帧耗时在PC上为7–23毫秒、机载平台15–61毫秒，取决于图像与点云分辨率。未来工作将加入在线光度标定以进一步提升性能与鲁棒性；系统适用于自动导航等需要实时稠密彩色地图的真实场景，但受计算资源与分辨率设置约束，代码、机械设计与数据集均已开源。\n\n### 关键要点\n- R³LIVE由LIO子系统（基于FAST-LIO，最小化点到平面残差以构建全局地图几何结构）与VIO子系统（渲染地图RGB颜色纹理）组成，整体可实时重建稠密、精确的3D RGB彩色点云地图。\n- VIO采用两步流水线：先用Lucas-Kanade光流进行帧间跟踪并最小化PnP重投影误差估计状态，再最小化帧到地图光度误差进一步降低漂移。\n- 核心机制为帧到地图光度误差 o(x,p,c)=c−γ，即地图点存储颜色与当前图像观测颜色之差；颜色是地图点固有属性，对相机平移与旋转均不变，因此无需构建图像金字塔。\n- 系统状态向量为29维，包含IMU姿态、位置、速度、陀螺仪与加速度计偏置、重力、相机-IMU外参、相机-IMU时间偏移及相机内参，均在ESIKF框架内在线估计。\n- 地图由固定大小体素（如0.1m×0.1m×0.1m）与6维点（3D位置+RGB颜色）构成，体素依据近期是否有点追加标记为激活/非激活，以加速渲染与跟踪时的点检索。\n- 纹理渲染通过贝叶斯更新融合新观测颜色与地图已有颜色，并将环境光照变化建模为脉冲随机游走过程噪声纳入协方差计算。\n- 实验一在同时LiDAR退化与视觉无纹理的狭窄\"T\"形通道中验证鲁棒性：最终漂移仅1.62°旋转与4.57厘米平移，估计协方差在整个轨迹上有界。\n- 实验二在香港科技大学校园采集四条轨迹（1191–1524米），无需回环检测等额外处理即可自闭合，平移漂移0.093–0.164米、旋转漂移0.285–3.925°。\n- 实验三在海港（Belcher Bay Promenade）以D-GPS RTK为真值，R³LIVE在相对旋转/平移误差（RRE/RTE）指标上优于LVI-SAM、R²LIVE、VINS-Mono与FAST-LIO2。\n- 运行时间分析表明VIO每帧耗时主要取决于图像尺寸与点云地图分辨率：PC（i7-9700K）上为7.08–23.17毫秒，机载平台（i7-8550u）上为15.30–61.34毫秒。\n- 数据采集使用自研手持设备（FLIR Blackfly全局快门相机、Livox AVIA LiDAR、DJI Manifold-2c计算平台），机械模块均为FDM可打印设计，代码、原理图与数据集已在Github开源。\n- 未来工作计划加入在线光度标定，以进一步提升R³LIVE的性能与鲁棒性。"
}