{
  "paper_id": "d359444af5ecdfb5fb4c977928edfb1e",
  "paper_title": "GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multisensor Fused Odometry With Gaussian Mapping",
  "paper_authors": [
    "Sheng Hong",
    "Chunran Zheng",
    "Yishu Shen",
    "Changze Li",
    "Fu Zhang",
    "Tong Qin",
    "Shaojie Shen"
  ],
  "publication_date": "",
  "publication_venue": "IEEE Transactions on Robotics",
  "abstract": "In recent years, 3-D Gaussian splatting (3D-GS) has emerged as a novel scene representation approach. However, existing vision-only 3D-GS methods often rely on hand-crafted heuristics for point-cloud densification and face challenges in handling occlusions and high graphics processing unit (GPU) memory and computation consumption. Light detection and ranging (LiDAR)-inertial-visual sensor configuration has demonstrated superior performance in precise localization and dense mapping by leveraging complementary sensing characteristics: rich texture information from cameras, precise geometric measurements from LiDAR, and high-frequency motion data from inertial measurement unit. Inspired by this, we propose a novel real-time Gaussian-based simultaneous localization and mapping system. Our map system comprises a global Gaussian map and a sliding window of Gaussians, along with an iterative error state Kalman filter (IESKF)-based real-time odometry utilizing Gaussian maps. The structure of the global Gaussian map consists of hash-indexed voxels organized in a recursive octree. This hierarchical structure effectively covers sparse spatial volumes while adapting to different levels of detail and scales in the environment. The Gaussian map is efficiently initialized through multisensor fusion and optimized with photometric gradients. Our system incrementally maintains a sliding window of Gaussians with minimal graphics memory usage, significantly reducing GPU computation and memory consumption by only optimizing the map within the sliding window, enabling real-time optimization. Moreover, we implement a tightly coupled multisensor fusion odometry with an IESKF, which leverages real-time updating and rendering of the Gaussian map to achieve competitive localization accuracy. Our system represents the first real-time Gaussian-based SLAM framework deployable on resource-constrained embedded systems (all implemented in C++/CUDA for efficiency), demonstrated on the NVIDIA Jetson Orin NX platform. The framework achieves real-time performance while maintaining robust multisensor fusion capabilities. All implementation algorithms, hardware designs, and CAD models and demo video of our GPU-accelerated system will be publicly available.",
  "keywords": [
    "Gaussian splatting",
    "multisensor fusion",
    "odometry"
  ],
  "paper_institutions": [
    "Hong Kong University of Science and Technology",
    "University of Hong Kong",
    "Shanghai Jiao Tong University"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "glm-5.3",
  "prompt_template": "simple",
  "extracted_at": "2026-09-17T09:31:02.454383Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n现有纯视觉3D高斯泼溅(3D-GS)SLAM方法依赖手工启发式进行点云致密化,难以处理遮挡且GPU显存与计算开销高昂;NeRF类(iMAP、NICE-SLAM、Co-SLAM等)与3D-GS类(SplaTAM、MonoGS、GS-SLAM等)神经SLAM虽提升了地图保真度,但其地图优化与更新常滞后于位姿估计,难以在动态和大尺度环境中维持真正实时的地图刷新,而LiDAR-惯性-视觉(LIV)融合虽具备互补感知优势,传统表示(彩色点云、稀疏patch、surfel)却无法提供照片级重建与新视角合成。为此,本文提出GS-LIVO——首个可部署于资源受限嵌入式平台的实时高斯SLAM系统:全局高斯地图以空间哈希索引的递归八叉树组织,高效覆盖稀疏空间体素并支持多细节层次(LoD);高斯经LiDAR-相机联合快速初始化(缩放由体素层级决定、旋转由表面法向构造、颜色由双线性插值初始化球谐零阶系数)后以光度梯度在线优化;增量维护的高斯滑动窗口(空间哈希表SHT+CPU缓冲CGB+GPU缓冲GGB)仅优化当前视场内高斯,大幅降低GPU计算与显存消耗;里程计采用序贯更新的迭代误差状态卡尔曼滤波(IESKF)紧耦合多传感器,其核心机制为最小化高斯渲染图像与真实图像间的光度残差 ^W T*_C = argmin_{T(ξ)} Σ_{G_i^3D∈θ_{k-1}} ||I_k − Î_k(^W T_C; θ_{k−1})||,并推导光度损失对IMU位姿的解析雅可比,使位姿及其协方差可在滤波框架内传播至后续传感器更新。实验选用FAST-LIVO2数据集(CBD03、HKU01,大尺度校园场景)、MARS-LVIG机载数据集(HKairport01、HKisland03,含D-RTK厘米级真值,验证大尺度自然与航拍环境)、Oxford Spires(Radcliffe01,1–2cm LiDAR-TLS真值)及动作捕捉真值的自采室内序列(Playground01/02,受控条件下的精度评估),并在高性能桌面(Intel i9-13900KF、128GB RAM、RTX 4090)与NVIDIA Jetson Orin NX嵌入式平台(全部以C++/CUDA实现)上部署,以验证精度、效率与资源受限实时性。结果表明:渲染方面GS-LIVO在HKU01上以82.5s优化时间与2.2GB内存取得25.34dB PSNR,在CBD03上以88.4s、2.2GB取得27.52dB,相较3D-GS(CBD03需1873.8s、12.5GB)与LetsGo(3573.6s、20.4GB)大幅降低时间与显存开销,印证了八叉树结构与滑动窗口策略的有效性;定位方面HKisland03的RMSE为0.58m,显著优于R3LIVE(1.71m)与LVI-SAM(4.12m),与FAST-LIVO(0.51m)相当,室内序列RMSE低至0.005–0.006m,且处理时间室内外均低于100ms(室内约10Hz、室外约3Hz地图更新),支撑了IESKF紧耦合融合与LiDAR-惯性先验的引入;消融实验进一步证明滑动窗口在维持约25dB PSNR的同时显著降低显存与耗时;在Jetson Orin NX上以48.3ms总耗时、23.52dB PSNR实现实时运行,验证了嵌入式部署能力。系统还被集成至完整自主导航流程(高斯地图生成二维占据栅格供A*全局规划,里程计供LQR轨迹跟踪),展示了真实机器人场景的适用性;主要约束在于室内外过渡场景表现欠佳,未来可通过尺寸自适应体素动态调整八叉树细分层级加以改进。\n\n### 关键要点\n\n- 全局高斯地图采用空间哈希索引的递归八叉树结构,根体素键由 HashKey=⌊^W p_i/v_s⌋ 计算,支持高效全局索引、协视性检查与多细节层次(LoD),适应大尺度稀疏场景。\n- 高斯初始化融合LiDAR与视觉数据:缩放矩阵依据哈希体素层级设定(法向方向取小值 s_δ 形成贴合物体表面的薄平面高斯),旋转矩阵由LiDAR-惯性系统导出的表面法向 ^W n_i 构造,零阶球谐系数由图像双线性插值初始化。\n- 高斯滑动窗口由空间哈希表(SHT)、CPU高斯缓冲(CGB)与GPU高斯缓冲(GGB)构成,通过四步增量流程(回写全局地图、删除与压缩、重叠识别与新增、追加叶子体素并传输至GPU)仅更新可见性变化的体素,避免整窗重建的冗余内存拷贝并缓解深度排序混叠问题。\n- 里程计基于IESKF序贯更新紧耦合LiDAR-惯性-视觉测量:视觉测量模型直接在当前帧上比较高斯渲染图像与真实图像的光度残差,并推导光度损失对IMU位姿的解析雅可比,使位姿及其协方差可传播至后续IMU与LiDAR更新,形成真正紧耦合的滤波系统。\n- 实验数据涵盖FAST-LIVO2的CBD03/HKU01(大尺度校园)、MARS-LVIG的HKairport01/HKisland03(机载山地海域场景,D-RTK真值)、Oxford Spires的Radcliffe01(1–2cm LiDAR-TLS真值)及动作捕捉真值的自采Playground01/02室内序列,兼顾大尺度室外、航拍与受控室内评估。\n- 渲染质量方面,GS-LIVO在HKU01上以82.5s、2.2GB内存达到25.34dB PSNR,在CBD03上以88.4s、2.2GB达到27.52dB,相比3D-GS(CBD03需1873.8s、12.5GB)、LetsGo(3573.6s、20.4GB)与S3Gaussian(3450.7s、15.2GB)大幅降低时间与显存开销。\n- 定位精度方面,室外HKisland03序列RMSE为0.58m,显著优于R3LIVE(1.71m)与LVI-SAM(4.12m),与FAST-LIVO(0.51m)相当;室内Playground01/02序列RMSE为0.006m/0.005m,处理时间48.5ms/63.4ms。\n- 与SplaTAM、MonoGS等高斯SLAM相比,GS-LIVO在Playground01上以48.5ms、1.2GB内存取得0.006m RMSE,而SplaTAM需612.8ms、12.5GB且RMSE为0.28m,MonoGS需541.5ms、21.0GB且RMSE为0.18m,且两者均无法在室外大尺度序列上运行。\n- 消融实验表明滑动窗口策略在室内外均维持约25dB的PSNR(视角切换后经迭代优化恢复至25–30dB),总处理时间稳定低于100ms,而不用滑动窗口时计算时间随地图增长持续上升,难以维持实时性。\n- 系统在NVIDIA Jetson Orin NX(16GB)上以0.5m根体素、两层细分、256×216图像分辨率与2万高斯滑动窗口配置实现实时运行:优化15.3ms、地图维护18.9ms、总流水线48.3ms,PSNR达23.52dB,是首个部署于ARM嵌入式平台的实时高斯SLAM系统。\n- 系统已集成至完整自主导航流程:高斯地图被处理生成二维占据栅格供A*全局路径规划,里程计提供实时定位供LQR轨迹跟踪,演示视频验证了实际机器人场景的可用性。\n- 主要局限在于室内外过渡场景表现欠佳,作者提出采用尺寸自适应体素(依据场景动态调整八叉树细分层级)作为未来研究方向。"
}