.
## Keywords
LiDAR-Inertial-Camera SLAM, Multi-Sensor Fusion, Photo-Realistic Dense Mapping
## 1 Introduction
Simultaneous localization and mapping (SLAM) serves as a fundamental technology that facilitates spatial perception in both mixed reality systems and robotic applications. Recent advances in radiance field representations, particularly Neural Radiance Fields (NeRF) Mildenhall et al. (2021) and 3D Gaussian Splatting (3DGS) Kerbl et al. (2023), have pioneered a new paradigm in SLAM, namely radiance-field-based SLAM Tosi et al. (2024). Powered by differentiable photo-realistic rendering, radiance field-based SLAM systems aim to provide both accurate pose estimation and photo-realistic 3D maps in real time, equipping robots with richer 3D scene understanding. These systems can benefit a wide range of tasks, including path planning Jin et al. (2024); Chen et al. (2025b); Lei et al. (2025), active exploration and mapping Chen et al. (2025a), and 3D mesh reconstruction Zhang et al. (2024b).
Initially, NeRF-based SLAM systems Sucar et al. (2021); Zhu et al. (2022); Yang et al. (2022); Wang et al. (2023a); Johari et al. (2023) leverage multi-layer perceptrons (MLPs), often further integrating learnable encodings, to represent the entire scene and generate high-quality dense maps with low memory consumption. Nonetheless, such implicit representations necessitate computationally intensive volumetric rendering based on sampling in 3D space, undermining the real-time capability, which is essential for robot applications. The emergence of 3DGS has shifted this landscape. It features fast rendering along with good visual quality and demonstrates greater potential for real-time use. Equipped with RGB-D or RGB sensors, most 3DGS-based SLAM systems Yan et al. (2024a); Keetha et al. (2024); Yugay et al. (2023); Matsuki et al. (2024); Huang et al. (2024b) focus on indoor environments and outperform NeRF-based approaches, but unfortunately struggle in challenging conditions such as violent ego-motion, varying illumination, and lack of visual textures, commonly arising in unbounded outdoor scenes. Several works Sun et al. (2024); Wu et al.
---
Figure 1. Overview of the system outputs (ordered left to right, top to bottom): (1) reconstructed Gaussian map, (2) RGB image rendered from a novel viewpoint, (3) sparse LiDAR point cloud map, (4) depth map rendered from the novel viewpoint, and (5) 3D mesh extracted from the Gaussian map.
(2024a); Xiao et al. (2024); Xie et al. (2025); Zhao et al. (2025); Hong et al. (2025) and our previous work Lang et al. (2025) address the issue by fusing LiDAR and IMU, among which the effectiveness of LiDAR-Inertial-Camera fusion has been extensively validated in traditional SLAM methods Zhang and Singh (2018); Zuo et al. (2019, 2020); Shan et al. (2021); Lin and Zhang (2022, 2024); Zheng et al. (2022, 2024); Yuan et al. (2024); Lv et al. (2023); Lang et al. (2023); Cong et al. (2024); Wu et al. (2025). Notably, the precise geometric priors provided by LiDAR contribute to both pose tracking and Gaussian mapping, and the significantly reduced cost of LiDARs nowadays has made them much more accessible for integration.
Despite recent efforts to incorporate Gaussian Splatting Kerbl et al. (2023) into LiDAR-Inertial-Camera SLAM systems Lang et al. (2025); Xiao et al. (2024); Xie et al. (2025); Zhao et al. (2025); Hong et al. (2025), several key issues remain unsolved or insufficiently addressed. First, current methods predominantly rely on the dense LiDAR with the FoV closely aligned with that of the camera, initializing Gaussians solely from LiDAR points. This can lead to underreconstruction in LiDAR blind spots, especially when the sparse LiDAR is used. Although adaptive density control (ADC) can alleviate this limitation by cloning and splitting Gaussians based on the magnitude of back-propagated gradients and scales of Gaussians Kerbl et al. (2023), it struggles in the incremental SLAM systems involving optimization of Gaussians with different maturity. ADC also requires aggregating gradients over multiple iterations of optimization, which leads to a lack of timeliness for real-time SLAM systems. Second, despite leveraging LiDAR data with precise geometric information, existing SLAM methods overemphasize the visual quality of the map while neglecting its geometric reconstruction quality, which limits its applicability in geometry-critical tasks, like obstacle avoidance. Pursuing both high-quality RGB and depth rendering while maintaining real-time performance remains a great challenge. Third, current approaches usually decouple LiDAR-Inertial-Camera odometry and Gaussian mapping. It is underexplored whether the incrementally constructed Gaussian map can benefit the traditional odometry during system operation. Furthermore, most methods focus merely on rendering quality from training views, overlooking the novel view synthesis capability in sequence or even out of sequence. Admittedly, LiDAR-Inertial-Camera SLAM datasets that enable joint evaluation of RGB and depth rendering across both in-sequence and out-of-sequence novel views are still scarce.
Evolved from our conference version, Gaussian-LIC Lang et al. (2025), this journal extension addresses the aforementioned challenges by proposing a real-time, geometry-aware, and photo-realistic SLAM system, termed Gaussian-LIC2. The proposed system achieves robust and accurate pose estimation while constructing photorealistic 3D Gaussian maps that capture both high-fidelity visual appearance and precise geometric structure. Main contributions are as follows:
• We propose the first LiDAR-Inertial-Camera Gaussian Splatting SLAM system that jointly takes care of visual quality, geometric accuracy, and real-time performance. It is capable of robustly and precisely estimating poses while constructing a photo-realistic and geometrically accurate 3D Gaussian map, all in real time.
- We propose integrating LiDAR and visual data through an efficient, lightweight, generalizable sparse depth completion network to rapidly predict depths for image patches not covered by the LiDAR, enabling more comprehensive 3D Gaussian initialization and mitigating under-reconstruction. During training, we fully leverage LiDAR-provided depth for supervision and accelerate the process with a series of carefully designed C++ and CUDA implementations.
- We explore jointly fusing photometric constraints from the incrementally built Gaussian map with raw LiDAR-Inertial data in a continuous-time framework, successfully helping the odometry overcome the LiDAR degeneration. Besides, we have extended our system to enable Gaussian map utilization for downstream applications such as video frame interpolation and rapid mesh generation.
• We curate a specialized LiDAR-Inertial-Camera dataset with ground-truth poses and depths, as well as carefully designed capturing trajectories, enabling the evaluation of out-of-sequence novel view synthesis. We conduct extensive experiments on public and self-collected datasets, proving the superiority of our approach and its adaptability to various types of LiDARs.
## 2 Related Works
### 2.1 Photo-Realistic Reconstruction with Radiance Field
Map representation deeply affects architecture designs and potential downstream applications of SLAM systems. Sparse SLAM approaches Qin et al. (2018); Campos et al. (2021) excel in pose estimation but yield only sparse keypoint
---
maps. On the contrary, dense SLAM methods produce dense maps beneficial for scene understanding. For instance, DTAM Newcombe et al. (2011b), REMODE Pizzoli et al. (2014), LSD-SLAM Engel et al. (2014), DSO Engel et al. (2017), DROID-SLAM Teed and Deng (2021), etc, can achieve accurate camera pose tracking and reconstruct dense point cloud maps. KinectFusion Newcombe et al. (2011a), ElasticFusion Whelan et al. (2015), and SurfelMeshing Schöps et al. (2019) model 3D scenes using truncated signed distance function (TSDF), surfel, and mesh, all of which are commonly utilized in the SLAM field. However, it is challenging to recover photo-realistic camera views from these map representations. Fortunately, the emergence of NeRF Mildenhall et al. (2021) has brought a promising solution. As a novel radiance field representation, NeRF implicitly models the geometry and texture of the scene through MLPs and achieves differentiable photo-realistic rendering even at novel views. To accelerate the training, Plenoxels Fridovich-Keil et al. (2022) optimizes spherical harmonics and density directly on a sparse voxel grid without neural networks, and instant-ngp Müller et al. (2022) introduces a multiresolution hash encoding paired with a tiny GPU-accelerated neural network. To improve the geometric accuracy of NeRF representation and extract a precise 3D mesh from it, VolSDF Yariv et al. (2021), NeuS Wang et al. (2021), and NeuS2 Wang et al. (2023b) replace the density field with signed distance function (SDF).
However, NeRF still falls slightly short in real-time reconstruction due to the computationally intensive ray-based volume rendering. In contrast, 3D Gaussian Splatting (3DGS) Kerbl et al. (2023) explicitly represents scenes using view-dependent anisotropic Gaussians and introduces a tile-based rasterization strategy for splats, achieving faster rendering speed while maintaining superior visual quality. Built upon 3DGS, a number of follow-up works have been proposed to further enhance its performance and flexibility. Scaffold-GS Lu et al. (2024) incorporates tiny MLPs to predict the properties of neural Gaussians, enabling more compact and expressive representations. Meanwhile, SuGaR Guédon and Lepetit (2024) and GOF Yu et al. (2024b) focus on efficient mesh extraction from 3DGS representations, facilitating downstream tasks such as geometry processing and simulation. 2DGS Huang et al. (2024a), GaussianSurfels Dai et al. (2024), and PGSR Chen et al. (2024a) flatten the Gaussians to accurately conform to the scene surface, while RaDe-GS Zhang et al. (2024a) introduces an enhanced depth rasterizing approach without the reformulation of the Gaussian primitives. NeuSG Chen et al. (2023a) and GSDF Yu et al. (2024a) try to combine neural SDF with 3DGS using mutual guidance and joint supervision.
Original NeRF and 3DGS methods typically rely solely on image data for photo-realistic reconstruction. However, the incorporation of LiDAR, which has become increasingly affordable and accessible, can significantly enhance the performance of both NeRF and 3DGS, particularly in unbounded outdoor environments. For NeRF, LiDAR data can guide ray sampling around surfaces and provide accurate depth supervision during optimization. URF Rematas et al. (2022) and EmerNeRF Yang et al. (2023) are NeRF variants tailored for autonomous driving scenarios and achieve outstanding rendering quality with the aid of LiDAR. Bootstrapped by a LiDAR SLAM system, SiLVR Tao et al. (2024) constructs multiple NeRF submaps efficiently. M2Mapping Liu et al. (2024a) further unifies surface reconstruction and photo-realistic rendering in LiDAR-Camera systems through an SDF-based NeRF formulation. In the context of 3DGS, LiDAR data can be leveraged not only for geometric supervision but also for the accurate and efficient initialization of Gaussians. PVG Chen et al. (2023b), DrivingGaussian Zhou et al. (2024a), StreetGaussians Yan et al. (2024b), and TCLC-GS Zhao et al. (2024) are the first to introduce LiDAR into the 3DGS framework, effectively modeling both static scenes and dynamic objects in autonomous driving environments. LIV-GaussMap Hong et al. (2024) and LetsGo Cui et al. (2024) both utilize the point cloud and poses from LiDAR-based SLAM to initialize 3DGS. Focusing more on geometric quality, LI-GS Jiang et al. (2024) employs 2DGS as the map representation to enhance surface alignment, while GS-SDF Liu et al. (2025) incorporates LiDAR into SDF-based 3DGS.
The aforementioned works are all per-scene optimization frameworks. Interestingly, a series of feed-forward models Yu et al. (2021); Chen et al. (2021); Charatan et al. (2024); Chen et al. (2024b) have emerged and achieved generalizable photorealistic reconstruction in an end-to-end manner. However, their accuracy still lag behind that of per-scene optimization methods, and their applicability is limited to a small number of high-quality images with minimal viewpoint variation.
### 2.2 Incremental Visual SLAM Systems with Radiance Field
Radiance-field-based reconstruction, whether per-scene optimized or feed-forward, is essentially an offline process with all the collected data accessible. In comparison, radiance-field-based SLAM systems incrementally perform photo-realistic reconstruction with sequentially input sensor data, utilizing radiance file map representations.
Given sequential RGB-D inputs, iMAP Sucar et al. (2021) is a pioneering work built upon the implicit neural representation to achieve watertight online reconstruction. Following this, NICE-SLAM Zhu et al. (2022) is able to scale up to larger indoor scenes by combining MLP representation with hierarchical feature grids. Vox-Fusion Yang et al. (2022) further adopts the octree to dynamically expand the volumetric map, eliminating the need for pre-allocated grids. Leveraging hash grids, tri-planes, and neural point cloud as their respective implicit representations, Co-SLAM Wang et al. (2023a), ESLAM Johari et al. (2023), and Point-SLAM Sandström et al. (2023) get enhancement in both localization and mapping. Moreover, H2-Mapping Jiang et al. (2023) handles the forgetting issue by a novel coverage-maximizing keyframe selection strategy. In terms of 3DGS-based SLAM, GS-SLAM Yan et al. (2024a), SplaTAM Keetha et al. (2024), and Gaussian-SLAM Yugay et al. (2023) demonstrate the advantages of 3DGS over existing map representations in SLAM systems for online photo-realistic mapping. By forcing binary opacity for each Gaussian, RTG-SLAM Peng et al. (2024) achieves real-time performance indoors with the compact scene representation. GSFusion Wei and Leutenegger (2024) jointly constructs the Gaussian map with a TSDF map and uses a quadtree data structure to reduce the number of Gaussians. MM3DGS-SLAM Sun et al. (2024)
---
loosely fuses RGB-D and IMU measurements to enable more robust and precise pose estimation.
A range of studies have also explored operating solely on monocular RGB images, among which NICER-SLAM Zhu et al. (2024b) and MonoGS Matsuki et al. (2024) are the representative radiance-field-map-centric approaches. The former fully makes use of the monocular geometric cues for supervision, and the latter introduces the isotropic regularization to address ambiguities in incremental reconstruction. However, decoupled tracking and mapping methods, which adopt the state-of-the-art visual odometry for pose tracking and radiance field optimization for photo-realistic mapping in parallel, usually demonstrate much more robustness. Orbeez-SLAM Chung et al. (2023) and Photo-SLAM Huang et al. (2024b) are both built upon ORB-SLAM Campos et al. (2021). NeRF-SLAM Rosinol et al. (2023) and IG-SLAM Sarikamis and Alatan (2024) utilize the dense depth maps estimated from the tracking front-end DROID-SLAM as additional information to supervise the training of instant-ngp and 3DGS. NeRF-VO Naumann et al. (2024) and MGS-SLAM Zhu et al. (2024a) instead employ the sparse DPVO Teed et al. (2023) as a faster tracker with network-predicted dense depth maps for supervision.
### 2.3 LiDAR-Integrated SLAM Systems with Radiance Field
Radiance-field-based Visual SLAM has achieved great performance in confined indoor scenes, and fusing IMU data can further improve the robustness. But they are still challenged by extreme violent motions, drastic lighting changes, and texture deficiency. Studies in both radiance-field-based reconstruction Rematas et al. (2022); Yang et al. (2023); Tao et al. (2024); Liu et al. (2024a); Chen et al. (2023b); Zhou et al. (2024a); Yan et al. (2024b); Zhao et al. (2024a); Hong et al. (2024b); Cui et al. (2024b); Jiang et al. (2024b); Liu et al. (2025); Zuo et al. (2020); Han et al. (2021); Lin and Zhang (2022, 2024); Zheng et al. (2022, 2024); Yuan et al. (2024); Lv et al. (2023); Lang et al. (2023); Li DARs. For radiance-field-based SLAM, the advantages of integrating LiDAR include, at the very least, enhanced robustness and accuracy in pose estimation, precise geometric supervision, facilitating efficient sampling for NeRF and accurate initialization for 3DGS.
Powered by the LiDAR data, SHINE-Mapping Zhong et al. (2023) is an incremental mapping framework that uses octree-based hierarchical neural SDF to perform large-scale 3D reconstruction in a memory-efficient way. Meanwhile, NF-Atlas Yu et al. (2023) organizes multiple neural submaps by a pose graph, and N3-Mapping Song et al. (2024) applies a voxel-oriented sliding window mechanism to alleviate the forgetting issue with a bounded memory footprint. Instead of purely geometric mapping, HGS-Mapping Wu et al. (2024b) incrementally builds a dense photo-realistic map with hybrid Gaussians in urban scenarios. All these mapping systems assume a priori ground-truth poses, whereas full SLAM systems that estimate poses simultaneously are inherently more complicated. NeRF-LOAM Deng et al. (2023) is a typical NeRF-based LiDAR odometry and mapping method, optimizing poses and voxel embeddings concurrently. Furthermore, LONER Isaacson et al. (2023) proposes a novel information-theoretic loss function to attain real-time performance. Benefiting from a semi-explicit representation, PIN-SLAM Pan et al. (2024) utilizes a neural point cloud map representation with elasticity for globally consistent mapping. Splat-LOAM Giacomini et al. (2025) exploits explicit 2DGS primitives with spherical projection for localization and LiDAR mapping.
Going beyond merely recovering the geometric structure of the scene, Rapid-Mapping Zhang et al. (2024b) utilizes NeRF to represent and render detailed scene textures. MM-Gaussian Wu et al. (2024a), LiV-GS Xiao et al. (2024), and PINGS Pan et al. (2025) perform coupled pose tracking and dense mapping within a unified 3DGS optimization framework using LiDAR-Camera data. While these methods demonstrate impressive performance, they are not capable of real-time operation, despite the inherent efficiency of 3DGS. By comparison, decoupled tracking and Gaussian mapping approaches have shown promising results. Our prior work Gaussian-LIC Lang et al. (2025) is the first real-time LiDAR-Inertial-Camera 3DGS-based SLAM system, combining continuous-time LiDAR-Inertial-Camera odometry for tracking with 3DGS for mapping, achieving high-fidelity photo-realistic reconstruction in real time. Subsequently, GS-LIVM Xie et al. (2025), LVI-GS Zhao et al. (2025), and GS-LIVO Hong et al. (2025) have also been built at top classical LiDAR-Inertial-Camera odometry. Among them, GS-LIVM and GS-LIVO both adopt voxel-based map structures and initialize Gaussian covariances.
### 2.4 Extension over Gaussian-LIC
This paper is evolved from our conference version, Gaussian-LIC Lang et al. (2025), and further introduces important systemic and technical contributions beyond it.
Systemically, to the best of our knowledge, Gaussian-LIC2 is the first LiDAR-Inertial-Camera Gaussian Splatting SLAM system which harmonizes high-fidelity visual realism, accurate geometry, and real-time efficiency. In contrast, Gaussian-LIC, as well as GS-LIVM, and GS-LIVO, focus primarily on visual quality or real-time performance at most.
Technically, there are enhancements in three key aspects:
- Accommodate LiDAR Blind Area: GS-LIVM, LVIGS, and GS-LIVO create Gaussians solely from LiDAR data, which can lead to under-reconstruction in LiDAR blind areas. Gaussian-LIC alleviates this limitation by incorporating visual features triangulated online; however, it struggles in textureless scenes and is sensitive to parallax between views, which degrades triangulation reliability. Gaussian-LIC2 leverages visual cues more effectively by fusing RGB appearance with sparse yet geometrically accurate LiDAR measurements through an efficient depth completion model. This enables the generation of accurate dense depth maps and helps better initialize Gaussians for carefully selected pixels unobserved by the LiDAR. Further insights are provided in Sec. 6.2.
• Geometric Accuracy with Visual Quality and Real-Time Performance as Prerequisites: Visual fidelity, real-time efficiency, and geometric accuracy — satisfying any single one is readily achievable, but balancing
---
all three is challenging. As a LiDAR-Inertial-Camera 3DGS-based SLAM system, Gaussian-LIC is the first to attain high visual quality in real time. Nevertheless, geometric accuracy, which affects the quality of rendered depth and the robustness of novel view synthesis, has been neglected by Gaussian-LIC and other approaches. Some attempts, such as flattened primitives or extra regularization terms, can enhance the geometry but tend to hinder the visual quality or real-time performance. To this end, Gaussian-LIC2 advocates achieving the most effective regularization under minimal supervision. It maximally exploits sparse LiDAR depth for map regularization and further enhances CUDA optimization strategies in Gaussian-LIC, enabling faster forward and backward propagation of depth rasterization. More importantly, the incremental insertion of Gaussians in SLAM can lead to depth underestimation for certain pixels. Gaussian-LIC2 mitigates this issue by scaling the rendered depth according to the rendered opacity, ensuring consistent and accurate depth supervision.
Gaussian Map Feedback to Conventional Odometry: The odometry and Gaussian mapping modules are fully decoupled in Gaussian-LIC and many existing methods. Gaussian-LIC2 investigates whether the incrementally built Gaussian map can, in turn, enhance the odometry performance by introducing a photometric constraint derived from the Gaussian map. Notably, Gaussian-LIC2 alleviates the impact of the varying camera exposure on the photometric constraint and validates the critical role of geometric accuracy in such a constraint, where more related insights are detailed in Sec. 9.3.1. Compared to the reprojection constraint from the LiDAR map as in Lang et al. (2023), the proposed Gaussian-map-based photometric constraint helps better handle situations where LiDAR degeneration and texture missing occur simultaneously.
Experimentally and practically, compared to Gaussian-LIC, Gaussian-LIC2 conducts broader experiments with a wider variety of LiDARs of different modalities and sparsity. Notably, a specialized LiDAR-Inertial-Camera dataset with ground-truth depths has been curated in Gaussian-LIC2 to enable assessment on out-of-sequence novel view synthesis. Overall, Gaussian-LIC2 achieves state-of-the-art novel-view RGB and depth rendering across various camera trajectories, with Depth L1 reduced by 62%. Finally, based on Gaussian-LIC2, two applications are presented. The code and the self-collected dataset will be released to benefit the community.
## 3 System Overview
The notations throughout the paper are given in Tab. 1. Fig. 2 depicts the overview of our proposed Gaussian-LIC2, which consists of two main modules: a continuous-time tightly-coupled LiDAR-Inertial-Camera Odometry and an incremental photo-realistic mapping back-end with 3DGS.
In the rest sections, we first present the formulation of continuous-time trajectory in Sec. 3.1 and introduce the preliminaries of 3DGS in Sec. 4. Next, in Sec. 5, we design a tightly-coupled LiDAR-Inertial-Camera odometry system as the front-end which supports two optional camera factors.
Table 1. Notations Glossary.
| Notation | Explanation |
| $^{A}\mathbf{p}$ | 3D point $\mathbf{p}$ in the frame $\{A\}$ |
| $^{A}\mathbf{p}_{B}$ | translation from frame $\{B\}$ to frame $\{A\}$ |
| $\mathbf{\beta}_{R}^{1} \in SO(3)$ | rotation matrix from frame $\{B\}$ to frame $\{A\}$ |
| $\mathbf{\beta}_{T}^{1} \in SE(3)$ | transformation matrix from frame $\{B\}$ to frame $\{A\}$ |
| $\mathbf{p}_{i}, \mathbf{R}_{i}$ | positional/rotational control point |
| $\mathbf{\Phi}_{(t_{\kappa-1}, t_{\kappa})}^{I}$ | control points of B-splines in $[t_{\kappa-1}, t_{\kappa}]$ |
| $I \omega_{m},^{l}a_{m}$ | angular velocity, linear acceleration in raw IMU measurements |
| $\mathbf{b}_{\omega}^{k-1},^{l}b_{a}^{k-1}$ | gyroscope and accelerometer biases during $[t_{\kappa-1}, t_{\kappa}]$ |
| $\mu, \Sigma$ | mean and covariance of a 3D Gaussian in frame $\{W\}$ |
| $\mathbf{S} \in \mathbb{R}^{3}$ | scale of a 3D Gaussian primitive |
| $o \in \mathbb{R}$ | opacity of a 3D Gaussian primitive |
| $\mathbf{SH} \in \mathbb{R}^{3 \times 16}$ | spherical harmonics coefficients of a 3D Gaussian primitive |
| $\mathbf{SH}_{0},^{l} \in \mathbb{R}^{16}$ | spherical harmonics in a certain viewing direction |
| $\mu', \Sigma'$ | 2D position and covariance of a 2D Gaussian splatted in a image plane |
| $\mathbf{c}$ | view-dependent color of the 3D Gaussian |
| $\mathcal{G}$ | Gaussian map in frame $\{W\}$ |
| $\mathbf{C}$ | raw RGB image |
| $\tilde{\mathbf{C}}(\mathcal{G}, _{C}^{W} \mathbf{T}(t))$ | rendered RGB image given camera pose $^{W}_{C} \mathbf{T}(t)$ |
| $\mathbf{D}_{s}$ | sparse depth map from the latest 5 frames |
| $\mathbf{D}_{c}$ | completed dense depth map |
| $\mathcal{P}$ | supplemented colorized point cloud selected from unprojected $\mathbf{D}_{c}$ |
tightly fused within a continuous-time factor graph, including constraints from the Gaussian map. We then utilize an efficient but generalizable depth model to fully initialize Gaussians and prepare mapping data for the back-end in Sec. 6. Finally, we perform photo-realistic Gaussian mapping with depth regularization and CUDA-related acceleration in Sec. 7.
### 3.1 Continuous-Time Trajectory Formulation
Two non-uniform cumulative B-splines, parameterizing the 3D rotation and the 3D translation, can jointly represent a continuous-time trajectory. The 6-DoF poses at time $t \in [t_i, t_{i+1})$ of a continuous-time trajectory are denoted by:
$$ \mathbf{R}(t)=F_{1}\left(\mathbf{R}_{i-3},\cdots,\mathbf{R}_{i},t_{i},t_{i+1},t\right), $$
$$ \mathbf{p}(t)=F_{2}\left(\mathbf{p}_{i-3},\cdots,\mathbf{p}_{i},t_{i},t_{i+1},t\right), $$
where $\mathbf{R}_n \in SO(3)$ and $\mathbf{p}_n \in \mathbb{R}^3$ denote control points ( $n \in \{i-3, \cdots, i\}$). $t_i$ and $t_{i+1}$ represent two adjacent knots. The functions $F_1$ and $F_2$ derive poses from control points, knots, and querying time. Please refer to Lv et al. (2023); Lang et al. (2023, 2022) for more details. The continuous-time trajectory of IMU in the world frame $\{W\}$ is denoted as $W_I(\mathbf{T}(t)) = [W_I(\mathbf{R}(t), W_{\mathbf{P}}(t))]$. Given known extrinsics between LiDAR/camera and IMU, we can accordingly get LiDAR trajectory $W_L(\mathbf{T}(t))$ and camera trajectory $W_C(\mathbf{T}(t))$.
## 4 3D Gaussian Splatting Representation
Due to the strengths of 3DGS illustrated in Sec. 2, we reconstruct a detailed photo-realistic map using a set of anisotropic 3D Gaussians. Each Gaussian is characterized by spatial position $\mu \in \mathbb{R}^3$, scale $\mathbf{S} \in \mathbb{R}^3$, rotation $\mathbf{R} \in \mathbb{R}^{3 \times 3}$, opacity $o \in \mathbb{R}$, and three-degree spherical harmonics coefficients $\mathbf{SH} \in \mathbb{R}^{3 \times 16}$ to encode view-dependent appearance of the scene Kerbl et al. (2023). Unlike several prior works Keetha et al. (2024); Sun et al. (2024), we do not enforce 3D Gaussians to be isotropic to compromise map quality for speed. Instead, we retain the original expressive parameterization Kerbl et al. (2023) and adopt the acceleration strategies described in Sec. 9.4 to pursue both quality and efficiency.
---
Figure 2. Pipeline of our real-time photo-realistic LiDAR-Inertial-Camera SLAM which represents the map using 3D Gaussians.
Representing the Gaussian’s ellipsoidal shape, the covariance of each Gaussian is parameterized as $\mathbf{\Sigma} = \mathbf{R}\mathbf{S}\mathbf{S}^T\mathbf{R}^T$. Given a camera pose $\mathbf{\Sigma}_W^C\mathbf{T} = \{\mathbf{\xi}_W^C\mathbf{R}, \mathbf{\xi}_W^C\mathbf{p}_W\}$, which maps a 3D point $\mathbf{W}\mathbf{p}$ from the world frame $\{\mathbf{W}\}$ to the camera frame $\{\mathbf{C}\}$, a 3D Gaussian $\mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ can be splatted onto the image screen, resulting in a corresponding elliptical 2D Gaussian $\mathcal{N}(\boldsymbol{\mu}^*, \boldsymbol{\Sigma}')$:
$$ \boldsymbol{\mu}^{\prime}=\boldsymbol{\pi}_{c}\left(\frac{\hat{\boldsymbol{\mu}}}{\mathbf{e}_{3}^{\top}\hat{\boldsymbol{\mu}}}\right),\hat{\boldsymbol{\mu}}=\mathbf{\Pi}_{W}^{C}\mathbf{R}\boldsymbol{\mu}+\mathbf{\Pi}^{C}\mathbf{p}_{W}, $$
$$ \mathbf{\Sigma}^{\prime}=\mathbf{J}_{\mathrm{~W~}}^{C}\mathbf{R}\mathbf{\Sigma}_{\mathrm{~W~}}^{C}\mathbf{R}^{T}\mathbf{J}^{T}, $$
where $\mathbf{e}_i$ is a $3 \times 1$ vector with its i-th element to be 1 and the other elements to be 0. Thus, $\mathbf{e}_3^\top \hat{\mu}$ gives the depth d of the 3D Gaussian in the camera frame. The function $\pi_c(\cdot)$ projects a 3D point on the normalized image plane to a pixel. $\mathbf{J} \in \mathbb{R}^{2 \times 3}$ represents the Jacobian of the affine approximation to the perspective projection Kerbl et al. (2023). The projected 2D Gaussian contributes to the image at pixel $\boldsymbol{\rho} = \begin{bmatrix} u & v \end{bmatrix}^\top$ with a weight computed as follows:
$$ \alpha=o\exp\left(-\frac{1}{2}(\boldsymbol{\mu}^{\prime}-\boldsymbol{\rho})^{T}(\boldsymbol{\Sigma}^{\prime})^{-1}(\boldsymbol{\mu}^{\prime}-\boldsymbol{\rho})\right). $$
By arranging all the successfully splatted 3D Gaussians in depth order, the color, depth, and opacity at pixel $\rho$ can be efficiently rendered using front-to-back $\alpha$-blending:
$$ \mathbf{C}(\boldsymbol{\rho})=\sum_{i=1}^{n}\mathbf{c}_{i}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j}), $$
$$ \mathbf{D}(\boldsymbol{\rho})=\sum_{i=1}^{n}d_{i}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j}), $$
$$ \mathbf{O}(\rho)=\sum_{i=1}^{n}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j}), $$
where $\mathbf{c}$ denotes the view-dependent color of the 3D Gaussian derived from spherical harmonics $\mathbf{SH}_b \in \mathbb{R}^{16}$ (as basis functions) and the corresponding coefficients $\mathbf{SH}$ (as weights), while $\mathbf{SH}_b$ is computed from the viewing direction between the Gaussian position $\boldsymbol{\mu}$ and camera position $^C \mathbf{p}_W$.
To investigate the constraint of the Gaussian map on conventional odometry, we analytically compute the Jacobian of the 3DGS representation w.r.t camera pose on the manifold according to MonoGS Matsuki et al. (2024), thereby avoiding the overhead of automatic differentiation. In contrast to MonoGS that models appearance with view-independent color, we further include the Jacobian of high-order spherical harmonics w.r.t the camera pose. The Jacobians are as follows:
$$ \begin{aligned}\frac{\partial\boldsymbol{\mu}^{\prime}}{\partial_{W}^{C}\mathbf{T}}&=\frac{\partial\boldsymbol{\mu}^{\prime}}{\partial\hat{\boldsymbol{\mu}}}\frac{\partial\hat{\boldsymbol{\mu}}}{\partial_{W}^{C}\mathbf{T}},\\\frac{\partial\boldsymbol{\Sigma}^{\prime}}{\partial_{W}^{C}\mathbf{T}}&=\frac{\partial\boldsymbol{\Sigma}^{\prime}}{\partial\mathbf{J}}\frac{\partial\mathbf{J}}{\partial\hat{\boldsymbol{\mu}}}\frac{\partial\hat{\boldsymbol{\mu}}}{\partial_{W}^{C}\mathbf{T}}+\frac{\partial\boldsymbol{\Sigma}^{\prime}}{\partial_{W}^{C}\mathbf{R}}\frac{\partial_{W}^{C}\mathbf{R}}{\partial_{W}^{C}\mathbf{T}},\\\frac{\partial\mathbf{c}}{\partial_{W}^{C}\mathbf{T}}&=\frac{\partial\mathbf{c}}{\partial\mathbf{S}\mathbf{H}_{b}}\frac{\partial\mathbf{S}\mathbf{H}_{b}}{\partial^{C}\mathbf{p}_{W}}\frac{\partial^{C}\mathbf{p}_{W}}{\partial_{W}^{C}\mathbf{T}}.\end{aligned} $$
The partial derivative on the manifold is defined as:
$$ \frac{\partial f(\mathbf{T})}{\partial\mathbf{T}}\triangleq\operatorname*{l i m}_{\pmb{\xi}\to0}\frac{f\big(\mathbf{T}\operatorname{E x p}(\pmb{\xi})\big)-f\big(\mathbf{T}\big)}{\pmb{\xi}}, $$
where $\mathbf{T} \in SE(3)$ and $\boldsymbol{\xi} \in \mathfrak{se}(3)$. $\text{Exp}(\cdot)$ maps Lie Algebra to Lie Group and $\text{Log}(\cdot)$ is its inverse operation.
## 5 LiDAR-Inertial-Camera Fusion
We draw on the insights of classic multi-sensor fused odometry to achieve real-time, robust, and accurate pose estimation, supporting for subsequent photo-realistic mapping. Similar to Gaussian-LIC, our odometry system is inherited from and further developed based on our prior work Coco-LIC Lang et al. (2023), a continuous-time tightly-coupled LiDAR-Inertial-Camera odometry using non-uniform B-spline. By leveraging the continuous-time trajectory representation, which inherently supports pose querying at any timestamp corresponding to sensor measurements, we tightly fuse asynchronous, high-frequency LiDAR-Inertial-Camera data without introducing interpolation error. The odometry here supports two types of camera factors, one of which is based on the Gaussian map.
### 5.1 Trajectory Extension
Our LiDAR-Inertial-Camera system is initialized from a stationary state, using a buffer of IMU measurements to initialize IMU biases and the gravity-aligned orientation Geneva et al.
---
(2020). After initialization, the system adopts an active sliding window of the latest 0.1 seconds for optimization.
Starting from time instant $t_{\kappa-1}$, we estimate the trajectory in $[t_{\kappa-1}, t_{\kappa})$ once the LiDAR-Inertial-Camera data in the sliding window time interval is ready, where $t_{\kappa} = t_{\kappa-1} + 0.1$. The accumulated data contains all LiDAR raw points $\mathcal{L}_{\kappa}$, all IMU raw data $\mathcal{I}_{\kappa}$, and the latest image frame $\mathcal{F}_{\kappa}$ in $[t_{\kappa-1}, t_{\kappa})$. For simplicity, we omit other image frames captured within this interval, given their negligible impact on the overall performance. As in Coco-LIC, we first adaptively initialize a variable number of control points based on the motion intensity inferred from the IMU data, and then insert these control points into the sliding window to extend the trajectory. The following states are then optimized within the sliding window:
$$ \begin{aligned}\mathcal{X}^{\kappa}&=\{\mathbf{\Phi}(t_{\kappa-1},t_{\kappa}),\mathbf{x}_{I_{b}}^{\kappa}\},\\\mathbf{x}_{I_{b}}^{\kappa}&=\{\mathbf{b}_{g}^{\kappa-1},\mathbf{b}_{a}^{\kappa-1},\mathbf{b}_{g}^{\kappa},\mathbf{b}_{a}^{\kappa}\},\end{aligned} $$
where $\mathbf{\Phi}(t_{\kappa-1}, t_{\kappa})$ denotes all control points within the interval $[t_{\kappa-1}, t_{\kappa})$, parameterizing the continuous-time trajectory of the IMU in the world frame. $\mathbf{x}_{t_b}^{\kappa}$ denotes the IMU bias, which includes the gyroscope bias $\mathbf{b}_g$ and the accelerometer bias $\mathbf{b}_a$. The IMU biases during $[t_{\kappa-1}, t_{\kappa})$ are assumed to be constant as $\mathbf{b}_g^{\kappa-1}$ and $\mathbf{b}_a^{\kappa-1}$. They are under Gaussian random walk and evolve to $\mathbf{b}_g^{\kappa}$ and $\mathbf{a}_g^{\kappa}$ at $t_{\kappa}$.
### 5.2 Continuous-Time Factor Graph
5.2.1 LiDAR Factor Different from Coco-LIC with LiDAR feature extraction and kd-Tree-organized map, we here directly register downsampled LiDAR raw points to the LiDAR map organized by the ikd-Tree for enhanced efficiency Xu et al. (2022). Given a raw LiDAR point $L_p \in L_k$ measured at time t, we first transform it to the world frame with the queried pose from the continuous-time trajectory, and then search for its five nearest neighbors in the LiDAR map to fit a 3D plane for a point-to-plane residual:
$$ \mathbf{r}_{L}=^{W}\mathbf{n}_{\pi}^{\top W}\hat{\mathbf{p}}+^{W}d_{\pi}, $$
$$ \mathbf{\hat{p}}^{W}=\mathbf{\Gamma}_{L}^{W}\mathbf{R}(t)^{L}\mathbf{\Gamma}\mathbf{p}+W\mathbf{\Gamma}\mathbf{p}_{L}(t)\mathbf{\Gamma}, $$
where $^{W}n_{\pi}$ and $^{W}d_{\pi}$ denote the unit normal vector and the distance of the plane to the origin, respectively.
5.2.2 Inertial Factor We define the following inertial factors:
$$ \mathbf{r}_{I}=\left[\begin{matrix}{^{I}\mathbf{\omega}(t)-^{I}\mathbf{\omega}_{m}+\mathbf{b}_{g}^{\kappa-1}}\\ {^{I}\mathbf{a}(t)-^{I}\mathbf{a}_{m}+\mathbf{b}_{a}^{\kappa-1}}\\ \end{matrix}\right], $$
$$ \mathbf{r}_{I_{b}}=\left[\begin{matrix}{\mathbf{b}_{g}^{\kappa}-\mathbf{b}_{g}^{\kappa-1}}\\ {\mathbf{b}_{a}^{\kappa}-\mathbf{b}_{a}^{\kappa-1}}\\ \end{matrix}\right], $$
where the former is the IMU factor and the latter is the bias factor based on the random walk process. $^{I}\omega_{m},^{I}\mathbf{a}_{m}$ are the raw measurements of angular velocity and linear acceleration of the IMU data at time t in $\mathcal{I}_{\kappa}$. $^{I}\omega(t)$ and $^{I}\mathbf{a}(t)$ are the corresponding predicted values computed from the derivatives of the continuous-time trajectory in Eq.(1) and Eq.(2).
5.2.3 Camera Factor (Option 1, Default) Our odometry incorporates two types of camera factors, the first of which is based on reprojection constraints using the reconstructed LiDAR map Lang et al. (2023). Specifically, we maintain a subset of global LiDAR points stored in voxels and associate them with image pixels by projection, KLT sparse optical flow Lucas and Kanade (1981), and RASAC-based outlier removal. Consider a global LiDAR point $^W p$ associated with the pixel $\rho = [u \quad v]^\top$ in the image frame at the timestamp t. The reprojection error for this LiDAR point is defined as:
$$ \mathbf{r}_{C_{1}}=\pi_{c}\left(\frac{^{C}\hat{\mathbf{p}}}{\mathbf{e}_{3}^{\top C}\hat{\mathbf{p}}}\right)-\left[\begin{matrix}{u}\\ {v}\\ \end{matrix}\right],\quad^{C}\hat{\mathbf{p}}=\mathbf{\Sigma}_{C}^{W}\mathbf{T}^{-1}(t)^{W}\mathbf{p}. $$
5.2.4 Camera Factor (Option 2) The second type of camera factor fully leverages the incrementally constructed Gaussian map. Thanks to the continuous-time trajectory formulation, we can obtain the camera pose $\mathbf{C}_c^W \mathbf{T}$ at any time instant within the interval $[t_{\kappa-1}, t_{\kappa})$. Given the timestamp t at which the raw RGB image $\mathbf{C}$ (treated as ground truth) of frame $\mathcal{F}_\kappa$ is captured, we render an RGB image $\hat{\mathbf{C}}(\mathcal{G}, \mathcal{C}_c^W \mathbf{T}(t))$ from the Gaussian map $\mathcal{G}$ using Eq. (6). The rendered image is expected to closely match the raw image, enabling optimization of the camera pose by minimizing the loss:
$$ \mathcal{L}=\frac{1}{2}\left\|\hat{\mathbf{C}}-\mathbf{C}\right\|_{1}+\frac{1}{2}\mathcal{L}_{\mathrm{D-S S I M}}(\hat{\mathbf{C}},\mathbf{C}), $$
where $\mathcal{L}_{D-SSIM}$ is a D-SSIM term Kerbl et al. (2023). Low-image-gradient or low-opacity pixels are penalized Matsuki et al. (2024), which is omitted here for simplicity. Besides, in unbounded outdoor scenes with highly variable illumination and under conditions of fluctuating camera exposure, a standalone RGB L1 loss is susceptible to these noises, which may corrupt gradient directions, leading to erroneous camera pose optimization. To this end, we additionally introduce the D-SSIM loss to enhance the optimization robustness by accounting for structural similarity. We perform $N_t$ iterations using the Adam optimizer, during which the Gaussian map $\mathcal{G}$ is kept fixed while the camera pose is iteratively optimized. The resulting optimized camera pose from Eq. (17) is denoted as $\mathbf{W}_c^{\mathbf{T}} = \{\mathbf{W}_c^{\mathbf{R}}, \mathbf{W}_{p_c}^{\mathbf{D}}\}$, and below is the derived photometric constraint from the Gaussian map:
$$ \mathbf{r}_{C_{2}}=\operatorname{L o g}\left(_{C}^{W}\mathbf{T}(t)_{C}^{W}\widetilde{\mathbf{T}}^{-1}\right). $$
### 5.3 LiDAR-Inertial-Camera Factor Graph Optimization
We jointly fuse LiDAR-Inertial-Camera data in the factor graph and formulate the following nonlinear least-squares problem to efficiently optimize states $\mathcal{X}^{\kappa}$:
$$ \begin{aligned}\arg\min_{\mathcal{X}^{\kappa}}\Bigg\{\sum\|\mathbf{r}_{L}\|_{\Sigma_{L}}^{2}+\sum\|\mathbf{r}_{I}\|_{\Sigma_{I}}^{2}+\sum\|\mathbf{r}_{I_{b}}\|_{\Sigma_{I_{b}}}^{2}+\\ \sum\|\mathbf{r}_{C_{x}}\|_{\Sigma_{C}}^{2}+\sum\|\mathbf{r}_{prior}\|_{\Sigma_{prior}}^{2}\Bigg\},\end{aligned} $$
which is solved via the Levenberg-Marquardt algorithm in Ceres Solver Agarwal et al. (2012) and accelerated through the analytical derivatives. The camera factor $\mathbf{r}_{C_x}$ can be chosen as either $\mathbf{r}_{C_1}$ or $\mathbf{r}_{C_2}$. The former is a reprojection constraint from the LiDAR map, which is somewhat handcrafted but very lightweight, while the latter is a photometric constraint from the Gaussian map, which consumes modest GPU computing resources but is more direct and natural. A comparative evaluation of the two factors
---
will be presented in Sec. 9. $\mathbf{r}_{\text{prior}}$ is the prior factor from marginalization Lang et al. (2023). $\mathbf{\Sigma}_L$, $\mathbf{\Sigma}_I$, $\mathbf{\Sigma}_{I_b}$, $\mathbf{\Sigma}_C$, $\mathbf{\Sigma}_{\text{prior}}$ are the corresponding measurement covariances.
## 6 Data Preparation for Gaussian Mapping
### 6.1 Mapping Data Grouping
After finishing the optimization of the latest trajectory segment in $[t_{\kappa-1}, t_{\kappa})$, we ultimately obtain the accurate camera pose for the latest image frame $\mathcal{F}_{\kappa}$. All LiDAR raw points within this time interval, namely $\mathcal{L}_{\kappa}$, can be easily transformed into the world frame based on the optimized continuous-time trajectory. The posed image, together with the transformed LiDAR points, is jointly regarded as a hybrid frame, and we treat every fifth hybrid frame as a keyframe for photo-realistic mapping. If the current hybrid frame is selected as a keyframe, we merge all LiDAR points from the latest five hybrid frames into a single point cloud. This point cloud is then projected onto the current image plane to generate a sparse depth map. Next, we downsample the point cloud by randomly retaining one out of every $N_{p}$ points. The downsampled point cloud is also projected onto the current image plane for coloring. Colorized points will be used for initializing 3D Gaussians, which will be discussed in Sec. 7.1.2. At this stage, the keyframe is fully constructed, consisting of a posed image, a sparse depth map, and a set of colorized LiDAR points.
### 6.2 LiDAR Blind Area Compensation
LiDARs are able to provide precise geometric priors for Gaussian initialization. However, due to the mismatch in FoV between the LiDAR and the camera, relying solely on LiDAR points would lead to under-reconstruction in LiDAR blind areas or when using the sparse LiDAR. Although the adaptive density control (ADC) from vanilla 3DGS Kerbl et al. (2023) offers a partial relief, its effectiveness is limited in SLAM. The incremental insertion of Gaussians leads to inconsistent convergence among different batches of Gaussians, which may misguide gradient-based operations like cloning and splitting. Moreover, the reliance on multi-iteration gradient accumulation hinders the real-time performance of the system. Gaussian-LIC leverages online triangulated visual features to compensate for regions unobserved by LiDAR. However, it struggles in textureless areas and is sensitive to interframe viewpoint variations. Therefore, further investigation is warranted to explore how visual cues can be more effectively leveraged for efficient Gaussian initialization. The central challenge lies in obtaining accurate depth for image pixels.
Why should it be depth completion? With the advancement of computer vision research, numerous methods have emerged for recovering dense depth from images. First, monocular relative depth estimation Yang et al. (2024); Wang et al. (2024b) has made significant progress in predicting depth from a single RGB image. However, it inherently suffers from scale ambiguity. While least-squares alignment with metric LiDAR depth can be applied, it often fails to recover accurate metric scale across all image pixels—particularly in regions with poor relative depth estimation or lacking LiDAR coverage. Zuo et al. (2025). Second, monocular metric depth estimation Hu et al. (2024); Bochkovskii et al. (2024) directly
Figure 3. Details on the LiDAR blind area compensation: (a) Projection of LiDAR points of the latest 5 frames. (b) Colorized LiDAR points of the latest 5 frames. It is worth noting that the area highlighted by the red arrow has been never scanned by the LiDAR during the entire data acquisition. (c) Completed dense depth map. (d) Dense colored point cloud from the completed depth map. (e) The image is divided into $30 \times 30$ patches, and the red dot denotes the selected pixels. (f) The red star indicates the 3D visual points obtained by back-projecting the selected pixels to compensate for the LiDAR blind area. (g) and (h) show the qualitative rendering results of Gaussian-LIC2 without and with LiDAR blind area compensation.
predicts scale-aware depth. However, its accuracy remains modest and lags far behind LiDAR-based methods, especially in large-scale outdoor environments. Third, multi-view stereo (MVS) approaches Cao et al. (2022); Xu et al. (2023) estimate metric depth from multiple posed images. Although MVS can produce accurate depth in favorable conditions, it is sensitive to parallax and exhibits limited generalization, particularly in large-scale outdoor scenes. In contrast, depth completion, which fuses sparse LiDAR data with a single RGB image, offers a practical and robust solution for recovering metric dense depth Liu et al. (2024b); Lin et al. (2024) in large-scale scenarios. It avoids the issue of scale ambiguity and can achieve LiDAR-comparable depth estimation accuracy.
In this paper, SPNet Wang et al. (2024a) is selected as the depth completer due to its efficiency, compactness, and strong generalization ability. We use it directly off the shelf, without any additional fine-tuning. As outlined in Alg. 1 and illustrated in Fig. 3, we feed the sparse depth map $D_s$ and the RGB image C of the keyframe in Sec. 6.1 into SPNet to generate a completed dense depth map $D_c$. We then compute the mean depth change in the known regions before and after completion. If the depth change exceeds a threshold $\epsilon_1$, the completion is considered a failure (rarely occurring). After successful completion, pixels in $D_c$ with negative depth values or high depth gradient magnitudes are discarded, producing the filtered dense depth map $D_{cf}$. Subsequently, we divide the input sparse LiDAR depth map into patches of size $30 \times 30$ and iterate over each of them. For patches without any valid LiDAR depth, we select the pixel
---
```pseudo
// Algorithm 1: LiDAR Blind Area Compensation
Input: Sparse depth map $D_s$, RGB image C
Output: Supplemented colored point cloud $\mathcal{P}$
```
1 $\mathcal{P} \leftarrow \emptyset$, $\mathcal{A} \leftarrow \emptyset$ // Initialize outputs
2 $\mathbf{M} \leftarrow \text{ValidMask}(\mathbf{D}_s)$
3 $\mathbf{D}_c \leftarrow \text{SPNet}(\mathbf{C}, \mathbf{D}_s, \mathbf{M})$
4 $\delta \leftarrow \text{Mean}(|(\mathbf{D}_c - \mathbf{D}_s)[\mathbf{M}]|)$
5 if $\delta < \epsilon_1$ then
6 $\mathbf{D}_{cf} \leftarrow \text{Filter}(\mathbf{D}_c)$
7 Divide $\mathbf{D}_s$ into $30 \times 30$ grid patches
8 \text{foreach patch } \mathcal{R} \text{ in grid do}
9 if IsEmpty($\mathbf{D}_s[\mathcal{R}]$) then
10 $\rho \leftarrow \text{MinValidDepthPixel}(\mathbf{D}_{cf}[\mathcal{R}])$
11 if $Depth(\rho) < \epsilon_2$ then
12 $\bigsqcup \mathcal{A} \leftarrow \mathcal{A} \cup \{\rho\}$
13 $\mathcal{P} \leftarrow \text{BackProject}(\mathcal{A}, \mathcal{I})$
14 return $\mathcal{P}$
with the smallest completed depth within the patch of $\mathbf{D}_{cf}$ and store it in a container $\mathcal{A}$ if its depth is less than $\epsilon_2$. All pixels in $\mathcal{A}$ are back-projected to form a supplemented colorized point cloud $\mathcal{P}$, compensating for LiDAR-unobserved regions. Finally, $\mathcal{P}$ is transformed to the world frame and merged with the colored LiDAR points of the keyframe.
## 7 Real-Time Photo-Realistic Mapping
The mapping back-end thread runs in parallel with the tracking front-end thread and continuously receives sequential keyframes from the tracker to perform real-time photo-realistic reconstruction. Each keyframe contains an estimated camera pose, an undistorted image, a sparse LiDAR depth map, and a set of colored LiDAR points augmented with supplemented visual points to compensate for the blind area.
### 7.1 Gaussian Map Management
Once a keyframe is received, the mapping thread will initialize or expand the Gaussian map and optimize it.
7.1.1 Initialization of Gaussian Map The Gaussian map is initialized using all the colorized LiDAR points and the supplemented visual points from the first keyframe. Specifically, for each point, a new Gaussian is instantiated at its 3D location, with the zeroth degree of $\mathbf{SH}$ initialized using its RGB color, opacity set to 0.1, and rotation initialized as the identity matrix. To mitigate aliasing artifacts Yan et al. (2024c), we adapt the scale of each Gaussian based on its distance to the image plane. The scale is modeled as $\mathbf{S} = \frac{d}{f}\mathbf{e}$, where $\mathbf{e}$ is a $3 \times 1$ vector of ones, d is the depth of the 3D point in the camera frame, and f is the focal length.
7.1.2 Expansion of Gaussian Map Each incoming keyframe typically captures new geometric and appearance information. To maintain efficiency, we focus on initializing new Gaussians for the newly observed parts of the scene. Specifically, we first render an opacity map $\mathbf{O}$ from the perspective of the current keyframe using Eq. (8). A binary mask $\mathbf{M}_o = \mathbf{O} < \tau$ ( $\tau$ is a constant threshold) is then constructed to identify potentially newly observed image regions. Finally, only the 3D points projected into these image regions are selected to initialize new Gaussians.
7.1.3 Depth-Regularized Map Optimization After the initialization or expansion, we randomly sample K keyframes out of all keyframes to optimize the Gaussian map, preventing catastrophic forgetting and ensuring global geometric consistency. The selected keyframes are shuffled and sequentially used to update the map by minimizing the following rendering loss:
$$ \mathcal{L}=\mathcal{L}_{c}+\gamma\mathcal{L}_{d}, $$
$$ \mathcal{L}_{c}=(1-\lambda)\left\|\hat{\mathbf{C}}-\mathbf{C}\right\|_{1}+\lambda\mathcal{L}_{\mathrm{D-S S I M}}(\hat{\mathbf{C}},\mathbf{C}), $$
$$ \mathcal{L}_{d}=\left\|\left(\frac{\hat{\mathbf{D}}}{\hat{\mathbf{O}}}-\mathbf{D}_{s}\right)\cdot\mathbf{M}_{d}\right\|_{1},\mathbf{M}_{d}=\mathbf{D}_{s}>0, $$
where $\mathcal{L}_c$ denotes the RGB rendering loss computed by the rendered image $\hat{C}$ based on Eq.(6) and the raw image C, while $\mathcal{L}_d$ is the depth rendering loss involved with the rendered depth map $\hat{D}$ via Eq.(7) and the sparse LiDAR depth map $\mathbf{D}_s$. Note that the incremental insertion of Gaussians in SLAM can cause the rendered depth of certain pixels to be significantly smaller than the ground-truth values. We here normalize the rendered depth map using the rendered opacity map $\hat{O}$ to correct the depth supervision.
### 7.2 CUDA-Related Acceleration Strategies
To improve real-time performance, previous works either forcibly limit the number of Gaussians and optimization iterations or simplify the Gaussian representation Keetha et al. (2024); Peng et al. (2024), which might work well in indoor but often degrades performance in unbounded and complex outdoor scenarios. On the contrary, we concentrate on speeding up the CUDA-based operations both in terms of algorithms and engineering, particularly the forward and backward propagation of the 3DGS rasterizer, so as to ensure real-time performance without compromising the quality.
7.2.1 Fast Tile-based Culling During the forward pass, 3D Gaussians are splatted onto the image, resulting in elliptical 2D Gaussians. To determine the tiles affected by each Gaussian, the elliptical 2D Gaussian is further dilated into a circle whose radius equals the major axis length Kerbl et al. (2023). However, such an approximation results in overly inflated tiles influenced by the Gaussian, particularly for highly anisotropic Gaussians, which are common in incremental mapping systems such as SLAM. To address this issue, we adopt a fast tile-based culling strategy, as illustrated in Fig. 4(a). For each 2D Gaussian $\mathcal{N}(\mu', \Sigma')$ and the tiles affected by it, we identify the pixel $\rho$ within every tile where the 2D Gaussian yields the highest contribution (with the max weight $\alpha$ in Eq.(5)), as in Radl et al. (2024). If the weight $\alpha$ at the pixel $\rho$ is smaller than the threshold $\frac{1}{255}$, we regard the tile as weakly affected and cull it. In this way, the number of Gaussians per tile can be significantly reduced, accelerating the subsequent forward and backward pass.
7.2.2 Per-Gaussian Backpropagation During the backward pass, gradients flow from RGB pixels back to Gaussians, which is the most time-consuming stage during the mapping Kerbl et al. (2023). Each pixel typically receives
---
Figure 4. Acceleration strategies in the forward and backward pass when optimizing the Gaussian map: (a) Culling the tiles weakly affected by the splatted Gaussians. (b1) Every 32 Gaussians within a tile ( $16 \times 16$ pixels) are grouped into a bucket. (b2) All buckets are processed in parallel for (256+31) iterations (row by row). In each iteration, all Gaussians within the bucket accumulate gradients from different pixels for optimization with respect to the rendered RGB, depth, and opacity maps—without incurring atomic collisions.
contributions from a different number of Gaussians, and the runtime is ultimately dominated by the pixel associated with the largest number of Gaussians. As a result, the remaining CUDA threads must wait idly, leading to significant inefficiency. Also, atomic gradient addition conflicts arise when multiple pixels backpropagate to the same Gaussian. To address these issues, we shift from pixel-wise parallelism to Gaussian-wise parallelism following Mallick et al. (2024), as illustrated in Fig. 4(b1) and (b2). To be specific, we divide the depth-sorted Gaussians within each tile into buckets of 32 Gaussians each. Then, all Gaussians across all buckets are processed in parallel. For a given bucket, all 32 Gaussians simultaneously iterate over pixels within the tile to accumulate gradients. As a result, the upper bound of the backward time is largely determined by the number of pixels, leading to more consistent computational load and reduced collisions. In addition to the RGB channels, we also incorporate per-Gaussian gradient backpropagation for the depth map and the opacity map, which has not been handled in Gaussian-LIC.
7.2.3 Additional Strategies a) Sparse Adam. The original 3DGS Kerbl et al. (2023) uses the Adam optimizer to update all Gaussians, including those not involved in the current rendering, by applying zero gradients. The computational burden becomes progressively unsustainable as the map size increases. Therefore, we adopt sparse Adam Mallick et al. (2024) to update only the valid Gaussians that participate in the rendering. b) Separated SH. In the original 3DGS framework, the low-order and high-order SH coefficients are concatenated before the forward and backward passes per iteration. However, the concatenation operation is time-consuming, especially when using three-degree SH in our method. To address this, we handle the low-order and high-order coefficients separately, eliminating the concatenation.
c) Efficient SSIM. We employ the highly efficient and differentiable SSIM implemented in Mallick et al. (2024), where separable Gaussian convolutions lead to reduced computation. d) Warp-Level Predicate Masking. The warp serves as a bridge between threads and blocks, which is crucial for parallel performance in GPU. Threads are typically operated in groups of 32 as a warp, and performance can degrade if threads within a warp diverge due to conditional branches. For example, in the forward pass of 3DGS, Gaussians handled by different threads may be deemed invalid—due to invalid depth, non-invertible 2D covariance, or insufficient weights—causing threads to exit under varying conditions, leading to divergence. To tackle this problem, we use flags to unify the return decisions in the final stage, avoiding divergence. Besides, we implement early stopping for warps in which all threads are flagged to return, improving efficiency. e) CPU to GPU. When optimizing the map, ground-truth images and sparse LiDAR depth maps need to be transferred from the CPU to the GPU, which incurs a certain amount of overhead. However, preloading them onto the GPU is not allowed due to limited memory. Thus, we store them in pinned memory and utilize non-blocking transfer to move data from the CPU to the GPU. Specifically, the pinned memory is a special memory region on the CPU that is directly accessible by the GPU, which accelerates the data transfer. The cudaMemcpyAsync() function is used to perform the transfer asynchronously, while CUDA streams are employed to manage concurrent execution of memory copy operations and GPU kernels. By utilizing multiple streams, data transfer and computation can occur simultaneously, minimizing idle time for the GPU and improving overall throughput.
## 8 Datasets for Evaluation
This section describes the datasets used for evaluation, encompassing both public and self-collected ones. The public datasets, including R3LIVE Lin and Zhang (2022), FASTLIVO Zheng et al. (2022, 2024), MCD Nguyen et al. (2024), and M2DGR Yin et al. (2021), are mainly employed for localization and in-sequence novel view rendering evaluation. Our self-collected dataset supports both in-sequence and out-of-sequence novel view rendering evaluation, and enables quantitative evaluation of rendered depth. Tab. 8 summarizes the duration and length of each sequence.
### 8.1 Public Dataset
The R3LIVE dataset Lin and Zhang (2022) and FAST-LIVO dataset Zheng et al. (2022, 2024) are both collected within the campuses using a handheld device equipped with a Livox Avia LiDAR at 10 Hz and its built-in IMU at 200 Hz, and a 15 Hz RGB camera. Closely aligned with the camera's FoV and performing non-repetitive scanning, the projected points of the adopted solid-state LiDAR are distributed across the image, while still suffering from non-negligible blind spots as shown in Fig. 3. Six outdoor sequences from the R3LIVE dataset that are well-suited for mapping are selected for evaluation. The FAST-LIVO dataset used in this work includes all sequences from FAST-LIVO and the first two sequences from FAST-LIVO2, whose high-quality images make the dataset particularly suitable for mapping tasks. Notably, the R3LIVE and FAST-LIVO datasets also provide challenging sequences with LiDAR or visual degeneration, such as degenerate-seq-00, degenerate-seq-01, LiDAR-Degenerate, and Visual-Challenge. Although the ground-truth trajectory
---
Figure 5. Two types of LIC sensor rig for self-collected dataset.
is unavailable, the start and end poses coincide, allowing localization evaluation via start-to-end drift. When stereo pairs are available, only the left image $(640 \times 512)$ is used.
The MCD dataset Nguyen et al. (2024) provides sequences captured in large-scale campuses. We here pick the sequences (tuhh_day_02-04) with the repetitive mechanical spinning Ouster OS1-64 LiDAR at 10Hz, a 30 Hz RGB camera (640 × 480) and a 400 Hz IMU. The spatial sparsity of spinning LiDAR poses greater challenges for LiDAR-based Gaussian mapping. Due to the extensive scale of the environment, we truncate the sequences to the first 200 seconds (~300 m). The dataset provides highly accurate ground-truth trajectories for quantitative evaluation of localization.
The M2DGR dataset Yin et al. (2021), collected by a ground robot using a Velodyne VLP-32C LiDAR at 10 Hz, an IMU at 100 Hz, and a 10 Hz RGB camera ( $640 \times 480$), presents a more challenging case due to the extremely sparse LiDAR measurements. A set of sequences (room_01-03) under favorable lighting are selected for evaluation.
### 8.2 Self-Collected Dataset
To further evaluate out-of-sequence novel view synthesis and quantitatively assess the accuracy of the rendered depth, we collect additional sequences leveraging two handheld devices in Fig. 5, enriching LiDAR modality diversity. The first setup integrates a Livox Mid-360 LiDAR sampled at 10 Hz paired with a 30 Hz RealSense D455 RGB camera (640 × 480). Compared to Avia, Mid-360 offers a wider horizontal FoV but shorter maximum ranging distance, presenting unique reconstruction challenges. Meanwhile, the second one follows the LiV_handheld* configuration, combining a Livox Avia LiDAR at 10 Hz with a Hikvision MV-CA013-21UC RGB camera at 15 Hz (640 × 512). Both handheld systems utilize the built-in 200 Hz IMU from the LiDAR, with the LiDAR and the IMU factory-synchronized and well-calibrated. The spatial and temporal extrinsics between the camera and the IMU are carefully calibrated through Furgale et al. (2013).
For each sequence, we capture a long and smooth trajectory with loop closures, ensuring the scene is revisited for out-of-sequence evaluation and making the captured data well-suited for mapping purposes. Due to unreliable GPS between buildings, GPS-based pose estimation is often unreliable. We generate continuous-time ground-truth trajectories using a LiDAR-Inertial-Camera SLAM framework that incorporates continuous-time loop closure Lv et al. (2021) to enhance accuracy. Note that our captured scenes are not excessively large (i.e., not at an urban scale), and the estimated ground-truth trajectories are sufficiently accurate for evaluation purposes. For generating ground-truth depth maps, we query the camera pose from the ground-truth continuous-time trajectory and project the temporally closest LiDAR frame onto the image. Each collected trajectory is divided into two segments, used respectively for in-sequence and out-of-sequence evaluations, as illustrated in Fig. 9.
## 9 Experiment Results
### 9.1 Implementation Details
We implement Gaussian-LIC2 fully in C++ and CUDA. The tracking and mapping modules run in parallel and communicate via ROS, with the mapping built upon the LibTorch framework. The depth completion model Wang et al. (2024a) is deployed with TensorRT for efficient inference.
For odometry, we reuse the ikd-Tree parameters from FAST-LIO2 Xu et al. (2022), set the camera pose optimization step $N_t$ in Sec. 5.2.4 to 30. For mapping data preparation, we set $N_p$ in Sec. 6.1 to 10 (Avia, Mid-360), 5 (OS1-64), and 1 (VLP-32C). $\epsilon_1$ and $\epsilon_2$ in Sec. 6.2 are empirically set to 0.1 m and 50 m, respectively. For map expansion, we set $\tau$ in Sec. 7.1.2 to 0.99 as in Gaussian-LIC. As for map optimization, we set the loss weighting $\lambda$ to 0.2 as in Kerbl et al. (2023) and $\gamma$ to 0.005. $\gamma$ is determined through rigorous adjustment to strike a balance between visual and geometric quality. The number K of selected keyframes is set to 100. All the learning rates for Gaussian attributes follow the vanilla 3DGS Kerbl et al. (2023) but do not decay with schedulers. We keep the same hyperparameters in all sequences with the same sensor setup to ensure a fair and comprehensive evaluation. Experiments are run on a desktop PC with an NVIDIA RTX 4090 GPU (24 GB VRAM), Intel i9-13900K F CPU, and 64 GB RAM.
### 9.2 Baselines
This paper focuses on developing an elaborated radiance-filed-based SLAM system capable of robust and accurate pose estimation while constructing photo-realistic 3D maps in real time. Therefore, our comparative analysis specifically targets authentic SLAM systems, explicitly excluding pure mapping systems that require a priori ground-truth poses.
We first evaluate localization performance against state-of-the-art traditional LiDAR-Inertial-Camera SLAM systems, including optimization-based LVI-SAM (Shan et al. (2021)) and filter-based R3LIVE Lin and Zhang (2022) and FASTLIVO2 Zheng et al. (2024). We also include two neural SLAM methods: PIN-SLAM Pan et al. (2024), a neural LiDAR SLAM approach utilizing point-based NeRF for geometry-only reconstruction, and DBA-Fusion Zhou et al. (2024b), which integrates IMU data into DROID-SLAM Teed and Deng (2021), currently the most advanced neural visual SLAM framework. We run PIN-SLAM by feeding in undistorted LiDAR point clouds.
Furthermore, to evaluate both the localization and photo-realistic mapping performances of the entire system, we compare against photo-realistic radiance-field-based SLAM,
---
Table 2. Localization Evaluation: The start-to-end drift error (translation | rotation) on challenging sequences. The best are in bold and the second best are $\underline{\text{underscored}}$. $\times$ means failure. Note that, except for the sequence LiDAR_Degenerate, which is an indoor sequence, all the others are outdoor sequences. Hybrid means using both the point cloud map and the 3DGS map for localization.
| Method | Sensor | Map Type (for Loc) | Sequence |
| degenerate_seq_00 | degenerate_seq_01 | LiDAR_Degenerate | Visual_Challenge |
| MonoGS Matsuki et al. (2024) | C | 3DGS | 14.54m | 67.40° | ✗ | | ✗ | | ✗ |
| Co-SLAM Wang et al. (2023a) | L + C | NeRF | 12.37m | 30.50° | ✗ | | ✗ | | ✗ |
| SplaTAM Keetha et al. (2024) | L + C | 3DGS | 17.96m | 16.75° | ✗ | | ✗ | | ✗ |
| MM3DGS-SLAM Sun et al. (2024) | L + I + C | 3DGS | 9.14m | 14.02° | ✗ | | ✗ | | ✗ |
| PIN-SLAM Pan et al. (2024) | L | NeRF | 9.07m | 30.05° | 13.53m | 29.29° | 4.13m | 3.71° | ✗ |
| DBA-Fusion Zhou et al. (2024b) | I + C | PointCloud | 0.53m | 8.98° | 1.87m | 1.93° | 0.46m | 4.66° | 3.19m |
| LVI-SAM Shan et al. (2021) | L + I + C | PointCloud | 0.08m | 6.60° | 0.11m | 2.67° | ✗ | | ✗ |
| R3LIVE Lin and Zhang (2022) | L + I + C | PointCloud | 0.04m | 0.41° | 0.11m | 0.55° | 8.52m | 2.99° | 0.21m |
| FAST-LIVO2 Zheng et al. (2024) | L + I + C | PointCloud | 5.07m | 8.38° | 2.41m | 6.20° | 0.02m | 2.40° | 0.02m |
| Gaussian-LIC Lang et al. (2025) | L + I + C | PointCloud | 0.04m | 0.54° | 0.06m | 0.62° | 0.05m | 2.63° | 0.07m |
| Gaussian-LIC2-LIO | L + I | PointCloud | 7.87m | 9.15° | 13.72m | 11.10° | 6.62m | 4.84° | 12.05m |
| Gaussian-LIC2 (c1) | L + I + C | PointCloud | 0.04m | 0.58° | 0.04m | 0.55° | 0.05m | 2.58° | 0.06m |
| Gaussian-LIC2 (c2) | L + I + C | Hybrid | 0.03m | 0.43° | 0.05m | 0.59° | 0.05m | 2.50° | 0.02m |
| Gaussian-LIC2 (c2-w/o-d) | L + I + C | Hybrid | 7.12m | 6.22° | 0.45m | 0.80° | 3.27m | 5.92° | 0.10m |
including both NeRF-based and 3DGS-based. 1) NeRF-based: As no related LiDAR-Camera system is publicly available, we adapt the state-of-the-art RGB-D NeRF-based method Co-SLAM Wang et al. (2023a) with pseudo RGB-D input by merging LiDAR scans. When rendering the full image, we use SPNet Wang et al. (2024a) to complete the LiDAR depth maps for depth-guided sampling. 2) 3DGS-based: We compare with the state-of-the-art RGB-only method MonoGS Matsuki et al. (2024), which elegantly derives pose gradients. Given the scarcity of available LiDAR-Camera 3DGS-based methods, we run the state-of-the-art RGB-D 3DGS-based approach SplaTAM Keetha et al. (2024) using pseudo RGB-D images. We also compare against the RGB-D inertial method, MM3DGS-SLAM Sun et al. (2024), which extends SplaTAM by incorporating IMU data, and include a comparison with our previous LiDAR-Inertial-Camera 3DGS-based SLAM framework Gaussian-LIC Lang et al. (2025), where the sky modeling is disabled to enable valid depth rendering. Additional comparison results, including those against the recent LiDAR-Inertial-Camera-based method GS-LIVM Xie et al. (2025), are provided in the supplementary material (Sec. A.1). For fairness, all methods are evaluated without loop closure and post-processing.
### 9.3 Experiment-1: Evaluation of Localization
We validate the robustness and accuracy of Gaussian-LIC2 tracking in both challenging degenerate and large-scale environments. In particular, we compare two types of camera factors presented in Sec. 5.2: option 1 (LiDAR map reprojection constraint, Gaussian-LIC2 (c1)) and option 2 (Gaussian map photometric constraint, Gaussian-LIC2 (c2)). Also, we investigate the effectiveness of the camera factor option 2 when the depth supervision (Eq. (22)) is disabled, denoted as Gaussian-LIC2 (c2-w/o-d). Gaussian-LIC2-LIO represents our LiDAR-Inertial odometry component without any constraints from images.
9.3.1 Challenging Degenerate Sequences Sequences degenerate_seq_00, degenerate_seq_01, LiDAR_Degenerate, and Visual_Challenge, provided in the R3LIVE and FASTLIVO datasets, exhibit severe degradation, such as the solid-state LiDAR with small FoV facing the ground or walls, and the camera facing textureless surfaces or undergoing
Figure 6. The trajectory and colored LiDAR point cloud map output by the continuous-time odometry efficiently fused with the Gaussian map. Even undergoing severe degradation when the solid-state LiDAR faces the plain ground, the odometry performs pose estimation accurately with minor start-to-end drift error, benefiting from the photometric constraints provided by the Gaussian map. (a)-(d) are the raw image of the current view and rendered images from poses predicted by the IMU, refined by Gaussian-map-based optimization, and further optimized through LiDAR-Inertial-Camera factor optimization, respectively. Color discrepancies between the ground-truth and rendered images reflect variable illumination and varying camera exposure, highlighting the importance of the D-SSIM loss in Eq. (17).
aggressive motions (Visual_Challenge). Tab. 2 reports the start-to-end drift (and rotation) errors. Methods that rely solely on the rendering loss from radiance field maps for localization, including MonoGS, Co-SLAM, SplaTAM, and MM3DGS-SLAM, achieve excellent localization accuracy in confined indoor scenarios with moderate view changes. However, they tend to fail in large-scale outdoor scenes with significant viewpoint variation, as the camera can easily enter regions where the map optimization has not yet converged. MonoGS is the only one that lacks an absolute scale, we align its estimated trajectory with that of Gaussian-LIC2 (c1) for obtaining the absolute scale.
---
Table 3. Localization Evaluation: The RMSE (m) of APE results on large-scale outdoor sequences in MCD dataset.
| tuhh_day_02 | tuhh_day_03 | tuhh_day_04 |
| MonoGS Matsuki et al. (2024) | 29.53 | 42.54 | 25.4 |
| Co-SLAM Wang et al. (2023a) | 47.26 | ✗ | ✗ |
| SplaTAM Keetha et al. (2024) | ✗ | 34.49 | ✗ |
| MM3DGS-SLAM Sun et al. (2024) | 17.53 | 20.14 | 26.33 |
| PIN-SLAM Pan et al. (2024) | 0.16 | 0.11 | 0.38 |
| DBA-Fusion Zhou et al. (2024b) | 1.86 | 1.25 | 2.71 |
| LVI-SAM Shan et al. (2021) | 0.14 | 0.13 | 0.18 |
| R3LIVE Lin and Zhang (2022) | 0.10 | 0.13 | 0.15 |
| FAST-LIVO2 Zheng et al. (2024) | 0.08 | 0.11 | 0.13 |
| Gaussian-LIC Lang et al. (2025) | 0.08 | 0.09 | 0.10 |
| Gaussian-LIC2 (c1) | 0.08 | 0.08 | 0.09 |
| Gaussian-LIC2 (c2) | 0.08 | 0.09 | 0.09 |
MM3DGS-SLAM, aided by IMU, completes the sequence degenerate\_seq\_00, but still fails on the rest. Based on a point-based NeRF presentation, PIN-SLAM optimizes the poses via point-to-model SDF loss Pan et al. (2024), which is similar to the point-to-plane metric in ICP. It builds a purely geometric map and achieves robust pose optimization even in challenging lighting conditions, but suffers in LiDAR degradation scenarios where environmental structures fail to provide sufficient constraints for pose optimization. Moreover, relying solely on the LiDAR data and constant velocity assumption for initial pose, PIN-SLAM crashes on the sequence Visual\_Challenge where aggressive motions happen. As a learning-based visual-inertial odometry system, DBA-Fusion exhibits greater robustness to LiDAR degradation, and its tight fusion with IMU data facilitates handling of aggressive motions. Nevertheless, excessive view changes in the sequence Visual\_Challenge adversely affect the accuracy of the predicted dense optical flow thus significantly deteriorates the pose tracking in DBA-Fusion.
Among all the comparisons, LiDAR-Inertial-Camera fusion-based methods with each module meticulously designed showcase more robust and accurate performance. Rather than separately fusing LiDAR-inertial and visual-inertial data like LVI-SAM, R3LIVE, and FAST-LIVO2, our method maintains a single unified system that jointly and tightly integrates LiDAR-Inertial-Camera data within a continuous-time factor graph, achieving superior overall performance. Note that due to the lack of rigorous hardware synchronization on the sequences degenerate_seq_00-01, the performance of FAST-LIVO2 is impacted. Fig. 6 shows the trajectory and colored LiDAR map produced by Gaussian-LIC2 (c2), where the odometry successfully overcomes severe LiDAR degradation challenges and returns to the origin with minor drift. A set of subfigures in Fig. 6 presents rendered images from three different viewpoints corresponding to poses predicted by the IMU, refined by Gaussian-map-based optimization ( $^W_C\tilde{T}$ in Eq. (18)), and further refined through continuous-time LiDAR-Inertial-Camera factor graph optimization (Sec. 5.3). The increasing similarity between the rendered and raw images visually demonstrates the progressive refinement of the estimated pose. Interestingly, Gaussian-LIC2 (c2) outperforms Gaussian-LIC2 (c1) on the Visual Challenge sequence, which features textureless white walls. This improvement can be attributed to the photometric constraints derived from the Gaussian map, which enhance the robustness of Gaussian-LIC2 (c2) in low-texture environments compared to the optical-flow-based Gaussian-LIC2 (c1). Without depth supervision, Gaussian-LIC2 (c2-w/o-d) shows reduced accuracy. The depth regularization prevents the Gaussian map from overfitting to training views, which enables more robust novel view synthesis with fewer artifacts, as discussed in Sec. 9.4. This is critical since localization based on Gaussian maps inherently requires continuous novel view synthesis from varying perspectives. More results on other severe degenerate sequences from FAST-LIVO2 Zheng et al. (2024) are available in the supplementary material (Sec. A.3).
9.3.2 Large-Scale Outdoor Sequences Tab. 3 displays the localization error across different methods in the large-scale scenes without sensor degradation. Rendering-based methods including MonoGS, Co-SLAM, SplaTAM, and MM3DGS-SLAM, exhibit reduced accuracy in pose estimation. Although MM3DGS-SLAM benefits from IMU-based pose initialization, its loosely-coupled fusion without accounting for IMU bias limits accuracy. In contrast, PIN-SLAM and DBA-Fusion perform better, with PIN-SLAM in particular approaching the performance of traditional multi-sensor-based methods. Attributed to the continuous-time trajectory representation, which effectively handles LiDAR distortion and efficiently fuses high-rate IMU data, our method achieves the best localization accuracy. In non-degenerate scenarios, visual information has a relatively minor impact on tracking accuracy, resulting in similar performance between Gaussian-LIC2 (c1) and Gaussian-LIC2 (c2).
### 9.4 Experiment-2: Evaluation of Mapping
9.4.1 Evaluation Protocols We assess the performance of photo-realistic mapping by evaluating the quality of rendered images generated from the Gaussian map. To this end, we adopt several widely used metrics, including Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) Kerbl et al. (2023). Consistent with MonoGS and SplaTAM, we use AlexNet Krizhevsky et al. (2012) as the backbone network for LPIPS evaluation. When ground-truth depth maps are available, we also evaluate geometric accuracy using Depth-L1 error computed over valid depth regions. Additionally, to better compare the geometric quality, we further assess the commonly used surface reconstruction metrics, namely accuracy, completeness, Chamfer-L1 distance, and F-score. It should be noted that we adopt Gaussian-LIC2 (c1) in the mapping evaluation and all the following experiments.
Many existing methods Xie et al. (2025); Zhao et al. (2025); Hong et al. (2025) report rendering performance at their respective training views. However, such evaluations may be biased due to overfitting and fail to reliably reflect photo-realistic mapping quality at novel views. To ensure a comprehensive and fair evaluation, we assess rendering quality on both in-sequence and out-of-sequence novel views, explicitly excluding training views.
For fair evaluations, we provide accurate poses for all methods to isolate mapping performance from pose estimation errors, and all methods are trained using the same keyframe set (Sec. 6.1), with non-keyframes reserved for in-sequence novel view rendering. Specifically, for our self-collected dataset where ground-truth poses are available, all methods
---
Figure 7. Qualitative results of RGB rendering (in-sequence novel view) on public datasets. Regions not observed by the LiDAR throughout the entire process are indicated by the red arrow, floaters are highlighted with green boxes, and key details are marked with yellow boxes. Gaussian-LIC2 consistently achieves sharper visual effects with fewer artifacts.
---
Figure 8. Qualitative results of depth rendering (in-sequence novel view) on public datasets. Sparse LiDAR depth at the viewpoints is also shown in the second column. Note that the last row shows the rendering results on the M2DGR dataset, which is impacted by the rolling shutter effect. Gaussian-LIC2 consistently achieves sharper visual effects with clearer geometric structures.
---
Table 4. Evaluation of RGB rendering (in-sequence novel view) on public datasets.
| Sequence | Rendering Performance (PSNR $\uparrow$ SSIM $\uparrow$ LPIPS $\downarrow$) | |
| MonoGS | Co-SLAM | MM3DGS-SLAM | Gaussian-LIC | Gaussian-LIC2 | |
| R3LIVE (Avia) | | | | | | | | | | | | | | |
| degenerate_seq_00 | 14.59 | 0.460 | 0.624 | 14.42 | 0.286 | 0.810 | 19.35 | 0.698 | 0.212 | 21.94 | 0.777 | 0.171 | 21.99 | 0.782 |
| degenerate_seq_01 | 14.66 | 0.438 | 0.692 | 14.88 | 0.267 | 0.822 | 18.57 | 0.621 | 0.277 | 22.80 | 0.787 | 0.169 | 22.94 | 0.790 |
| hku_campus_00 | 15.41 | 0.464 | 0.675 | 15.50 | 0.302 | 0.794 | 18.70 | 0.621 | 0.319 | 24.38 | 0.787 | 0.162 | 24.34 | 0.787 |
| hku_campus_01 | 8.39 | 0.065 | 0.873 | 12.92 | 0.294 | 0.796 | 16.97 | 0.593 | 0.271 | 20.55 | 0.671 | 0.275 | 20.52 | 0.674 |
| hku_park_00 | 13.24 | 0.319 | 0.733 | 14.02 | 0.218 | 0.790 | 14.98 | 0.363 | 0.396 | 17.09 | 0.466 | 0.354 | 17.17 | 0.469 |
| hku_park_01 | 12.22 | 0.289 | 0.790 | 13.65 | 0.207 | 0.815 | 15.79 | 0.392 | 0.409 | 19.45 | 0.529 | 0.340 | 19.46 | 0.529 |
| FAST-LIVO (Avia) | | | | | | | | | | | | | | |
| LiDAR_Degenerate | 25.44 | 0.779 | 0.645 | 23.59 | 0.622 | 0.496 | 28.58 | 0.828 | 0.157 | 30.09 | 0.829 | 0.155 | 30.36 | 0.831 |
| Visual_Challenge | 15.36 | 0.563 | 0.674 | 15.96 | 0.365 | 0.711 | 17.62 | 0.712 | 0.240 | 23.26 | 0.821 | 0.162 | 23.44 | 0.822 |
| hku1 | 13.44 | 0.385 | 0.787 | 15.98 | 0.269 | 0.738 | 21.76 | 0.713 | 0.162 | 23.82 | 0.757 | 0.153 | 23.74 | 0.758 |
| hku2 | 21.24 | 0.559 | 0.691 | 20.07 | 0.398 | 0.631 | 26.29 | 0.754 | 0.156 | 29.11 | 0.798 | 0.153 | 29.08 | 0.798 |
| Retail_Street | 18.74 | 0.574 | 0.537 | 17.32 | 0.375 | 0.669 | 21.55 | 0.694 | 0.162 | 24.15 | 0.770 | 0.128 | 24.37 | 0.775 |
| CBD_Building_01 | 17.11 | 0.630 | 0.640 | 18.16 | 0.508 | 0.644 | 22.13 | 0.812 | 0.126 | 25.16 | 0.851 | 0.104 | 25.20 | 0.851 |
| MCD (OS1-64) | | | | | | | | | | | | | | |
| tuhh_day_02 | 8.72 | 0.158 | 0.893 | 12.65 | 0.295 | 0.766 | 11.15 | 0.478 | 0.350 | 19.99 | 0.621 | 0.312 | 20.35 | 0.626 |
| tuhh_day_03 | 8.09 | 0.136 | 0.896 | 12.48 | 0.407 | 0.684 | 11.36 | 0.542 | 0.301 | 21.09 | 0.666 | 0.273 | 21.38 | 0.672 |
| tuhh_day_04 | 11.73 | 0.250 | 0.805 | 13.02 | 0.413 | 0.626 | 13.86 | 0.384 | 0.398 | 19.27 | 0.528 | 0.329 | 19.27 | 0.528 |
| M2DGR (VLP-32C) | | | | | | | | | | | | | | |
| room_01 | 14.92 | 0.573 | 0.643 | 15.01 | 0.317 | 0.880 | 12.27 | 0.554 | 0.458 | 17.04 | 0.697 | 0.365 | 17.45 | 0.721 |
| room_02 | 13.57 | 0.515 | 0.697 | 14.94 | 0.318 | 0.877 | 11.44 | 0.565 | 0.465 | 17.32 | 0.705 | 0.375 | 17.72 | 0.725 |
| room_03 | 16.15 | 0.627 | 0.587 | 14.88 | 0.370 | 0.841 | 12.50 | 0.573 | 0.475 | 17.19 | 0.701 | 0.401 | 17.38 | 0.709 |
use ground-truth poses, while on public datasets lacking ground-truth, both our method and the baselines are provided with our estimated poses for evaluation.
9.4.2 In-Sequence Novel View Synthesis Tab. 4 and Tab. 5 show quantitative rendering results of in-sequence novel views across both public and self-collected datasets. Fig. 7 and Fig. 8 present the novel view renderings of RGB and depth in sequence. The RGB-only method MonoGS initializes Gaussians at a preset constant depth with random noise and subsequently inserts new Gaussians based on the rendered depth statistics. While the optimization of the inserted Gaussians may gradually converge in small-scale indoor scenarios, it becomes much more challenging in larger scenes, where misplaced Gaussians accumulate, causing floaters and poor visual quality. Incorporating LiDAR geometric priors can substantially alleviate this problem. For example, Co-SLAM leverages the LiDAR depth for ray sampling and supervision of the neural implicit map optimization, rendering relatively clear structures. It is capable of rendering beyond the LiDAR FoV based on the optimized neural representation, as shown in Fig. 7 and Fig. 8. However, the renderings of Co-SLAM appear noisy with severe artifacts, even when guided by completed depth maps. In contrast, the LiDAR-incorporated 3DGS-based methods MM3DGS-SLAM and Gaussian-LIC exhibit better rendering performance. They reliably initialize Gaussians from precise LiDAR points. Nonetheless, MM3DGS-SLAM represents the scene as isotropic and view-independent. Gaussians to accelerate, but unfortunately still fails to achieve real-time performance while sacrificing visual quality. Note that MM3DGS-SLAM uses around three times as many Gaussians as our method to model the scene. Gaussian-LIC underutilizes precise LiDAR depth and ignores to optimize the geometric quality of the Gaussian map, leading to poor-quality rendered depth. In addition, both methods struggle to accurately reconstruct areas unobserved by the LiDAR.
Gaussian-LIC2 attains the best in-sequence novel view rendering performance, both quantitatively and qualitatively. The system renders sharper RGB images with fewer artifacts at novel views, beyond only overfitting to the training views. It successfully reconstructs regions never scanned by the LiDAR, such as the ceiling beyond the reach of the VLP-32C, as demonstrated in the last row of Fig. 7. Moreover, despite relying on sparse LiDAR depth, our method is able to produce high-quality depth maps across different LiDAR modalities. It is worth noting that severe rolling shutter distortion in the images of the M2DGR dataset affects the quality of the rendered depth, as illustrated in the last row of Fig. 8.
9.4.3 Out-of-Sequence Novel View Synthesis We further evaluate out-of-sequence novel view rendering on our self-collected dataset, which presents a greater challenge than in-sequence rendering. For this assessment, views are sampled at 10 Hz along the out-of-sequence trajectory. As shown in Tab. 5, our method achieves the best results. Fig. 9 illustrates the camera trajectories and the corresponding novel view rendering results along the out-of-sequence path. Compared to Gaussian-LIC, the Gaussian map of Gaussian-LIC2 is more effectively regularized and optimized, leading to sharper and higher-fidelity novel view synthesis with fewer artifacts.
9.4.4 Surface Reconstruction Tab. 6 reports the geometric quality quantitatively on our self-collected dataset, where the denoised ground-truth point cloud is provided. The metrics are calculated from the ground-truth and the predicted mesh (Sec. 10.2). Note that we compute F-score with a 0.2m error threshold. As can be seen, with visual quality and real-time performance as prerequisites, Gaussian-LIC2 achieves more accurate and more complete reconstruction of the environment.
---
Table 5. Evaluation of RGB and depth rendering (in-sequence & out-of-sequence novel view) on our datasets. Depth-L1 (m).
| LiDAR | Sequence | Method | In-Sequence Novel View | Out-of-Sequence Novel View |
| PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Depth-L1 $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Depth-L1 $\downarrow$ |
| Mid-360 | Liberal_Arts_Group_01 | MM3DGS-SLAM | 22.89 | 0.721 | 0.298 | 0.65 | 21.05 | 0.653 | 0.360 | 0.65 |
| Gaussian-LIC | 25.12 | 0.759 | 0.297 | 0.92 | 22.08 | 0.670 | 0.350 | 0.81 |
| Gaussian-LIC2 | 25.35 | 0.761 | 0.278 | 0.29 | 22.10 | 0.671 | 0.332 | 0.30 |
| Liberal_Arts_Group_02 | MM3DGS-SLAM | 17.96 | 0.574 | 0.271 | 0.67 | 16.98 | 0.444 | 0.351 | 0.72 |
| Gaussian-LIC | 23.02 | 0.703 | 0.210 | 0.66 | 19.95 | 0.520 | 0.275 | 0.71 |
| Gaussian-LIC2 | 23.42 | 0.713 | 0.186 | 0.30 | 20.05 | 0.528 | 0.256 | 0.37 |
| Liberal_Arts_Group_03 | MM3DGS-SLAM | 17.74 | 0.493 | 0.385 | 0.96 | 13.87 | 0.347 | 0.478 | 1.22 |
| Gaussian-LIC | 22.59 | 0.656 | 0.236 | 0.89 | 18.28 | 0.458 | 0.351 | 1.25 |
| Gaussian-LIC2 | 22.79 | 0.662 | 0.222 | 0.44 | 18.28 | 0.459 | 0.345 | 0.67 |
| Medical_Building_01 | MM3DGS-SLAM | 16.45 | 0.642 | 0.266 | 1.24 | 16.43 | 0.574 | 0.321 | 1.34 |
| Gaussian-LIC | 22.77 | 0.737 | 0.199 | 1.28 | 19.40 | 0.632 | 0.268 | 1.64 |
| Gaussian-LIC2 | 22.89 | 0.741 | 0.188 | 0.62 | 19.54 | 0.633 | 0.257 | 0.84 |
| Medical_Building_02 | MM3DGS-SLAM | 16.29 | 0.685 | 0.217 | 0.91 | 19.62 | 0.679 | 0.222 | 0.75 |
| Gaussian-LIC | 23.15 | 0.758 | 0.177 | 1.53 | 21.33 | 0.685 | 0.216 | 1.35 |
| Gaussian-LIC2 | 23.11 | 0.760 | 0.174 | 0.54 | 21.57 | 0.691 | 0.213 | 0.53 |
| Avia | Lecture_Hall | MM3DGS-SLAM | 19.54 | 0.671 | 0.198 | 0.74 | 18.73 | 0.624 | 0.236 | 0.75 |
| Gaussian-LIC | 21.60 | 0.710 | 0.178 | 0.68 | 20.32 | 0.669 | 0.207 | 0.73 |
| Gaussian-LIC2 | 21.78 | 0.719 | 0.167 | 0.39 | 20.40 | 0.672 | 0.205 | 0.45 |
| Robot_Center | MM3DGS-SLAM | 18.83 | 0.631 | 0.190 | 0.59 | 16.63 | 0.558 | 0.260 | 0.47 |
| Gaussian-LIC | 22.89 | 0.693 | 0.206 | 0.92 | 18.81 | 0.600 | 0.264 | 0.79 |
| Gaussian-LIC2 | 22.95 | 0.699 | 0.188 | 0.29 | 18.80 | 0.602 | 0.257 | 0.25 |
| Bell_Tower_01 | MM3DGS-SLAM | 18.13 | 0.600 | 0.300 | 0.67 | 16.40 | 0.592 | 0.299 | 0.74 |
| Gaussian-LIC | 25.38 | 0.717 | 0.282 | 0.92 | 22.87 | 0.676 | 0.324 | 0.91 |
| Gaussian-LIC2 | 25.53 | 0.720 | 0.256 | 0.27 | 22.87 | 0.676 | 0.298 | 0.35 |
| Bell_Tower_02 | MM3DGS-SLAM | 16.49 | 0.661 | 0.278 | 0.87 | 17.01 | 0.620 | 0.320 | 1.14 |
| Gaussian-LIC | 26.42 | 0.751 | 0.285 | 1.18 | 23.68 | 0.716 | 0.321 | 1.77 |
| Gaussian-LIC2 | 26.85 | 0.756 | 0.242 | 0.29 | 24.17 | 0.708 | 0.276 | 0.41 |
| Bell_Tower_03 | MM3DGS-SLAM | 16.61 | 0.651 | 0.256 | 0.90 | 14.08 | 0.603 | 0.294 | 1.02 |
| Gaussian-LIC | 27.29 | 0.748 | 0.296 | 1.47 | 25.02 | 0.735 | 0.278 | 1.68 |
| Gaussian-LIC2 | 27.63 | 0.756 | 0.244 | 0.33 | 25.43 | 0.738 | 0.238 | 0.42 |
Table 6. Evaluation of surface reconstruction on self-collected dataset. Metrics in meters: accuracy (Acc.), completeness (Comp.), Chamfer-L1 distance (C-L1). F-score in % is calculated with a 0.2m error threshold. The results are averaged separately over the Mid-360 dataset and the Avia dataset. RT means real time. Best results are in bold and second-best results are underlined. Gaussian-LIC2* means the modified version in Sec. 10.2 for the specific application of rapid mesh production.
| LiDAR | Method | Acc. $\downarrow$ | Comp. $\downarrow$ | C-L1 $\downarrow$ | F-score $\uparrow$ | PSNR $\uparrow$ | RT |
| Mid-360 | Co-SLAM | 0.197 | 0.461 | 0.333 | 49.81 | 16.15 | ✗ |
| MM3DGS-SLAM | 0.181 | 0.402 | 0.292 | 58.32 | 18.27 | ✗ |
| Gaussian-LIC | 0.189 | 0.424 | 0.326 | 51.49 | 23.33 | ✓ |
| Gaussian-LIC2 | 0.133 | 0.218 | 0.186 | 70.77 | 23.51 | ✓ |
| Gaussian-LIC2 $^{{*}}$ | 0.125 | 0.178 | 0.162 | 75.34 | 21.95 | ✓ |
| Avia | Co-SLAM | 0.226 | 0.490 | 0.351 | 46.42 | 13.55 | ✗ |
| MM3DGS-SLAM | 0.152 | 0.377 | 0.264 | 61.21 | 17.92 | ✗ |
| Gaussian-LIC | 0.149 | 0.442 | 0.295 | 59.21 | 24.72 | ✓ |
| Gaussian-LIC2 | 0.107 | 0.189 | 0.147 | 75.68 | 24.95 | ✓ |
| Gaussian-LIC2 $^{{*}}$ | 0.081 | 0.153 | 0.118 | 83.72 | 23.99 | ✓ |
than the compared methods. Furthermore, as in the application described in Sec. 10.2, by sacrificing a small amount of visual quality, Gaussian-LIC2 can achieve higher geometric quality, which can be specifically utilized for rapid mesh generation.
As a SLAM system, our method incrementally estimates poses and constructs a photo-realistic map from sequential sensor data, enabling real-time perception for robotic applications. In contrast, offline methods disregard incremental or online processing capabilities and typically perform heavy,
### 9.5 Experiment-3: Offline Gaussian Mapping
computationally intensive optimization only after all sensor data has been collected. Existing offline 3DGS mapping methods typically fall into two paradigms, namely per-scene optimization and generalizable feed-forward models.
9.5.1 Per-Scene Optimization We first compare our method to the state-of-the-art LiDAR-based 3DGS framework, LetsGo Cui et al. (2024), on the 105-second Lecture-Hall sequence. To ensure a fair comparison, LetsGo is provided with the same sparse LiDAR depth maps, our estimated poses, and the same set of training views, trained for 30,000 iterations. Tab. 7 presents the results, reporting the total time from data acquisition start to reconstruction completion. Our method achieves comparable performance while completing the reconstruction within the duration of data acquisition, whereas LetsGo requires time-consuming optimization after receiving the full dataset. Furthermore, our method significantly outperforms LetsGo after an additional 10,000 iterations of refinement (requiring only 24 seconds). This demonstrates that our approach not only supports real-time reconstruction but can also achieve higher accuracy when post-acquisition optimization is permitted.
9.5.2 Feed-Forward Model We also compare our method with the state-of-the-art feed-forward Gaussian mapping approach, MVSplat Chen et al. (2024b), which predicts a Gaussian map from a small set of posed images (two by default). For achieving good performance with MVSplat, we carefully select a pair of images that provides sufficient
---
Figure 9. Qualitative results of RGB and depth rendering (out-of-sequence novel view) on our self-collected dataset. (a-d) The green path represents the trajectory for collecting training views, the red path shows the out-of-sequence trajectory for evaluation, and the yellow stars indicate the selected out-of-sequence novel views. The sky regions in rendered depth maps are masked in black with the segmentation model.
Table 7. Comparison with LiDAR-based offline Gaussian mapping method on the sequence Lecture_Hall (span: 105 seconds). The total time consumption reported here is the duration of data acquisition time + reconstruction time.
| PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Depth-L1 (m) $\downarrow$ | Time (s) $\downarrow$ |
| LetsGo Cui et al. (2024) | 21.81 | 0.708 | 0.170 | 0.37 | 105 + 347 |
| Gaussian-LIC2 | 21.78 | 0.719 | 0.167 | 0.39 | 105 |
| Gaussian-LIC2-W-Refine | 22.28 | 0.730 | 0.148 | 0.37 | 105 + 24 |
Figure 10. Comparison with the feed-forward method, MVSplat Chen et al. (2024b). Image renderings from both MVSplat and Gaussian-LIC2 are shown.
Table 8. Runtime Analysis (Time unit: s). Fwd.: forward time; Bwd.: backward time; Adam: optimizer update time; Total: total mapping time; Dur.: rosbag duration; Len.: trajectory length (m).
| Fwd. | Bwd. | Adam | Total | Dur. | Len. |
| R3LIVE |
| degenerate_seq_00 | 9 | 16 | 9 | 38 | 87 | 42 |
| degenerate_seq_01 | 11 | 15 | 8 | 38 | 86 | 61 |
| hku_campus_seq_00 | 69 | 69 | 45 | 195 | 202 | 172 |
| hku_campus_seq_01 | 154 | 73 | 48 | 285 | 304 | 337 |
| hku_park_00 | 62 | 87 | 55 | 218 | 228 | 212 |
| hku_park_01 | 138 | 112 | 73 | 340 | 362 | 354 |
| FAST-LIVO |
| LiDAR_Degenerate | 7 | 12 | 6 | 29 | 78 | 38 |
| Visual_Challenge | 18 | 29 | 14 | 71 | 162 | 78 |
| hku1 | 20 | 28 | 15 | 71 | 128 | 64 |
| hku2 | 13 | 21 | 11 | 51 | 105 | 59 |
| Retail_Street | 15 | 26 | 15 | 64 | 135 | 66 |
| CBD_Building_01 | 12 | 19 | 10 | 48 | 119 | 33 |
| MCD |
| tuhh_day_02 | 74 | 57 | 35 | 179 | 200 | 306 |
| tuhh_day_03 | 70 | 56 | 34 | 173 | 200 | 276 |
| tuhh_day_04 | 43 | 46 | 27 | 127 | 187 | 298 |
| M2DGR |
| room01 | 9 | 17 | 11 | 40 | 75 | 27 |
| room02 | 8 | 16 | 10 | 38 | 89 | 46 |
| room03 | 21 | 43 | 28 | 100 | 195 | 71 |
| Self-collected |
| Liberal_Arts_Group_01 | 8 | 9 | 3 | 24 | 101 | 42 |
| Liberal_Arts_Group_02 | 23 | 35 | 16 | 88 | 225 | 59 |
| Liberal_Arts_Group_03 | 39 | 50 | 25 | 132 | 285 | 102 |
| Medical_Building_01 | 9 | 10 | 4 | 28 | 113 | 53 |
| Medical_Building_02 | 10 | 12 | 4 | 32 | 120 | 38 |
| Lecture_Hall | 9 | 15 | 6 | 36 | 105 | 45 |
| Robot_Center | 18 | 25 | 13 | 63 | 136 | 53 |
| Bell_Tower_01 | 36 | 30 | 15 | 90 | 150 | 67 |
| Bell_Tower_02 | 13 | 18 | 9 | 45 | 103 | 39 |
| Bell_Tower_03 | 18 | 19 | 9 | 52 | 120 | 48 |
parallax and maximal scene coverage, using poses estimated by our method. Fig. 10 shows the rendering results on the degenerate\_seq\_00 and hku1 sequences. While MVSplat is capable of predicting pixel-aligned Gaussians within seconds, it often produces blurry renderings and suffers from parallax sensitivity. On the contrary, our method consistently generates sharp, photo-realistic images.
### 9.6 Runtime Analysis
We evaluate the real-time performance across all datasets. Following the definition in Gaussian-LIC Lang et al. (2025), a system is considered to be real-time capable if it completes
---
Figure 11. Application – Video Frame Interpolation: The images in the middle are the interpolated frames at the intermediate timestamps of the left and right images.
processing within the duration of sensor data acquisition, without extensive post-processing. We evaluate every module of Gaussian-LIC(c1) during runtime analysis. The average pose estimation frequency is 10 Hz, while depth completion inference requires only 10–20 ms per frame. Therefore, we primarily focus on the time consumption in the mapping thread. As shown in Tab. 8, the most time-consuming components are primarily the forward and backward pass of the rasterizer as well as the Adam optimizer update. Thanks to carefully designed CUDA-related acceleration strategies in Sec. 7.2, the total mapping time remains within the bag duration, showcasing the real-time capability of our method. We also report the runtime of other methods and CUDA ablation in the supplementary material (Sec. A.2). The real-time demonstration video can be found on the website $^{\dagger}$.
## 10 Applications
### 10.1 Video Frame Interpolation
The continuous-time trajectory enables pose querying at any valid timestamp, while the Gaussian map allows rendering from arbitrary viewpoints. Interestingly, combining these capabilities facilitates spatiotemporal interpolation, which can be applied to video frame interpolation. As in Fig. 11, after obtaining the continuous-time trajectory and Gaussian map from Gaussian-LIC2, we successfully double the frame rate of the sequence Visual\_Challenge from 15 Hz to 30 Hz by rendering intermediate frames at the queried mid-time poses.
### 10.2 Rapid 3D Mesh Extraction
Compared to previous LiDAR-Inertial-Camera 3DGS-based SLAM methods, Gaussian-LIC2 balances visual quality, geometric accuracy, and computational efficiency. In this application, we showcase how our method can be adapted for rapid meshing. We increase the depth regularization weight $\gamma$ (see Eq.(22)) to be 2, and run our system in real time within the duration of data acquisition. Subsequently, based on the reconstructed Gaussian map, we render RGB images and depth maps from all viewpoints. These rendered depth maps are fused using TSDF fusion Curless and Levoy (1996) (with a voxel size of 0.05m) to reconstruct a 3D-consistent map. An accurate mesh is then extracted from the TSDF volume using the Marching Cubes algorithm Lorenzen and Cline (1998). Fig. 12 illustrates the resulting mesh, textured with RGB colors or colorized by surface normal directions.
## 11 Conclusions and Future Work
In this paper, We propose a novel LiDAR-Inertial-Camera Gaussian Splatting SLAM system that jointly considers visual quality, geometric accuracy, and real-time performance. Our method enables accurate pose estimation and photorealistic map construction in real time, supporting high-quality RGB and depth rendering. Markedly, we incorporate a zero-shot depth completion model that fuses RGB and sparse LiDAR data to generate dense depth maps, which facilitates initialization of Gaussians in large-scale scenarios. The training of the Gaussian map is efficiently supervised by our curated sparse LiDAR depth and accelerated with meticulously designed CUDA-related strategies. Meanwhile, we explore jointly fusing the visual photometric constraints derived from the Gaussian map with the LiDAR-Inertial data within the continuous-time framework, effectively overcoming the LiDAR degeneration. We also extend our system to support downstream tasks such as video interpolation and rapid mesh generation. Finally, we introduce a dedicated LiDAR-Inertial-Camera dataset for benchmarking photometric and geometric mapping in large-scale scenarios, with ground-truth poses and depth maps, as well as out-of-sequence trajectories. Extensive experiments show that our system outperforms existing methods in various aspects.
Our method enhances the real-time perception capabilities of mobile robotic systems in large-scale scenarios. However, several limitations remain for future work. (1) Map Compactness: We currently do not impose constraints on map size, which may result in large memory usage. It is worth investigating to reduce the number of Gaussians while maintaining the map quality. (2) Geometric Accuracy: There remains a trade-off between visual quality and geometric accuracy. We aim to further improve geometric precision without sacrificing visual fidelity. (3) Applicability in Giga Scene: In extremely large-scale environments, our system may accumulate pose drift over time. We plan to introduce loop closures to mitigate this issue. (4) Integration with Foundation Models: A promising direction involves leveraging foundation models for feed-forward Gaussian prediction, combined with generative models, to improve novel view synthesis performance in out-of-sequence scenarios.
## Funding
This work was supported by the National Natural Science Foundation of China under Grant No. 62525309.
## References
Agarwal S, Mierle K and Others (2012) Ceres solver. .
Bochkovskii A, Delaunoy A, Germain H, Santos M, Zhou Y, Richter SR and Koltun V (2024) Depth pro: Sharp monocular metric depth in less than a second. arXiv preprint arXiv:2410.02073.
---
Figure 12. Application – Mesh Extraction: Textured and normal-colorized meshes generated from our reconstructed Gaussian map.
Campos C, Elvira R, Rodríguez JJG, Montiel JM and Tardós JD (2021) Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE transactions on robotics 37(6): 1874–1890.
Cao C, Ren X and Fu Y (2022) Mvsformer: Multi-view stereo by learning robust image features and temperature-based depth. arXiv preprint arXiv:2208.02541.
Charatan D, Li SL, Tagliasacchi A and Sitzmann V (2024) pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 19457–19467.
Chen A, Xu Z, Zhao F, Zhang X, Xiang F, Yu J and Su H (2021) Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 14124–14133.
Chen D, Li H, Ye W, Wang Y, Xie W, Zhai S, Wang N, Liu H, Bao H and Zhang G (2024a) Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction. IEEE Transactions on Visualization and Computer Graphics.
Chen H, Li C and Lee GH (2023a) Neusg: Neural implicit surface reconstruction with 3d gaussian splatting guidance. arXiv preprint arXiv:2312.00846.
Chen L, Zhan H, Chen K, Xu X, Yan Q, Cai C and Xu Y (2025a) Activegamer: Active gaussian mapping through efficient rendering. arXiv preprint arXiv:2501.06897.
Chen T, Shorinwa O, Bruno J, Swann A, Yu J, Zeng W, Nagami K, Dames P and Schwager M (2025b) Splat-nav: Safe real-time
robot navigation in gaussian splatting maps. IEEE Transactions on Robotics.
Chen Y, Gu C, Jiang J, Zhu X and Zhang L (2023b) Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXiv:2311.18561.
Chen Y, Xu H, Zheng C, Zhuang B, Pollefeys M, Geiger A, Cham TJ and Cai J (2024b) Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In: European Conference on Computer Vision. Springer, pp. 370–386.
Chung CM, Tseng YC, Hsu YC, Shi XQ, Hua YH, Yeh JF, Chen WC, Chen YT and Hsu WH (2023) Orbeez-slam: A real-time monocular visual slam with orb features and nerf-realized mapping. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 9400–9406.
Cong Y, Chen C, Yang B, Zhong R, Sun S, Xu Y, Yan Z, Zou X and Tu Z (2024) Or-lim: Observability-aware robust lidar-inertial-mapping under high dynamic sensor motion. ISPRS Journal of Photogrammetry and Remote Sensing 218: 610–627.
Cui J, Cao J, Zhao F, He Z, Chen Y, Zhong Y, Xu L, Shi Y, Zhang Y and Yu J (2024) Letsgo: Large-scale garage modeling and rendering via lidar-assisted gaussian primitives. ACM Transactions on Graphics (TOG) 43(6): 1–18.
Curless B and Levoy M (1996) A volumetric method for building complex models from range images. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. pp. 303–312.
---
Dai P, Xu J, Xie W, Liu X, Wang H and Xu W (2024) High-quality surface reconstruction using gaussian surfels. In: ACM SIGGRAPH 2024 Conference Papers. pp. 1–11.
Deng J, Wu Q, Chen X, Xia S, Sun Z, Liu G, Yu W and Pei L (2023) Nerf-loam: Neural implicit representation for large-scale incremental lidar odometry and mapping. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8218–8227.
Engel J, Koltun V and Cremers D (2017) Direct sparse odometry. IEEE transactions on pattern analysis and machine intelligence 40(3): 611–625.
Engel J, Schöps T and Cremers D (2014) Lsd-slam: Large-scale direct monocular slam. In: European conference on computer vision. Springer, pp. 834–849.
Fridovich-Keil S, Yu A, Tancik M, Chen Q, Recht B and Kanazawa A (2022) Plenoxels: Radiance fields without neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5501–5510.
Furgale P, Rehder J and Siegwart R (2013) Unified temporal and spatial calibration for multi-sensor systems. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 1280–1286.
Geneva P, Eckenhoff K, Lee W, Yang Y and Huang G (2020) Openvins: A research platform for visual-inertial estimation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 4666–4672.
Giacomini E, Di Giammarino L, De Rebotti L, Grisetti G and Oswald MR (2025) Splat-loam: Gaussian splatting lidar odometry and mapping. arXiv preprint arXiv:2503.17491.
Guédon A and Lepetit V (2024) Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5354–5363.
Hong S, He J, Zheng X, Zheng C and Shen S (2024) Liv-gaussmap: Lidar-inertial-visual fusion for real-time 3d radiance field map rendering. IEEE Robotics and Automation Letters.
Hong S, Zheng C, Shen Y, Li C, Zhang F, Qin T and Shen S (2025) Gs-livo: Real-time lidar, inertial, and visual multi-sensor fused odometry with gaussian mapping. arXiv preprint arXiv:2501.08672.
Hu M, Yin W, Zhang C, Cai Z, Long X, Chen H, Wang K, Yu G, Shen C and Shen S (2024) Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Huang B, Yu Z, Chen A, Geiger A and Gao S (2024a) 2d gaussian splatting for geometrically accurate radiance fields. In: ACM SIGGRAPH 2024 conference papers. pp. 1–11.
Huang H, Li L, Cheng H and Yeung SK (2024b) Photo-slam: Real-time simultaneous localization and photorealistic mapping for monocular stereo and rgb-d cameras. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21584–21593.
Isaacson S, Kung PC, Ramanagopal M, Vasudevan R and Skinner KA (2023) Loner: Lidar only neural representations for real-time slam. IEEE Robotics and Automation Letters 8(12): 8042–8049.
Jiang C, Gao R, Shao K, Wang Y, Xiong R and Zhang Y (2024) Li-gs: Gaussian splatting with lidar incorporated for accurate large-scale reconstruction. IEEE Robotics and Automation Letters
Jiang C, Zhang H, Liu P, Yu Z, Cheng H, Zhou B and Shen S (2023) H_{2}mapping: Real-time dense mapping using hierarchical hybrid representation. IEEE Robotics and Automation Letters 8(10): 6787–6794.
Jin R, Gao Y, Wang Y, Wu Y, Lu H, Xu C and Gao F (2024) Gs-planner: A gaussian-splatting-based planning framework for active high-fidelity reconstruction. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 11202–11209.
Johari MM, Carta C and Fleuret F (2023) Eslam: Efficient dense slam system based on hybrid representation of signed distance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17408–17419.
Keetha N, Karhade J, Jatavallabhula KM, Yang G, Scherer S, Ramanan D and Luiten J (2024) Splatam: Splat track & map 3d gaussians for dense rgb-d slam. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21357–21366.
Kerbl B, Kopanas G, Leimkühler T and Drettakis G (2023) 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42(4).
Krizhevsky A, Sutskever I and Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25.
Lang X, Chen C, Tang K, Ma Y, Lv J, Liu Y and Zuo X (2023) Coco-lic: continuous-time tightly-coupled lidar-inertial-camera odometry using non-uniform b-spline. IEEE Robotics and Automation Letters 8(11): 7074–7081.
Lang X, Li L, Wu C, Zhao C, Liu L, Liu Y, Lv J and Zuo X (2025) Gaussian-lic: Real-time photo-realistic slam with a gaussian splatting and lidar-inertial-camera fusion. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 8500–8507.
Lang X, Lv J, Huang J, Ma Y, Liu Y and Zuo X (2022) Ctrl-vio: Continuous-time visual-inertial odometry for rolling shutter cameras. IEEE Robotics and Automation Letters 7(4): 11537–11544.
Lei X, Wang M, Zhou W and Li H (2025) Gaussnav: Gaussian splatting for visual navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Li H, Zou Y, Chen N, Lin J, Liu X, Xu W, Zheng C, Li R, He D, Kong F et al. (2024) Mars-lvig dataset: A multi-sensor aerial robots slam dataset for lidar-visual-inertial-gnss fusion. The International Journal of Robotics Research 43(8): 1114–1127.
Lin H, Peng S, Chen J, Peng S, Sun J, Liu M, Bao H, Feng J, Zhou X and Kang B (2024) Prompting depth anything for 4k resolution accurate metric depth estimation. arXiv preprint arXiv:2412.14015.
Lin J and Zhang F (2022) R 3 live: A robust, real-time, rgb-colored, lidar-inertial-visual tightly-coupled state estimation and mapping package. In: 2022 International Conference on Robotics and Automation (ICRA). IEEE, pp. 10672–10678.
Lin J and Zhang F (2024) R $^{3}$LIVE++: A Robust, Real-time, Radiance Reconstruction Package with a Tightly-coupled LiDAR-Inertial-Visual State Estimator. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Liu J, Wan Y, Wang B, Zheng C, Lin J and Zhang F (2025) Gssdf: Lidar-augmented gaussian splatting and neural sdf for geometrically consistent rendering and reconstruction. arXiv
---
preprint arXiv:2503.10170.
Liu J, Zheng C, Wan Y, Wang B, Cai Y and Zhang F (2024a) Neural surface reconstruction and rendering for lidar-visual systems. arXiv preprint arXiv:2409.05310.
Liu Z, Cheng KL, Wang Q, Wang S, Ouyang H, Tan B, Zhu K, Shen Y, Chen Q and Luo P (2024b) Depthlab: From partial to complete. arXiv preprint arXiv:2412.18153.
Lorensen WE and Cline HE (1998) Marching cubes: A high resolution 3d surface construction algorithm. In: Seminal graphics: pioneering efforts that shaped the field. ACM Press, pp. 347–353.
Lu T, Yu M, Xu L, Xiangli Y, Wang L, Lin D and Dai B (2024) Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20654–20664.
Lucas BD and Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: IJCAI'81: 7th international joint conference on Artificial intelligence, volume 2. pp. 674–679.
Lv J, Hu K, Xu J, Liu Y, Ma X and Zuo X (2021) Clins: Continuous-time trajectory estimation for lidar-inertial system. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 6657–6663.
Lv J, Lang X, Xu J, Wang M, Liu Y and Zuo X (2023) Continuous-time fixed-lag smoothing for lidar-inertial-camera slam. IEEE/ASME Transactions on Mechatronics 28(4): 2259–2270.
Mallick SS, Goel R, Kerbl B, Steinberger M, Carrasco FV and De La Torre F (2024) Taming 3dgs: High-quality radiance fields with limited resources. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11.
Matsuki H, Murai R, Kelly PH and Davison AJ (2024) Gaussian splatting slam. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18039–18048.
Mildenhall B, Srinivasan PP, Tancik M, Barron JT, Ramamoorthi R and Ng R (2021) Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65(1): 99–106.
Müller T, Evans A, Schied C and Keller A (2022) Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG) 41(4): 1–15.
Naumann J, Xu B, Leutenegger S and Zuo X (2024) Nerf-vo: Real-time sparse visual odometry with neural radiance fields. IEEE Robotics and Automation Letters.
Newcombe RA, Izadi S, Hilliges O, Molyneaux D, Kim D, Davison AJ, Kohi P, Shotton J, Hodges S and Fitzgibbon A (2011a) Kinectfusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE international symposium on mixed and augmented reality. Ieee, pp. 127–136.
Newcombe RA, Lovegrove SJ and Davison AJ (2011b) Dtam: Dense tracking and mapping in real-time. In: 2011 international conference on computer vision. IEEE, pp. 2320–2327.
Nguyen TM, Yuan S, Nguyen TH, Yin P, Cao H, Xie L, Wozniak M, Jensfelt P, Thiel M, Ziegenbein J et al. (2024) Mcd: Diverse large-scale multi-campus dataset for robot perception. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22304–22313.
Pan Y, Zhong X, Jin L, Wiesmann L, Popović M, Behley J and Stachniss C (2025) Pings: Gaussian splatting meets distance fields within a point-based implicit neural map. arXiv preprint
arXiv:2502.05752.
Pan Y, Zhong X, Wiesmann L, Posewsky T, Behley J and Stachniss C (2024) Pin-slam: Lidar slam using a point-based implicit neural representation for achieving global map consistency. IEEE Transactions on Robotics.
Peng Z, Shao T, Liu Y, Zhou J, Yang Y, Wang J and Zhou K (2024) Rtg-slam: Real-time 3d reconstruction at scale using gaussian splatting. In: ACM SIGGRAPH 2024 Conference Papers. pp. 1–11.
Pizzoli M, Forster C and Scaramuzza D (2014) Remode: Probabilistic, monocular dense reconstruction in real time. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp. 2609–2616.
Qin T, Li P and Shen S (2018) Vins-mono: A robust and versatile monocular visual-inertial state estimator. IEEE transactions on robotics 34(4): 1004–1020.
Radl L, Steiner M, Parger M, Weinrauch A, Kerbl B and Steinberger M (2024) Stopthepop: Sorted gaussian splatting for view-consistent real-time rendering. ACM Transactions on Graphics (TOG) 43(4): 1–17.
Rematas K, Liu A, Srinivasan PP, Barron JT, Tagliasacchi A, Funkhouser T and Ferrari V (2022) Urban radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12932–12942.
Rosinol A, Leonard JJ and Carlone L (2023) Nerf-slam: Real-time dense monocular slam with neural radiance fields. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 3437–3444.
Sandström E, Li Y, Van Gool L and Oswald MR (2023) Point-slam: Dense neural point cloud-based slam. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 18433–18444.
Sarikamis FA and Alatan AA (2024) Ig-slam: Instant gaussian slam. arXiv preprint arXiv:2408.01126.
Schöps T, Sattler T and Pollefeys M (2019) Surfelmeshing: Online surfel-based mesh reconstruction. IEEE transactions on pattern analysis and machine intelligence 42(10): 2494–2507.
Shan T, Englot B, Ratti C and Rus D (2021) Lvi-sam: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp. 5692–5698.
Song S, Zhao J, Huang K, Lin J, Ye C and Feng T (2024) N $^{3}$-mapping: Normal guided neural non-projective signed distance fields for large-scale 3d mapping. IEEE Robotics and Automation Letters.
Sucar E, Liu S, Ortiz J and Davison AJ (2021) Imap: Implicit Mapping and Positioning in Real-Time. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6229–6238.
Sun LC, Bhatt NP, Liu JC, Fan Z, Wang Z, Humphreys TE and Topcu U (2024) Mm3dgs slam: Multi-modal 3d gaussian splatting for slam using vision, depth, and inertial measurements. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 10159–10166.
Tao Y, Bhalgat Y, Fu LFT, Mattamala M, Chebrolu N and Fallon M (2024) Silvr: Scalable lidar-visual reconstruction with neural radiance fields for robotic inspection. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 17983–17989.
---
Teed Z and Deng J (2021) Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems 34: 16558–16569.
Teed Z, Lipson L and Deng J (2023) Deep patch visual odometry. Advances in Neural Information Processing Systems 36: 39033–39051.
Tosi F, Zhang Y, Gong Z, Sandström E, Mattoccia S, Oswald MR and Poggi M (2024) How nerfs and 3d gaussian splatting are reshaping slam: a survey. arXiv preprint arXiv:2402.13255.
Wang H, Wang J and Agapito L (2023a) Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13293–13302.
Wang H, Yang M, Zheng X and Hua G (2024a) Scale propagation network for generalizable depth completion. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Wang P, Liu L, Liu Y, Theobalt C, Komura T and Wang W (2021)
Neus: Learning neural implicit surfaces by volume rendering
for multi-view reconstruction. arXiv preprint arXiv:2106.10689
Wang R, Xu S, Dai C, Xiang J, Deng Y, Tong X and Yang J (2024b) Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. arXiv preprint arXiv:2410.19115.
Wei J and Leutenegger S (2024) Gsfusion: Online rgb-d mapping where gaussian splatting meets tsdf fusion. IEEE Robotics and Automation Letters.
Wang Y, Han Q, Habermann M, Daniilidis K, Theobalt C and Liu L (2023b) Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3295–3306.
Whelan T, Leutenegger S, Salas-Moreno RF, Glocker B and Davison AJ (2015) Elasticfusion: Dense slam without a pose graph. In: Robotics: science and systems, volume 11. Rome, p. 3.
Wu C, Duan Y, Zhang X, Sheng Y, Ji J and Zhang Y (2024a) Mmgawssian: 3d gaussian-based multi-modal fusion for localization and reconstruction in unbounded scenes. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 12287–12293.
Wu K, Zhang K, Zhang Z, Tie M, Yuan S, Zhao J, Gan Z and Ding W (2024b) Hgs-mapping: Online dense mapping using hybrid gaussian representation in urban scenes. IEEE Robotics and Automation Letters.
Wu W, Chen C, Yang B, Zou X, Liang F, Xu Y and He X (2025) Dali-slam: Degeneracy-aware lidar-inertial slam with novel distortion correction and accurate multi-constraint pose graph optimization. ISPRS Journal of Photogrammetry and Remote Sensing 221: 92–108.
Xiao R, Liu W, Chen Y and Hu L (2024) Liv-gs: Lidar-vision integration for 3d gaussian splatting slam in outdoor environments. IEEE Robotics and Automation Letters.
Xie Y, Huang Z, Wu J and Ma J (2025) Gs-livm: Real-time photo-realistic lidar-inertial-visual mapping with gaussian splatting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision.
Xu H, Zhang J, Cai J, Rezatofighi H, Yu F, Tao D and Geiger A (2023) Unifying flow, stereo and depth estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(11): 13941–13958.
Xu W, Cai Y, He D, Lin J and Zhang F (2022) Fast-lio2: Fast direct lidar-inertial odometry. IEEE Transactions on Robotics 38(4): 2053–2073.
Yan C, Qu D, Xu D, Zhao B, Wang Z, Wang D and Li X (2024a)
Gs-slam: Dense visual slam with 3d gaussian splatting. In:
Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition. pp. 19595–19604.
Yan Y, Lin H, Zhou C, Wang W, Sun H, Zhan K, Lang X, Zhou X and Peng S (2024b) Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In: European Conference on Computer Vision. Springer, pp. 156–173.
Yan Z, Low WF, Chen Y and Lee GH (2024c) Multi-scale 3d gaussian splatting for anti-aliased rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20923–20931.
Yang J, Ivanovic B, Litany O, Weng X, Kim SW, Li B, Che T, Xu D, Fidler S, Pavone M et al. (2023) Emernerf: Emergent spatial-temporal scene decomposition via self-supervision. arXiv preprint arXiv:2311.02077.
Yang L, Kang B, Huang Z, Zhao Z, Xu X, Feng J and Zhao H (2024) Depth anything v2. Advances in Neural Information Processing Systems 37: 21875–21911.
Yang X, Li H, Zhai H, Ming Y, Liu Y and Zhang G (2022) Voxfusion: Dense tracking and mapping with voxel-based neural implicit representation. In: 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp. 499–507.
Yariv L, Gu J, Kasten Y and Lipman Y (2021) Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems 34: 4805–4815.
Yin J, Li A, Li T, Yu W and Zou D (2021) M2dgr: A multi-sensor and multi-scenario slam dataset for ground robots. IEEE Robotics and Automation Letters 7(2): 2266–2273.
Yu A, Ye V, Tancik M and Kanazawa A (2021) pixelnerf: Neural radiance fields from one or few images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4578–4587.
Yu M, Lu T, Xu L, Jiang L, Xiangli Y and Dai B (2024a) Gsdf: 3dgs meets sdf for improved rendering and reconstruction. arXiv preprint arXiv:2403.16964.
Yu X, Liu Y, Mao S, Zhou S, Xiong R, Liao Y and Wang Y (2023) Nf-atlas: Multi-volume neural feature fields for large scale lidar mapping. IEEE Robotics and Automation Letters 8(9): 5870–5877.
Yu Z, Sattler T and Geiger A (2024b) Gaussian opacity fields: Efficient adaptive surface reconstruction in unbounded scenes. ACM Transactions on Graphics (TOG) 43(6): 1–13.
Yuan Z, Deng J, Ming R, Lang F and Yang X (2024) Sr-livo: Lidar-inertial-visual odometry and mapping with sweep reconstruction. IEEE Robotics and Automation Letters 9(6): 5110–5117.
Yugay V, Li Y, Gevers T and Oswald MR (2023) Gaussian-slam: Photo-realistic dense slam with gaussian splatting. arXiv preprint arXiv:2312.10070.
Zhang B, Fang C, Shrestha R, Liang Y, Long X and Tan P (2024a) Rade-gs: Rasterizing depth in gaussian splatting. arXiv preprint arXiv:2406.01467.
Zhang H, Zou Y, Yan Z and Cheng H (2024b) Rapid-mapping: Lidar-visual implicit neural representations for real-time dense mapping. IEEE Robotics and Automation Letters.
---
Zhang J and Singh S (2018) Laser–visual–inertial odometry and mapping with high robustness and low drift. Journal of field robotics 35(8): 1242–1264.
Zhao C, Sun S, Wang R, Guo Y, Wan JJ, Huang Z, Huang X, Chen YV and Ren L (2024) Tclc-gs: Tightly coupled lidar-camera-gaussian splatting for autonomous driving. arXiv preprint arXiv:2404.02410.
Zhao H, Guan W and Lu P (2025) Lvi-gs: Tightly-coupled lidar-visual-inertial slam using 3d gaussian splatting. IEEE Transactions on Instrumentation and Measurement.
Zheng C, Xu W, Zou Z, Hua T, Yuan C, He D, Zhou B, Liu Z, Lin J, Zhu F et al. (2024) Fast-livo2: Fast, direct lidar-inertial-visual odometry. IEEE Transactions on Robotics.
Zheng C, Zhu Q, Xu W, Liu X, Guo Q and Zhang F (2022) Fast-livo: Fast and tightly-coupled sparse-direct lidar-inertial-visual odometry. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 4003–4009.
Zhong X, Pan Y, Behley J and Stachniss C (2023) Shine-mapping: Large-scale 3d mapping using sparse hierarchical implicit neural representations. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 8371–8377.
Zhou X, Lin Z, Shan X, Wang Y, Sun D and Yang MH (2024a) Drivinggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 21634–21643.
Zhou Y, Li X, Li S, Wang X, Feng S and Tan Y (2024b) Dba-fusion: tightly integrating deep dense visual bundle adjustment with multiple sensors for large-scale localization and mapping. IEEE Robotics and Automation Letters.
Zhu P, Zhuang Y, Chen B, Li L, Wu C and Liu Z (2024a) Mgs-slam: Monocular sparse tracking and gaussian mapping with depth smooth regularization. IEEE Robotics and Automation Letters.
Zhu Z, Peng S, Larsson V, Cui Z, Oswald MR, Geiger A and Pollefeys M (2024b) Nicer-slam: Neural implicit scene encoding for rgb slam. In: 2024 International Conference on 3D Vision (3DV). IEEE, pp. 42–52.
Zhu Z, Peng S, Larsson V, Xu W, Bao H, Cui Z, Oswald MR and Pollefeys M (2022) Nice-slam: Neural implicit scalable encoding for slam. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12786–12796.
Zuo S, Xie Z, Zheng W, Xu S, Li F, Jiang S, Chen L, Yang ZX and Lu J (2025) Dvgt: Driving visual geometry transformer. arXiv preprint arXiv:2512.16919.
Zuo X, Geneva P, Lee W, Liu Y and Huang G (2019) Liefusion: Lidar-inertial-camera odometry. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 5848–5854.
Zuo X, Yang Y, Geneva P, Lv J, Liu Y, Huang G and Pollefeys M (2020) Lic-fusion 2.0: Lidar-inertial-camera odometry with sliding-window plane-feature tracking. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 5112–5119.
Figure 13. SLAM results of Gaussian-LIC2 (c2) in severely degenerated LiDAR scenarios of FAST-LIVO2. The odometry overcomes the degeneration and succeeds in returning to the origin. Note that as our system does not incorporate a detailed point cloud reconstruction module, the colored point cloud map is produced by downsampling and stitching the point clouds, leading to a somewhat coarse visual appearance.
Figure 14. Real-time localization and photo-realistic mapping of Gaussian-LIC2 on the sequence HKisland03 and HKairport03 with a trajectory length of 1.95 km and 2.1 km, respectively.
## A Supplementary Material
### A.1 Comparison with Other Related Methods
Tab. 9 and Tab. 10 also present the comparison results with GS-LIVM Xie et al. (2025), another recently released LiDAR-Inertial-Camera 3DGS-based SLAM method.
### A.2 Additional Results on Runtime
Tab. 10 displays the supplementary results of the runtime with different types of NVIDIA GPUs. Note that except for tile culling, the other CUDA-related acceleration strategies hardly affect the mapping results. Tab. 11 presents the runtime
---
Figure 15. Qualitative rendering results (in-sequence novel view) of Gaussian-LIC2 on the sequence HKisland03 and HKairport03.
Table 9. PSNR results (in-sequence novel view) on our self-collected dataset. All methods use ground-truth poses.
| MM3DGS-SLAM | GS-LIVM | Gaussian-LIC | Gaussian-LIC2 |
| Lecture_Hall | 19.54 | 17.49 | 21.60 | 21.78 |
| Robot_Center | 18.83 | 17.92 | 22.89 | 22.95 |
| Bell_Tower_01 | 18.13 | 21.12 | 25.38 | 25.53 |
| Bell_Tower_02 | 16.49 | 22.44 | 26.42 | 26.85 |
| Bell_Tower_03 | 16.61 | 23.07 | 27.29 | 27.63 |
Table 10. Runtime performance across different methods on the sequence Bell_Tower_01 (span: 150 seconds). All approaches rely on their estimated poses. Here is Gaussian-LIC2 (c1).
| PSNR $\uparrow$ | Depth-L1 (m) $\downarrow$ | Processing Time (s) $\downarrow$ |
| RTX 4090 | RTX 3090 |
| MonoGS | ✗ | ✗ | ✗ | ✗ |
| Co-SLAM | 13.26 | 1.54 | 1030 | 1473 |
| MM3DGS-SLAM | 16.07 | 0.99 | 2460 | 3525 |
| GS-LIVM | 21.05 | 1.19 | 103 | 147 |
| Gaussian-LIC | 25.36 | 0.93 | 102 | 145 |
| Gaussian-LIC2 | 25.51 | 0.27 | 90 | 128 |
| Gaussian-LIC2 w/o Tile Culling | 25.48 | 0.28 | 109 | 156 |
| Gaussian-LIC2 w/o Per-Gaussian | 25.51 | 0.27 | 145 | 207 |
| Gaussian-LIC2 w/o Sparse Adam | 25.51 | 0.27 | 126 | 180 |
| Gaussian-LIC2 w/o Separated SH | 25.51 | 0.27 | 98 | 139 |
| Gaussian-LIC2 w/o Efficient SSIM | 25.51 | 0.27 | 102 | 146 |
| Gaussian-LIC2 w/o WLPM | 25.51 | 0.27 | 105 | 152 |
| Gaussian-LIC2 w/o CPU to GPU | 25.51 | 0.27 | 100 | 144 |
| Gaussian-LIC2 w/o full | 25.48 | 0.28 | 234 | 335 |
breakdown excluding mapping. For each 0.1 s LIC data segment, the tracking front-end consistently performs pose estimation within 0.1 s, achieving real-time performance.
### A.3 More Localization Results in Severely Degenerated Sequences from FAST-LIVO2
Tab. 12 and Fig. 13 show the supplementary results of Gaussian-LIC2 under severe LiDAR degeneration in sequences HIT_Graffiti_Wall, Bright_Screen_Wall, and CBD_Building from FAST-LIVO2 Zheng et al. (2024). Similar
Table 11. Runtime breakdown excluding mapping on the sequence Bell_Tower_01 (span: 150 seconds). Here is Gaussian-LIC2 (c1). Values outside/inside the parentheses correspond to RTX 3090/RTX 4090, respectively. A scan contains LIC (LiDAR-Inertial-Camera) data spanning a duration of 0.1 s.
| Component | Time per Scan (ms) |
| LIC Data Preprocessing | 0.30 |
| Trajectory Extension | 0.01 |
| Trajectory Optimization | 15.95 |
| Visual Local Map Update | 2.07 |
| Optical Flow Tracking | 2.46 |
| LiDAR Local Map Update | 1.66 |
| kNN Search | 5.27 |
| Marginalization | 2.96 |
| Render LiDAR Depth | 1.29 |
| Depth Completion | 10.00 (23.06) |
| Total | 41.97 (55.03) |
Table 12. The start-to-end drift error (translation | rotation) in severely degenerated LiDAR scenarios of FAST-LIVO2.
| CBD_Building_02 | HIT_Graffiti_Wall_01 | Bright_Screen_Wall |
| Gaussian-LIC2-LIO | 35.17m | 1.30 $^{\circ}$ | 8.81m | 2.55 $^{\circ}$ | 3.72m | 1.18 $^{\circ}$ |
| FAST-LIVO2 | 0.01m | 0.77 $^{\circ}$ | 0.03m | 3.92 $^{\circ}$ | 0.02m | 2.71 $^{\circ}$ |
| Gaussian-LIC2 (c1) | 0.03m | 0.88 $^{\circ}$ | 0.03m | 4.26 $^{\circ}$ | 0.04m | 2.82 $^{\circ}$ |
| Gaussian-LIC2 (c2) | 0.01m | 0.75 $^{\circ}$ | 0.03m | 4.01 $^{\circ}$ | 0.02m | 2.54 $^{\circ}$ |
to the main text, Gaussian-LIC2 (c2) slightly outperforms Gaussian-LIC2 (c1) in sequences CBD_Building_02 and Bright_Screen_Wall containing textureless walls.
### A.4 More Mapping Results in Large-Scale UAV Sequences from MARS-LVIG Dataset
We additionally conduct experiments on the public MARS-LVIG Li et al. (2024) dataset, which provides data collected in large-scale natural environments using unmanned aerial vehicles. Fig. 14 presents visualization snapshots during the execution of Gaussian-LIC2, where “rendered_image” and
---
Figure 16. Qualitative rendering results of Gaussian-LIC2 with 0-order SH and 3rd-order SH. Regions with significant differences are highlighted by yellow boxes and red arrows.
“rendered_depth” denote the 3DGS renderings of the current frame. Fig. 15 shows the in-sequence novel view synthesis results on the sequence HKisland03 and HKairport03.
Table 13. Ablation study on Adaptive Density Control (ADC).
| Sequence | Gaussian-LIC2 | Gaussian-LIC2 w/ ADC |
| PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| Retail_Street | 24.37 | 0.775 | 0.121 | 22.59 | 0.754 | 0.132 |
| CBD_Building_01 | 25.20 | 0.851 | 0.103 | 23.19 | 0.830 | 0.116 |
Figure 17. Adaptive density control (ADC) struggles in incremental SLAM systems, often introducing floaters.
### A.5 Ablation Study on Spherical Harmonics
As shown in Fig. 16, Gaussian-LIC2 with 3rd-order SH achieves better visual quality compared to the system that uses 0th-order SH, especially on non-Lambertian surfaces.
### A.6 Ablation Study on Adaptive Density Control
In incremental systems such as SLAM, Gaussians are added in successive batches, which results in different maturities among Gaussians from different batches. ADC tends to split and clone recently added Gaussians that have poor convergence, leading to many floaters and a reduction in visual quality, as depicted in Tab. 13 and Fig. 17.