# GS-LIVM: Real-Time Photo-Realistic LiDAR-Inertial-Visual Mapping with Gaussian Splatting

Yusen Xie $^{1}$ Zhenmin Huang $^{2}$ Jin Wu $^{2}$ Jun Ma $^{1,2}$

 $^{1}$Robotics and Autonomous Systems Thrust

The Hong Kong University of Science and Technology (Guangzhou)

 $^{2}$Department of Electronic and Computer Engineering

The Hong Kong University of Science and Technology

{<mailto:yxie827@connect.hkust-gz.edu.cn>; <mailto:zhuangdf@connect.ust.hk>; <mailto:jwucp@connect.ust.hk>; <mailto:jun.ma@ust.hk>}

<https://github.com/xieyuser/GS-LIVM>

![Image](images/85e421e52a122770ade18405b866e0643eebf233f903492e786566f1af9a5bcf.jpg)

<div style="text-align: center;">Figure 1. Illustration of our photo-realistic mapping results on Botanic Garden Sequence 1018_13. The pink dashed line represents the running trajectory, with the entire trajectory being approximately 200 meters. The green square is the starting point and the red pentagram is the endpoint. The rendering images at different locations are shown in the corners. The quantitative metrics are shown in the table.</div>

## Abstract

In this paper, we introduce GS-LIVM, a real-time photo-realistic LiDAR-Inertial-Visual mapping framework with Gaussian Splatting tailored for outdoor scenes. Compared to existing methods based on Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), our approach enables real-time photo-realistic mapping while ensuring high-quality image rendering in large-scale unbounded outdoor environments. In this work, Gaussian Process Regression (GPR) is employed to mitigate the issues resulting from sparse and unevenly distributed LiDAR observations. The voxel-based 3D Gaussians map representation facilitates real-time dense mapping in large outdoor environments with acceleration governed by custom CUDA kernels. Moreover, the overall framework is designed in a covariance-centered manner, where the estimated covariance is also provided.

ance is used to initialize the scale and rotation of 3D Gaussians, as well as update the parameters of the GPR. We evaluate our algorithm on several outdoor datasets, and the results demonstrate that our method achieves state-of-the-art performance in terms of mapping efficiency and rendering quality. The source code is available on GitHub.

### 1. Introduction

Over the past two decades, simultaneous localization and mapping (SLAM) has emerged as a cornerstone technology underpinning advancements in robotics [^6] and autonomous driving [^4] [^5]. Typically, it relies on sensor fusion techniques that leverage sparse feature representations [^18] [^19] [^42] [^43] [^48] [^52]. While effective in various contexts, these approaches have shown limitations in achieving high-quality performance in 3D reconstruction and photo

---

realistic environmental rendering. The landscape of research in SLAM begins to shift with the advent of powerful neural scene representation techniques, notably Neural Radiance Fields (NeRF) [^22] [^27] and 3D Gaussian Splatting (3DGS) [12–14]. This paradigm shift leads to the development of photo-realistic neural SLAM methods, which greatly improve the fidelity of novel view-synthesis (NVS) and environmental representation.

Despite these advancements, it leaves an open problem in the application of these novel scene representation techniques to large-scale unbounded outdoor scenes. Current methods have demonstrated remarkable performance in indoor settings utilizing RGB-D sensors [^8] [^12] [^21] [^35] [^44] [^49] and monocular cameras [^8] [^21]. However, their applicability to outdoor scenes, where data acquisition is generally performed through multi-line spinning LiDAR and non-repetitive scanning LiDAR, is constrained. These constraints are further compounded by the uncertainty of point cloud scanning trajectories, leading to uneven point cloud distributions. Moreover, in outdoor SLAM scenarios, the movement is predominantly unidirectional, which leads to a growing bias towards the camera direction during the optimization of 3D Gaussians [^13] [^21]. This directional bias results in a significant degradation of rendering quality when observed from new viewpoints. Several studies [^7] [^16] [^46] [^54] have explored the integration of dense neural scene representations with conventional SLAM frameworks for outdoor scenes, yet significant challenges are encountered. The extensive optimization time required for offline mapping can result in over-fitting of 3D Gaussian representations to supervised information, yielding higher image evaluation metrics from these specific viewpoints [^7] [^46] [^54]. These leads to both inefficient processing and compromised image synthesis quality from new perspectives, highlighting the complexities of developing effective and versatile outdoor SLAM systems.

With the aforementioned discussions as a backdrop, we propose GS-LIVM to achieve real-time performance on photo-realistic 3DGS reconstruction for large-scale unbounded outdoor scenes. Our approach initiates with tightly-coupled LiDAR-Inertial-Visual odometry [^18] [^48] for precise state estimation and coordinate transformation of colored point clouds. To address the issues of sparsity and uneven distribution inherent in LiDAR point clouds, we apply Gaussian Process Regression (GPR) [^17] [^28] [^31] to predict an evenly distributed point cloud as part of our photo-realistic mapping process. The covariance output from the GPR module informs the update of GPR parameters and contributes to initializing scales and rotations of 3D Gaussian, which accelerates 3D Gaussian optimization process. We further leverage LiDAR's new observations of the environment to iteratively update the parameters of GPR and the structure of 3D Gaussians. By continuously refining our framework with covariance estimates and image rendering data, we achieve high-quality 3D photo-realistic reconstruction and photo-realistic rendering in outdoor scenes. Figure 1 shows the results of our real-time photo-realistic mapping in Botanic Garden 1018_13 sequence. To the best knowledge of the authors, our development is the first real-time photo-realistic SLAM framework tailored for LiDAR-Inertial-Visual fusion applications in large-scale unbounded outdoor scenes.

In this paper, our main contributions are summarized as follows:

- We employ GPR within voxel-level (Voxel-GPR) to construct a uniform point cloud mesh grid, which effectively addresses sparse and unevenly distributed LiDAR observations. Voxel-GPR facilitates the utilization of fewer 3D Gaussians than the original input without compromising the performance, and this substantially reduces GPU memory consumption and enhances the speed of 3D Gaussians optimization.

- We develop a fast initialization technique for the scale and rotation parameters of the 3D Gaussians, which enables rapid convergence during map expansion. This approach alleviates the decline in rendering quality resulting from inadequate optimization iterations during fast movements of the sensor platform.

- We devise an iterative optimization framework centered on variance, for iteratively reducing noise and hastening the convergence of the Voxel-GPR module. Moreover, leveraging new observations from LiDAR, we introduce a structural similarity regularization term to the existing 3D Gaussian map, thus bolstering the robustness of NVS.

- We conduct thorough validation on various outdoor SLAM datasets using various LiDAR, IMU, and camera configurations. The results showcase the superior capability of our algorithm to achieve real-time photo-realistic mapping in large-scale unbounded outdoor scenes while ensuring high-quality image rendering.

### 2. Related Works

Conventional Multi-Sensor Fusion SLAM. Conventional SLAM systems typically rely on filter-based [^42] [^43] [^52] [^53] or graph-based [^32] [^33] [^50] solvers for ego state estimation, which generally allows for robust, accurate, and fast localization performance. In recent developments, LiDAR-Inertial-Visual SLAM systems have emerged as a versatile framework, which offers high-rate motion compensation, precise geometric structures, and rich texture information [^18] [^19] [^33] [^52] that effectively mitigate issues in challenging environments. Nevertheless, these methods still primarily concentrate on geometric mapping, which limits their capacity for achieving photometric image rendering.

NeRF-based SLAM. NeRF uses a multi-layer perception (MLP) neural network and volume rendering to implic

---

itly learn a 3D scene, with the network storing the structural information of the scene [^22]. However, this approach leads to a significant increase in computational time in large-scale environments. Moreover, the primary drawback of the NeRF-based method is its extremely poor generalization capability across different scenes. Some methods integrate voxel [^47] [^55] or hash-tri-plane [^11] [^29] [^38] representations with MLPs as a way to construct maps, enabling reconstruction over larger areas. NeRF-SLAM [^27] utilizes the dense depth maps estimated from DROID-SLAM [^37] as additional information to supervise the training of InstantNGP [^23]. $H_{2}$-Mapping [^9] and $H_{3}$-Mapping [^10] utilize hierarchical hybrid representations to enhance the reconstruction and achieve impressive performance in indoor scenes via RGB-D camera. NeRF-LOAM [^3] and EINR [^45] gains better performance in large-scale environments, but they mainly focus on geometry structure reconstruction instead of photo-realistic rendering. Despite these aforementioned advancements, implicit representation methods still face challenges regarding real-time performance, particularly in outdoor scenes.

3DGS-based SLAM. To improve computational efficiency, 3DGS utilizes sphere-based explicit scene representation, replacing time-consuming ray-based volume rendering with fast rasterization. This method accelerates NVS and produces promising rendering quality. Aimed at photo-realistic mapping, methods like SplaTAM [^12], Gaussian-SLAM [^49], and GS-SLAM [^44] showcase the substantial benefits of 3DGS over conventional map representations in SLAM tasks. MonoGS [^21] utilizes a single RGB sensor and introduces geometric regularization to address ambiguities in incremental reconstruction, demonstrating promising results with monocular images in small-scale scenes. Additionally, Photo-SLAM [^8] leverages the classical visual odometry method ORB-SLAM3 [^1] for precise state estimation and constructs a hybrid Gaussian map integrated with ORB features. They typically rely on RGB-D/RGB camera sensors to achieve photo-realistic reconstruction in indoor scene. On the other hand, DrivingGaussian [^54], StreetGaussian [^46], and LIV-GaussMap [^7] realize photorealistic reconstructions based on multi-sensor fusion in outdoor scenes. However, these approaches are all offline methods. Gaussian-LIC [^16], developed within a fusion SLAM framework [^15], claims to provide high-quality photo-realistic mapping performance. However, it is reported that each keyframe iteration takes about one second, which is far slower than the 100 milliseconds requiring for real-time mapping. Additionally, it struggles to process entire outdoor sequences due to GPU memory overflow, as it employs millions of 3D Gaussians to reconstruct the complete scene. MM3DGS SLAM [^36], MM-Gaussian [^40], and HGS-Mapping [^41] complete multi-modal sensor fusion reconstruction in urban scenes. However, these methods

![Image](images/cc80029d2047c2f9c67135807d65227dfc87fb35b11aa7aa2b693e06e2ba40e9.jpg)

<div style="text-align: center;">Figure 2. The system processes input from point cloud data obtained from LiDAR, motion data from an IMU, and monocular color image captured by a camera. In the tracking thread, the ESIKF algorithm is employed to track motion, producing odometry output at the IMU frequency. In the mapping thread, the rendered color point cloud is utilized for Voxel-GPR, after which the initialized 3D Gaussian data is integrated into a dense 3D Gaussian map for rendering optimization. The final output is a high-quality, dense 3D Gaussian map. Notations C, D, and S denote the rasterized color image, depth image, and silhouette image, respectively.  $\oplus$ represents the global hash-colored map, which provides neighboring query points for the recently scanned LiDAR points.</div>

ods are similar to offline method [^7], which extend the optimization iteration to enhance performance metrics, yet it remains insufficient for achieving real-time capabilities.

### 3. Methodology

In this work, we rely on the online LiDAR-Inertial-Visual fusion SLAM framework [^18] [^48] for robust state estimation and point coordinate transformation. To address sparsity issue of LiDAR point cloud, we introduce voxel-level GPR (Voxel-GPR) in Section 3.1. In Section 3.2, we introduce a method to estimate the initial scale and rotation parameters of 3D Gaussians based on Voxel-GPR to speed up the convergence. Following this, we present our iterative system in Section 3.3, which encompasses updates to the GPR parameters, map expansion, similarity regularization, and implementation details. The brief overview of our framework is illustrated in Figure 2.

#### 3.1. Voxel-Level Gaussian Process Regression

A series of studies [^7] [^16] [^53] demonstrate that uneven point clouds in the 3DGS framework lead to memory inefficiency and reduced optimization. Offline methods can directly utilize original point clouds since occlusion and insufficient supervision are managed by density control algorithms. However, in dynamic scenarios, rapid perspective changes cause uneven gradient supervision, hindering quick convergence (refer to ablation experiments in Section 4.4). To address this, we introduce Voxel-GPR, which employs GPR to uniformly transform point clouds at the voxel level, enhancing the efficiency of 3D Gaussian map optimization.

For each scanned voxel in continuous frame, we utilize

---

```pseudo
// Algorithm 1: Voxel-GPR for processing single-frame collected point clouds.

Input: $P_f$, C (Global Stored Point Cloud in Voxels)
Output: $P_{f*}$

```

1 $\mathcal{S}_{\mathrm{updated}} \leftarrow \mathrm{ALG.STORE}(P_f, C)$
2 (CUDABatched) for $H_p$ in $\mathcal{S}_{\mathrm{updated}}$ do
3 $\left|\begin{array}{c}\mathcal{P}_{\alpha} = \mathcal{S}_{\mathrm{updated}}[H_p] \rightarrow \text{points} \\ \mathbf{V}_{\alpha} = \mathrm{ALG.PCA}(\mathcal{P}_{\alpha})\end{array}\right.$
4 $\mathrm{switch}\ \mathcal{V}_{\alpha}$ do
5 6 $\left|\begin{array}{c}\text{case X do} \\ \left|\begin{array}{c}\mathbf{f}_{\alpha} \leftarrow \mathcal{P}_{\alpha}^x, \mathbf{x}_{\alpha} \leftarrow \left[\mathcal{P}_{\alpha}^y, \mathcal{P}_{\alpha}^z\right]^\top \\ \mathbf{x}_{\alpha^*} \leftarrow \mathrm{ALG.MESHGRID}(\mathcal{P}_{\alpha^*}^y, \mathcal{P}_{\alpha^*}^z)\end{array}\right.\end{array}\right.$
7 8 9 10 11 12 13 14 15 16 17 end
18 (CUDABatched) $\mathbf{K} \leftarrow \kappa(\mathbf{x}, \mathbf{x}_*)$
19 (CUDABatched) $\mathbf{K}_{**} \leftarrow \kappa(\mathbf{x}, \mathbf{x}_*)$
20 (CUDABatched) $\mathbf{K}_{**} \leftarrow \kappa(\mathbf{x}, \mathbf{x}_*)$
21 (CUDABatched) $\mu_{*}, \Sigma_{*} \leftarrow \mathrm{ALG.SOLVE}(3)$
22 $P_{f*}=(\mathbf{f}_*, \mathbf{x}_*)$

Voxel-GPR for point cloud completion and to generate an evenly sampled point cloud $P_{f*}$ based on the uneven input $P_{f}$. The pseudocode for Voxel-GPR implemented using CUDA is provided in Algorithm 1. We first apply a hash function to $P_{f}$ and simultaneously record the visited voxel hash points as $S_{updated}$. The pseudocode from $P_{f}$ to $S_{updated}$ in line 1 is introduced in the supplementary materials. Then we perform Voxel-GPR processing on each voxel parallelly.

For the $\alpha$th voxel, when the number of points contained is sufficient to meet the point count threshold $\tau$, we conduct Voxel-GPR operations on it. Let n denote the number of points in $\mathcal{P}_\alpha \cdot \mathcal{P}_\alpha \in \mathbb{R}^{n \times 3}$ represents the training subset of point clouds contained within this voxel. Inspired by [^28], we compute the eigenvector of point cloud $\mathcal{P}_\alpha$ by principal component analysis (PCA), and then determine the angles between $\mathbf{V}_\alpha$ and the three axes $\mathbf{X}$, $\mathbf{Y}$, and $\mathbf{Z}$, respectively. Then, the axis with the smallest angle is defined as the value axis and the projection of $\mathcal{P}_\alpha$ onto the value axis is denoted as $\mathbf{f}_\alpha$ ($\mathbf{f}_\alpha \in \mathbb{R}^{n}$), while the other two axes are considered as the parameter axes and the projection of $\mathcal{P}_\alpha$ onto the parameters axis is denoted as $\mathbf{x}_\alpha$ ($\mathbf{x}_\alpha \in \mathbb{R}^{n \times 2}$). Note that $\mathbf{f}_\alpha = \mathbf{y}_\alpha + \varepsilon$, where $\mathbf{y}_\alpha$ is noise-free observations, noise $\varepsilon \sim \mathcal{N}(0, \sigma^2)$ is independently added to each observation, where $\sigma$ is the pre-defined sensor variance (or the updated variance from Section 3.3). $\mathcal{P}_\alpha$ in $\alpha$th voxel is assigned a random variable $\mathbf{f}_\alpha$ and the joint distribution is given by

 $$ p(\mathbf{f}_{\alpha}\mid\mathbf{x}_{\alpha})\sim\mathcal{N}(\mathbf{f}_{\alpha}\mid\mu_{\alpha},\mathbf{K}_{\alpha}) $$

where $\mu_\alpha$ is the mean value in each dimension. $\mathbf{K}_\alpha = \kappa(\mathbf{x}_\alpha, \mathbf{x}_\alpha)$ and $\kappa$ is a positive definite kernel function defined as $\kappa(\mathbf{x}_{\alpha i}, \mathbf{x}_{\alpha j}) = \exp\left(-\lambda(\mathbf{x}_{\alpha i} - \mathbf{x}_{\alpha j})(\mathbf{x}_{\alpha i} - \mathbf{x}_{\alpha j})^\top\right)$, which is a scale value representing the distance between variable $\mathbf{x}_{\alpha i}$ and $\mathbf{x}_{\alpha j}$, and $\lambda$ is a constant ( $\lambda = 1$ in our experiments).

![Image](images/e9839d5288de15e594c6a9dda5114c581e7fa302f9cd3c90b2a42cc1331f8f48.jpg)

<div style="text-align: center;">(a)</div>

![Image](images/90202d07578c637768e50a852f5971deaa2ce0940830abe1cd0ae2249bd21914.jpg)

<div style="text-align: center;">(b)</div>

<div style="text-align: center;">Figure 3. (a) Illustration of Voxel-GPR for the  $\alpha$th voxel involves processing voxels that contain a sufficient number  $\tau$ of points. The depicted surface illustrates the distribution of the predicted points, where the variance decreases from areas marked in red to those in green. (b) Illustration depicts the initialization of 3D Gaussians. As indicated by the curved orange arrow, we consider the points within the neighborhood (black dash region). The green spheres represent the fitted 3D Gaussians.</div>

Each side of $\alpha$th voxel consists of $n_s$ intervals. To generate the point cloud $\mathcal{P}_{\alpha^*} \in \mathbb{R}^{n_s \times 3}$ ($n_s = n_s^2$), we first generate evenly spaced mesh grids $\mathbf{x}_\alpha$ on the plane determined by parameter axes, which are then used as queries to the VoxelGPR module to obtain $\mathbf{f}_{\alpha^*} \in \mathbb{R}^{n_s}$ along value axis. Following the definition of a general GPR problem [^26], the joint distribution of observations $\mathbf{f}_\alpha$ and predictions $\mathbf{f}_{\alpha^*}$ is again a Gaussian distribution which can be partitioned into

 $$ \left(\begin{matrix}{\mathbf{f}_{\alpha}}\\ {\mathbf{f}_{\alpha*}}\\ \end{matrix}\right)\sim\mathcal{N}\left(\mathbf{0},\left(\begin{array}{c c}{\mathbf{K}_{\alpha}+\sigma^{2}\mathbf{I}}&{\mathbf{K}_{\alpha*}}\\ {\mathbf{K}_{\alpha*}^{\top}}&{\mathbf{K}_{\alpha**}}\\ \end{array}\right)\right) $$

where $\mathbf{K}_{\alpha*} = \kappa(\mathbf{x}_{\alpha}, \mathbf{x}_{\alpha*})$ and $\mathbf{K}_{\alpha**} = \kappa(\mathbf{x}_{\alpha*}, \mathbf{x}_{\alpha*})$. With n training data and $n_*$ prediction data, the predicted $\mu_{\alpha^*}$ and $\Sigma_{\alpha^*}$ are

 $$ \begin{align*}p\left(\mathbf{f}_{\alpha*}\mid\mathbf{x}_{\alpha*},\mathbf{x}_{\alpha},\mathbf{f}_{\alpha}\right)&=\mathcal{N}\left(\mathbf{f}_{\alpha*}\mid\boldsymbol{\mu}_{\alpha*},\boldsymbol{\Sigma}_{\alpha*}\right)\\\mathbf{f}_{\alpha*}&=\boldsymbol{\mu}_{\alpha*}=\mathbf{K}_{\alpha*}^{\top}(\mathbf{K}_{\alpha}+\sigma^{2}\mathbf{I})^{-1}\mathbf{f}_{\alpha}\quad( 洛斯尔 \\\boldsymbol{\Sigma}_{\alpha*}&=\mathbf{K}_{\alpha**}-\mathbf{K}_{\alpha*}^{\top}(\mathbf{K}_{\alpha}+\sigma^{2}\mathbf{I})^{-1}\mathbf{K}_{\alpha*}.\end{align*} $$

The prediction points $\mathcal{P}_{\alpha*}$ is given by $(\mathbf{f}_{\alpha*}, \mathbf{x}_{\alpha*})$, where $\mathbf{x}_{\alpha*}$ is the evenly spaced mesh grids. Moreover, $\Sigma_{\alpha*} \in \mathbb{R}^{n_x \times n_y}$ is the estimated variance of $\mathcal{P}_{\alpha*}$. The prediction $\mathcal{P}_{\alpha*}$ and $\Sigma_{\alpha*}$ serve as representatives for $\alpha$th voxel and will be included in the initialization of the 3D Gaussians in Section 3.2. Results of Voxel-GPR is depicted in Figure 3(a).

---

By leveraging CUDA's parallelization capabilities, we efficiently handle hundreds or thousands of voxels simultaneously, even in large-scale map expansions, it takes less than 30 milliseconds. The pseudocode for the CUDA batched algorithm ALG.SOLVE in line 21 of the VoxelGPR algorithm is provided in the supplementary materials.

#### 3.2. Efficient Initialization of 3D Gaussians

In our implementation of map management, each Gaussian $\mathcal{M}_k$ is defined by position $\mathbf{p}_k \in \mathbb{R}^3$, covariance matrix $\boldsymbol{\Phi}_k \in \mathbb{R}^{3 \times 3}$, opacity $\Lambda_k \in \mathbb{R}$, and zero degree Spherical Harmonics (SHs) $Y_k \in \mathbb{R}^3$ per color channel. The scene of our dense 3DGS map $\mathcal{M}$ is

 $$ \mathcal{M}=\left\{\left(\mathcal{M}_{k}:\mathbf{p}_{k},\boldsymbol{\Phi}_{k},\boldsymbol{\Lambda}_{k},\boldsymbol{Y}_{k}\right)\mid k=1,2,\cdots,N\right\} $$

where N is the number of 3D Gaussians in the dense map.

In the original implementation of 3DGS [^13], 3D covariance matrix $\Phi_k \in \mathbb{R}^{3 \times 3}$ of kth Gaussian is initiated as

 $$ \Phi_{k}=R_{k}S_{k}S_{k}^{\top}R_{k}^{\top} $$

where $S_k$ is the scale vector calculated by finding the nearest distance to surrounding neighbors as described in [^30], and $R_k$ is uniformly initialized as a constant and stored as a 4D quaternion. Benefited from Voxel-GPR, we develop an efficient initialization algorithm for calculating the covariance matrix $\Phi_k$ of $\mathcal{M}_k$.

In Section 3.1, the prediction side interval of the $\alpha$th voxel is $n_s$, and the prediction results within this voxel can be divided into $n_s \times n_s$ subgrids. In our actual code implementation, we predict the surrounding $n_r$ neighboring points for each interval additionally. Therefore, the number of predicted points on each side of the $\alpha$th voxel is $n_s \times n_r$. Next, we will discuss the algorithm to initialize 3D Gaussians for each subgrid.

Points in the $\beta$th subgrid are defined as $\mathcal{G}^{\beta} = \{(\mathcal{P}_{f*}^{\alpha})_{i}^{\beta} \in \mathbb{R}^{3}, w_{i}^{\beta} \in \mathbb{R}\}$, where $w_{i}^{\beta}$ is the weight coefficient for the ith point in the $\beta$th grid, given by $w_{i}^{\beta} = 1/\Sigma_{*i}^{\alpha}$, and $\Sigma_{*i}^{\alpha}$ is the covariance for the ith point. We use a sphere to regress $\mathcal{G}^{\beta}$ in this subgrid, as shown in Figure 3(b). Following this, the position $\mathbf{p}^{\beta}$ of the initial 3D Gaussian in this $\beta$th subgrid is given by

 $$ \mathbf{p}^{\beta}=\frac{\sum_{i=1}^{n_{r}^{2}}(\mathcal{P}_{f*}^{\alpha})_{i}^{\beta}\cdot w_{i}^{\beta}}{\sum_{i=1}^{n_{r}^{2}}w_{i}^{\beta}} $$

For scale and rotation parameters of each Gaussian, we fit neighbor by calculating the covariance matrix $\Phi^{\beta}$ in $\beta$th grid as

 $$ \Phi^{\beta}=\frac{\mathbf{Q}^{\top}\cdot d i a g(w_{1}^{\beta},w_{2}^{\beta},\ldots,w_{n_{r}^{2}}^{\beta})\cdot\mathbf{Q}}{\sum_{i=1}^{n_{r}^{2}}w_{i}^{\beta}} $$

where $\mathbf{Q} = (\mathcal{P}_{f*}^{\alpha})^{\beta} - \mathbf{p}^{\beta}$. The calculation of Q described above involves subtracting $\mathbf{p}^{\beta}$ from each row of $(\mathcal{P}_{f*}^{\alpha})^{\beta}$, resulting in Q having a dimension of $n_r^2 \times 3$. The scale parameter $S^{\beta}$ in $\beta$th subgrid is given by

 $$ S^{\beta}={d i a g}(\Phi^{\beta}) $$

and $R^{\beta}$ of this 3D Gaussian is set to identity quaternion. $S^{\beta}$ and $R^{\beta}$ are used to estimate initial shape of 3D Gaussians. For color information, we reproject $\mathbf{p}^{\beta}$ to current image by camera extrinsic, grab the color in this RGB image, and calculate the initial SHs Y.

By employing the aforementioned methods, we compute the parameters of $n_s \times n_s$ Gaussians for the $\alpha$th voxel. These 3D Gaussians serve as representatives of $\alpha$th voxel in space. Experimental results demonstrate that this approach significantly accelerates the optimization efficiency of 3D Gaussians while ensuring high-quality rendering.

#### 3.3. Iterative photo-realistic Mapping Framework

Map Expansion and Covariance Update. For the voxels in the scene, we categorize them into four types: (a) un-explored voxels or those that do not meet the processing threshold $\tau$ (Figure 4(a)); (b) voxels that meet the processing threshold but have not yet been added to the map (Figure 4(b)); (c) voxels that are already included in the map but are still in an active state, meaning that their variance has not yet reached the convergence threshold $\eta$ and they requires further Voxel-GPR optimization (Figure 4(c)); and (d) voxels that have completed convergence in Voxel-GPR module (Figure 4(d)).

![Image](images/2d0ac6481b1b692bde91e83218fd41290efd6c69d489876492265b5fd9120cb7.jpg)

<div style="text-align: center;">(a)</div>

![Image](images/4a4fc4065b9fa02decccb2ef88f861645d42be778ba9a3cae0300f5515c14b60.jpg)

<div style="text-align: center;">(b)</div>

![Image](images/29c0611b8a7389036aca4bb280c395fa9c48b11c1294cc4a867e2b7c633c88ee.jpg)

<div style="text-align: center;">(c)</div>

![Image](images/b43409ee51cb9d183a96d9c70b1a8b1240303c78f6bc8f4e753253ed55764404.jpg)

<div style="text-align: center;">(d)</div>

<div style="text-align: center;">Figure 4. This illustration presents four types of voxels in hash voxel map. Small dots marked in black represent new scanned points cloud from LiDAR, with their variance corresponding to the sensor's inherent noise. Larger dots highlighted in dark blue are from the meshgrid, poised for processing via Voxel-GPR, while the small dots in light blue have undergone variance updates. The ellipsoids are shaded in various colors, where each color signifies the magnitude of their variance.</div>

To increase the processing speed, we do not process types (a) and (d) by the Voxel-GPR module. Our main focus is on processing the type (b) voxels, after which we expand these updated voxels to the dense map $\mathcal{M}$. The expansion is determined by whether the number of updated voxels exceeds a certain threshold, which can be found in the details of our code. For type (c) voxels, we update the original variance with the estimated value from the previous iteration. We then conduct the Voxel-GPR calculation again by

---

stacking new observation points with the original points until the Voxel-GPR process converges for this voxel, at which point it transits to type (d).

Rendering and Delta Depth Similarity Loss. Similar to [^13], we rasterize 3D Gaussians $\mathcal{M}$ to the observation camera with pose T and camera intrinsic $\pi$. The color of one pixel is rendered by $\mathcal{N}$ ordered 3D Gaussians in depth, photometric image C is rendered as

 $$ C=\sum_{i\in\mathcal{N}}c_{i}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j}) $$

where $c_i$ represents color obtained by learned SHs coefficients and $\alpha_i$ is the density computed by multiplying 2D covariance $\sigma'$ with opacity $\Lambda_i$. We also rasterize per-pixel depth D and silhouette image S to determine that if a pixel contains information from the current map by $\alpha$-blending rendering similar to [^12].

In the delta depth similarity loss, we rasterize two depth image $(D_{\mathcal{F}_c}, D_{\mathcal{F}_{c+1}})$ and two silhouette image $(S_{\mathcal{F}_c}, S_{\mathcal{F}_{c+1}})$ to calculate delta depth similarity loss $\mathcal{L}_d$ between relative camera $\mathcal{F}_c = \{D_{\mathcal{F}_c}, S_{\mathcal{F}_c}, \pi_{\mathcal{F}_c}, \mathbf{T}_{\mathcal{F}_c}\}$ and $\mathcal{F}_{c+1} = \{D_{\mathcal{F}_{c+1}}, S_{\mathcal{F}_{c+1}}, \pi_{\mathcal{F}_{c+1}}, \mathbf{T}_{\mathcal{F}_{c+1}}\}$. $\mathcal{L}_d$ supervises the optimization of the 3DGS dense map through gradient back propagation. $\mathcal{L}_d$ is defined as

 $$ \mathcal{L}_{d}=\|\boldsymbol{S}_{\mathcal{F}_{c+1}}\circ\boldsymbol{D}_{\mathcal{F}_{c+1}}-\boldsymbol{\pi}_{\mathcal{F}_{c+1}}\mathbf{T}_{\mathcal{F}_{c+1}}\mathbf{T}_{\mathcal{F}_{c}}^{-1}\boldsymbol{\pi}_{\mathcal{F}_{c}}^{-1}(\boldsymbol{S}_{\mathcal{F}_{c}}\circ\boldsymbol{D}_{\mathcal{F}_{c}})\|_{1} $$

where $\circ$ denotes the operation applied element-wisely to the rendered depth image.

Structure Similarity Loss. For the scanned point cloud in the latest frame $P_f$, we introduce the similarity loss $\mathcal{L}_p$ to accelerate the optimization process. This loss measures the Euclidean distance between the current 3D Gaussian map $\mathcal{M} = \{\mathcal{M}_k : \mathbf{p}_k \in \mathbb{R}^3, S_k \in \mathbb{R}^3\} \mid k = 1,2,\cdots,m\}$ and the latest scanned point cloud frame $P_f$, m and n are the number of spheres and points, respectively. For each scanned point within the frame, we identify the nearest 3D Gaussian and calculate the Euclidean distance. Structure similarity loss $\mathcal{L}_p$ is defined as the mean of the nearest distances calculate by all points in $P_f$. $\mathcal{L}_p$ can be expressed as

 $$ \mathcal{L}_{p}=\frac{1}{n}\sum_{j=0}^{n}\min_{k\in\{1,2,\ldots,m\}}\left(\|(P_{f})_{j}-\mathbf{p}_{k}\|_{2}-\bar{S}_{k}\right) $$

where $\bar{S}_{k}$ is the mean value of kth 3D Gaussian scaling parameters.

Training. The mapping thread in our system maintains a dense map composed of 3D Gaussians $\mathcal{M}$ and a continuous camera queue. We separate this camera queue into two parts: current visited camera window $Q_{curr}$ and historical camera queue $Q_{hist}$. $Q_{curr}$ consists of $\mathcal{T}$ latest added cameras. We select $k_{curr}$ frames $\mathcal{F}_{curr}$ for map optimization during each iteration. To prevent catastrophic forgetting of historical data, each iteration also randomly selects $k_{hist}$ historical frame $\mathcal{F}_{hist}$ from $Q_{hist}$ for joint optimization. Rasterized image loss, delta depth similarity loss, and structure similarity loss will be iterated to optimize the 3D Gaussian dense map by minimizing

 $$ \mathcal{L}=(1-\lambda_{s})\|C-C_{g t}\|_{1}+\lambda_{s}\mathcal{L}_{s s i m} + \lambda_{d}\sum_{\mathcal{F}_{*}\in\{\mathcal{F}_{c u r r},\mathcal{F}_{h i s t}\}}\mathcal{L}_{d}(\mathcal{F}_{*},\mathcal{F}_{*+1})+\lambda_{p}\mathcal{L}_{p} $$

where $C_{gt}$ and C are the observed image and the image rendered by (9), respectively. $\mathcal{L}_{ssim}$ is an SSIM [^39] term. $\mathcal{L}_{d}$ is the delta depth similarity loss calculated by $k_{curr}$ current frames $\mathcal{F}_{curr}$ and $k_{hist}$ historical frame $\mathcal{F}_{hist}$ in (10). $\mathcal{L}_{p}$ is a structure similarity loss calculated in (11). $\lambda_{s}$, $\lambda_{d}$, and $\lambda_{p}$ are weights of SSIM loss, structure similarity loss, and delta depth similarity loss, respectively.

### 4. Experiments

In this section, we first introduce the experimental setup in Section 4.1, including datasets, baseline, and parameter settings, etc. Then in Section 4.2, we mainly compare the render performance of dense maps. Section 4.3 provides the analysis of mapping time and GPU memory consumption of the framework. Section 4.4 shows ablation experiments of the proposed framework.

#### 4.1. Experiment Setup

Datasets. We carry out experiments on four public LiDAR Inertial-Visual datasets, including R³LIVE [^18] dataset, FAST-LIVO [^52] dataset, NTU-VIRAL [^24] dataset, and Botanic Garden [^20] dataset. The first two datasets are collected within the campuses of HKU and HKUST using a handheld device equipped with a Livox Avia LiDAR at 10 Hz and its built-in IMU at 200 Hz, and a 15 Hz RGB camera (image resolution: 640×512). The NTU-VIRAL [^24] dataset is collected by Ouster-16 multi-line spinning LiDAR, which is more sparse than Livox LiDAR and the image resolution is 752×480. The Botanic Garden [^20] dataset is collected by a wheeled robot traversing through a luxuriant botanic garden, point clouds from both multi-line spinning LiDAR Velodyne VLP-16 and Livox Avia LiDAR are collected and image resolution is 480×300.

Baselines and Metrics. We compare our method with the existing state-of-the-art NeRF-based dense visual SLAM NeRF-SLAM [^27] and 3DGS-based SLAM [^21]. Additionally, the original offline implementation 3DGS [^13] is also used as a baseline for comparison. It should be noted that RGB-D methods [^8] [^12] [^35] [^44] [^49] are not suitable in outdoor scenes due to the poor performance, so these methods are not included in comparison. We evaluate the rendering performance using PSNR, SSIM [^39], and LPIPS [^51]. Note that we run all the baselines on datasets and calculate the mean metric among all observation images.

---

<div style="text-align: center;">Table 1. Quantitative rendering performance comparison between radiance-field-based SLAM method [^27], 3DGS-based SLAM method [^21], and original offline optimization [^13] on R $^{3}$LIVE dataset [^18], FAST-LIVO dataset [^52], NTU dataset [^24], and Botanic Garden dataset [^20]. The results ranked from best to worst are highlighted as first, second, and third.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td rowspan="2">Sequence</td><td style='text-align: center; word-wrap: break-word;'>LiDAR</td><td colspan="3">Nerf-SLAM [^27]</td><td colspan="3">MonoGS [^21]</td><td colspan="3">3DGS [^13]</td><td colspan="3">Ours</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Type</td><td style='text-align: center; word-wrap: break-word;'>PSNR $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>PSNR $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>PSNR $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>PSNR $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $\downarrow$</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hku-campus_seq_00</td><td style='text-align: center; word-wrap: break-word;'>Livox Avia</td><td style='text-align: center; word-wrap: break-word;'>13.232</td><td style='text-align: center; word-wrap: break-word;'>0.410</td><td style='text-align: center; word-wrap: break-word;'>0.653</td><td style='text-align: center; word-wrap: break-word;'>12.142</td><td style='text-align: center; word-wrap: break-word;'>0.368</td><td style='text-align: center; word-wrap: break-word;'>0.608</td><td style='text-align: center; word-wrap: break-word;'>21.744</td><td style='text-align: center; word-wrap: break-word;'>0.719</td><td style='text-align: center; word-wrap: break-word;'>0.302</td><td style='text-align: center; word-wrap: break-word;'>22.430</td><td style='text-align: center; word-wrap: break-word;'>0.719</td><td style='text-align: center; word-wrap: break-word;'>0.247</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Visual_Challenge</td><td style='text-align: center; word-wrap: break-word;'>Livox Avia</td><td style='text-align: center; word-wrap: break-word;'>12.981</td><td style='text-align: center; word-wrap: break-word;'>0.408</td><td style='text-align: center; word-wrap: break-word;'>0.592</td><td style='text-align: center; word-wrap: break-word;'>13.478</td><td style='text-align: center; word-wrap: break-word;'>0.579</td><td style='text-align: center; word-wrap: break-word;'>0.414</td><td style='text-align: center; word-wrap: break-word;'>18.552</td><td style='text-align: center; word-wrap: break-word;'>0.545</td><td style='text-align: center; word-wrap: break-word;'>0.378</td><td style='text-align: center; word-wrap: break-word;'>21.806</td><td style='text-align: center; word-wrap: break-word;'>0.717</td><td style='text-align: center; word-wrap: break-word;'>0.289</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>eee_02</td><td style='text-align: center; word-wrap: break-word;'>Ouster-16</td><td style='text-align: center; word-wrap: break-word;'>8.316</td><td style='text-align: center; word-wrap: break-word;'>0.320</td><td style='text-align: center; word-wrap: break-word;'>0.693</td><td style='text-align: center; word-wrap: break-word;'>11.632</td><td style='text-align: center; word-wrap: break-word;'>0.407</td><td style='text-align: center; word-wrap: break-word;'>0.514</td><td style='text-align: center; word-wrap: break-word;'>20.388</td><td style='text-align: center; word-wrap: break-word;'>0.686</td><td style='text-align: center; word-wrap: break-word;'>0.382</td><td style='text-align: center; word-wrap: break-word;'>20.718</td><td style='text-align: center; word-wrap: break-word;'>0.700</td><td style='text-align: center; word-wrap: break-word;'>0.319</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>1005_00</td><td style='text-align: center; word-wrap: break-word;'>Livox Avia</td><td style='text-align: center; word-wrap: break-word;'>9.790</td><td style='text-align: center; word-wrap: break-word;'>0.362</td><td style='text-align: center; word-wrap: break-word;'>0.750</td><td style='text-align: center; word-wrap: break-word;'>14.563</td><td style='text-align: center; word-wrap: break-word;'>0.602</td><td style='text-align: center; word-wrap: break-word;'>0.406</td><td style='text-align: center; word-wrap: break-word;'>22.167</td><td style='text-align: center; word-wrap: break-word;'>0.657</td><td style='text-align: center; word-wrap: break-word;'>0.380</td><td style='text-align: center; word-wrap: break-word;'>21.123</td><td style='text-align: center; word-wrap: break-word;'>0.679</td><td style='text-align: center; word-wrap: break-word;'>0.258</td></tr></table>

Implementation Details. Our framework is implemented by CUDA/C++ using the LibTorch framework [^2] under Robot Operating System (ROS) [^25], incorporating CUDA code for Gaussian Splatting and trained on a desktop PC with a 5.50 GHz Intel Core i9-13900HX CPU, 64 GB RAM, and a NVIDIA RTX 4060 Laptop 8 GB GPU. In all sequences, the hyperparameters used in our experiments are listed in the supplementary materials.

![Image](images/83836c5d34bfe9e97745e5a0b2a978ab7cde79f16a69ae107980bcadbcba0fb0.jpg)

<div style="text-align: center;">Figure 5. Illustration of the rendering performance on the NeRF-based method [^27] and the 3DGS-based method [^13] [^21] on sequence hkust_campus_seq_00, Visual_Challenge, eee_03, 1005_00, respectively.</div>

#### 4.2. Rendering Evaluation

We conduct rendering comparisons using the 3DGS original implementation [^13], NeRF-SLAM [^27], and Gaussian Splatting SLAM [^21]. All of these methods rely solely on images as supervision. Due to limitations in GPU memory and poor tracking performance, these methods [^13] [^21] [^27] are unable to complete the reconstruction task of the entire environment. To facilitate comparison, we reduce the size of the reconstruction scene and use camera poses and depth maps calculated by COLMAP [^30] as input for methods [^13] [^21] [^27]. The camera pose in our framework is estimated by multi-sensor fusion odometry. The reconstruction sequence in the scene is limited to within 100 frames, and the reconstruction time is limited to within 5 minutes. The comparison of rendering performance are shown in Table 1 and Figure 5. It can be observed that due to the strict time constraints, the reconstruction results of NeRF-SLAM [^27] and MonoGS [^21] are relatively poor. Although 3DGS performs well in some scenes, its offline optimization performs poorly on NVS. More details of rendering experiments can be found in the supplementary materials.

![Image](images/1ddde22ca47772f4af3226ba22355daa07d4cd0a2410678fd5e4fb14f3b6bb99.jpg)

<div style="text-align: center;">Figure 6. Time consumption and GPU memory consumption on Botanic Garden sequence 1018_13.</div>

<div style="text-align: center;">Table 2. Duration (DT), mapping Time (MT), count of 3D Gaussians, and maximum memory cost (Mem) in photo-realistic mapping in different sequences.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Sequence</td><td style='text-align: center; word-wrap: break-word;'>DT (s)</td><td style='text-align: center; word-wrap: break-word;'>MT (s)</td><td style='text-align: center; word-wrap: break-word;'>Count</td><td style='text-align: center; word-wrap: break-word;'>Mem (Mb)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hku_campus_seq_00</td><td style='text-align: center; word-wrap: break-word;'>202</td><td style='text-align: center; word-wrap: break-word;'>202</td><td style='text-align: center; word-wrap: break-word;'>1,209,666</td><td style='text-align: center; word-wrap: break-word;'>2495</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Visual_Challenge</td><td style='text-align: center; word-wrap: break-word;'>162</td><td style='text-align: center; word-wrap: break-word;'>162</td><td style='text-align: center; word-wrap: break-word;'>353,334</td><td style='text-align: center; word-wrap: break-word;'>1353</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>eee_02</td><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>758,213</td><td style='text-align: center; word-wrap: break-word;'>1523</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>1005_00</td><td style='text-align: center; word-wrap: break-word;'>611</td><td style='text-align: center; word-wrap: break-word;'>612</td><td style='text-align: center; word-wrap: break-word;'>2,177,464</td><td style='text-align: center; word-wrap: break-word;'>3492</td></tr></table>

#### 4.3. Runtime and Memory Cost Analysis

The real-time performance criterion in our experiments is that all collected image frames are processed, meaning that the processing time for each frame t should satisfy $t \leq D/C$, where $\mathcal{D}$ is the duration of collected dataset and C is the size of added tracking camera queue. The fill-between plot in mapping time consumption over time in the Botanic Garden sequence 1018_13 (duration: 208s) is shown in Figure 6. It can be observed that as time progresses, the overall system consumption time remains mostly below the real-time limit. The time consumption graph on longer sequences can be found in the supplementary materials.

---

As the mapping process advances, the consumption of GPU memory escalates with an increasing number of 3D Gaussians. All sequences can be fully processed through photo-realistic mapping on an 8 GB GPU when $n_s = 3$. Table 2 outlines experiment results on various sequences.

#### 4.4. Ablation Study

We conduct an ablation study on the Voxel-GPR module and the 3D Gaussians initialization algorithm module in our system, as well as the structure similarity loss and the delta depth similarity loss, as shown in Figure 7. It can be seen that the construction of 3D Gaussians based on the original point cloud (Figure 7(a)) does not consider the geometric dependencies in environment, which can lead to uneven distributions of 3D Gaussians. Moreover, achieving the desired optimization effect with Figure 7(b) and Figure 7(c) takes 1.3 times longer, indicating that a reasonable initialization method can accelerate the convergence. By adding the structure similarity loss in Figure 7(d) and the delta depth similarity loss in Figure 7(e) based on Figure 7(c), the representation of details in the environment is enhanced, resulting in images that are closer to real collected images.

Quantitative ablation experiments are shown in Table 3, mainly comparing the differences in mapping time and mean rendering metrics on all observation images of different hyper parameters. The first and second rows of each sequence mainly compare the influence of parameter $\eta$ on mapping time and rendering metrics. Setting an appropriate value for $\eta$ can significantly impact mapping time for different LiDAR configurations. A smaller $\eta$ value results in longer mapping times, but does not provide a clear improvement in render metrics. The 2nd to 4th rows compare the effects of structure similarity loss and delta depth loss on render metrics. It can be observed that adding these two losses does not impact mapping time but can enhance render metrics. The 4th and 5th rows in Table 3 and Figure 7(e)-Figure 7(f) compare the impact of $n_s$ on the results. In smaller scenes (e.g., 1018_13 in Botanic Garden [^20], hkust_campus_seq_00 in R³LIVE [^18]), $n_s$ does not affect mapping time and significantly improves render metrics, but in larger scenes (e.g., hku_main_building in R³LIVE [^18]), $n_s$ may consume excessive time and GPU memory. This illustrates the trade-off between accuracy and speed. Expanded ablation experiments on more sequences can be referred to the supplementary materials.

### 5. Conclusion

In conclusion, this paper presents GS-LIVM, a novel real-time photo-realistic LiDAR-Inertial-Visual mapping framework with Gaussian Splatting. Our approach enables real-time photo-realistic mapping while ensuring high-quality image rendering in large, uncontrolled outdoor scenes. In this work, we leverage Voxle-GPR to

![Image](images/4d2801953cd9a7d2db1c418d02464f3693cd37cecad90cc536e0a018a089e5ec.jpg)

<div style="text-align: center;">(a) w/o Voxel-GPR</div>

![Image](images/2d0b00d04623eac79c08a21bf33b9c40ded3f27d139acc4d9d7ea25401f4812d.jpg)

<div style="text-align: center;">(b) w/o our initialization</div>

![Image](images/6f7d41e23f93fc7a24a01dbf69e3a7925bf912f680b1e7756c33009f937c4f8b.jpg)

![Image](images/43d9524547315c849d15d261fcb4fe9e47b8ae063edc1fc4781de66456acac25.jpg)

<div style="text-align: center;">(d) w/o DDS</div>

<div style="text-align: center;">(c) w/o SS</div>

![Image](images/d78bb93393404a933dff69dc67856703272a34723906ab84ba93ef3b2de273ca.jpg)

<div style="text-align: center;">(e) Full  $(n_{s}=4)$</div>

![Image](images/569697ee5e26f6322dbc0ac6ebfe27eb3c95ad09ee48ddf5eaba81e217d04cb9.jpg)

<div style="text-align: center;">(f) Full  $(n_{s} = 3)$</div>

<div style="text-align: center;">Figure 7. (a) Similar to [^7] [^16], each collected point corresponds to a generated 3D Gaussian. (b) The use of a uniform initialization method. (c) There is no structural similarity loss (SS) for the single frame point cloud. (d) There is no delta depth similarity loss (DDS) for the relative frame. (e) The combined results of all the modules and losses are presented. Note that,  $n_s = 4$ in (a-e). (f) The combined results of all the modules ( $n_s = 3$).</div>

<div style="text-align: center;">Table 3. Quantitative ablation comparison of hyperparameters in various sequences, such as duration (DT), mapping time (MT), interval of 3D Gaussians in voxel side $n_{s}$, convergence variance of Voxel-GPR $\eta$, structure similarity loss (SS), and delta depth similarity loss (DDS).</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Sequence</td><td style='text-align: center; word-wrap: break-word;'>DT (s)</td><td style='text-align: center; word-wrap: break-word;'>MT (s)</td><td style='text-align: center; word-wrap: break-word;'>$n_{{s}}$</td><td style='text-align: center; word-wrap: break-word;'>$\eta$</td><td style='text-align: center; word-wrap: break-word;'>SS</td><td style='text-align: center; word-wrap: break-word;'>DDS</td><td style='text-align: center; word-wrap: break-word;'>PSNR</td><td style='text-align: center; word-wrap: break-word;'>SSIM</td></tr><tr><td rowspan="5">hku_campus_seq_00</td><td rowspan="5">202</td><td style='text-align: center; word-wrap: break-word;'>1240</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.1</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>18.494</td><td style='text-align: center; word-wrap: break-word;'>0.637</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>202</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>19.005</td><td style='text-align: center; word-wrap: break-word;'>0.647</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>202</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>19.944</td><td style='text-align: center; word-wrap: break-word;'>0.648</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>202</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>19.292</td><td style='text-align: center; word-wrap: break-word;'>0.640</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>202</td><td style='text-align: center; word-wrap: break-word;'>4</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>20.143</td><td style='text-align: center; word-wrap: break-word;'>0.672</td></tr><tr><td rowspan="5">1018_13</td><td rowspan="5">208</td><td style='text-align: center; word-wrap: break-word;'>891</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.2</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>14.754</td><td style='text-align: center; word-wrap: break-word;'>0.449</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>208</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>16.039</td><td style='text-align: center; word-wrap: break-word;'>0.475</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>208</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>15.988</td><td style='text-align: center; word-wrap: break-word;'>0.488</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>208</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>15.732</td><td style='text-align: center; word-wrap: break-word;'>0.461</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>208</td><td style='text-align: center; word-wrap: break-word;'>4</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>17.982</td><td style='text-align: center; word-wrap: break-word;'>0.493</td></tr></table>

tackle the sparsity issue in LiDAR point clouds. The voxel-based 3D Gaussians map representation facilitates real-time photo-realistic mapping in large-scale unbounded outdoor scenes with the acceleration provided by custom CUDA kernels. Additionally, the framework is structured in a covariance-centered manner, where the estimated covariance is utilized to initialize the scale and rotation of 3D Gaussian, as well as update the parameters of the Voxle-GPR. Experimental evaluations on various outdoor datasets validate that our algorithm achieves state-of-the-art performance in terms of mapping efficiency and rendering quality. Moreover, the source code is publicly available, facilitating further research and development in this area.

---

## References

[^1]: Carlos Campos, Richard Elvira, Juan J Gómez Rodríguez, José MM Montiel, and Juan D Tardós. ORB-SLAM3: An Accurate Open-source Library for Visual, Visual-Inertial, and Multimap SLAM. IEEE Trans. Robot., 37(6):1874–1890, 2021. 3

[^2]: PyTorch Contributors. Installing C++ Distributions of PyTorch, 2024. 7

[^3]: Junyuan Deng, Xieyuanli Chen, Songpengcheng Xia, Zhen Sun, Guoqing Liu, Wenxian Yu, and Ling Pei. NeRFLOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping. In Int. Conf. Comput. Vis., pages 8184–8193, 2023. 3

[^4]: Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In IEEE Conf. Comput. Vis. Pattern Recog., 2012. 1

[^5]: Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision Meets Robotics: The KITTI Dataset. Int. J. Rob. Res., 32(11):1231–1237, 2013. 1

[^6]: Wolfgang Hess, Damon Kohler, Holger Rapp, and Daniel Andor. Real-time Loop Closure in 2D LiDAR SLAM. In IEEE Int. Conf. Robot. Autom., pages 1271–1278, 2016. 1

[^7]: Sheng Hong, Junjie He, Xinhu Zheng, Chunran Zheng, and Shaojie Shen. LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering. IEEE Robot. Autom. Lett., 2024. 2, 3, 8

[^8]: Huajian Huang, Longwei Li, Hui Cheng, and Sai-Kit Yeung. Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular Stereo and RGB-D Cameras. In IEEE Conf. Comput. Vis. Pattern Recog., pages 21584–21593, 2024. 2, 3, 6

[^9]: Chenxing Jiang, Hanwen Zhang, Peize Liu, Zehuan Yu, Hui Cheng, Boyu Zhou, and Shaojie Shen. $H_{2}$-Mapping: Real-time Dense Mapping Using Hierarchical Hybrid Representation. IEEE Robot. Autom. Lett., 2023. 3

[^10]: Chenxing Jiang, Yiming Luo, Boyu Zhou, and Shaojie Shen. $H_{3}$-Mapping: Quasi-Heterogeneous Feature Grids for Real-time Dense Mapping Using Hierarchical Hybrid Representation. arXiv preprint arXiv:2403.10821, 2024. 3

[^11]: Mohammad Mahdi Johari, Camilla Carta, and François Fleuret. ESLAM: Efficient Dense SLAM System based on Hybrid Representation of Signed Distance Fields. In IEEE Conf. Comput. Vis. Pattern Recog., pages 17408–17419, 2023. 3

[^12]: Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. SplaTAM: Splat Track & Map 3D Gaussians for Dense RGB-D SLAM. In IEEE Conf. Comput. Vis. Pattern Recog., pages 21357–21366, 2024. 2, 3, 6

[^13]: Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph., 42(4):139–1, 2023. 2, 5, 6, 7, 1

[^14]: Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. A Hierarchical 3D Gaussian Representation for Real-time

Rendering of Very Large Datasets. ACM Trans. Graph., 43 (4):1–15, 2024. 2

[^15]: Xiaolei Lang, Chao Chen, Kai Tang, Yukai Ma, Jiajun Lv, Yong Liu, and Xingxing Zuo. Coco-LIC: Continuous-time Tightly-coupled LiDAR-Inertial-Camera Odometry using Non-uniform B-spline. IEEE Robot. Autom. Lett., 2023. 3

[^16]: Xiaolei Lang, Laijian Li, Hang Zhang, Feng Xiong, Mu Xu, Yong Liu, Xingxing Zuo, and Jiajun Lv. Gaussian-LIC: Photo-realistic LiDAR-Inertial-Camera SLAM with 3D Gaussian Splatting. arXiv preprint arXiv:2404.06926, 2024. 2, 3, 8

[^17]: Bo Li, Yingqiang Wang, Yu Zhang, Wenjie Zhao, Jianyuan Ruan, and Ping Li. GP-SLAM: Laser-based SLAM Approach based on Regionalized Gaussian Process Map Reconstruction. Auton. Rob., 44(6):947–967, 2020. 2

[^18]: Jiarong Lin and Fu Zhang. R $^{3}$LIVE: A Robust, Real-Time, RGB-colored, LiDAR-Inertial-Visual Tightly-coupled State Estimation and Mapping Package. In IEEE Int. Conf. Robot. Autom., pages 10672–10678, 2022. 1, 2, 3, 6, 7, 8, 4

[^19]: Jiarong Lin, Chunran Zheng, Wei Xu, and Fu Zhang. R $^{2}$LIVE: A Robust, Real-Time, LiDAR-Inertial-Visual Tightly-coupled State Estimator and Mapping. IEEE Robot. Autom. Lett., 6(4):7469–7476, 2021. 1,2

[^20]: Yuanzhi Liu, Yujia Fu, Minghui Qin, Yufeng Xu, Baoxin Xu, Fengdong Chen, Bart Goossens, Hongwei Yu, Chun Liu, Long Chen, et al. Botanic Garden: A High-quality and Large-scale Robot Navigation Dataset in Challenging Natural Environments. arXiv preprint arXiv:2306.14137, 2023. 6, 7, 8, 2, 4

[^21]: Hidenobu Matsuki, Riku Murai, Paul HJ Kelly, and Andrew J Davison. Gaussian Splatting SLAM. In IEEE Conf. Comput. Vis. Pattern Recog., pages 18039–18048, 2024. 2, 3, 6, 7

[^22]: Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In Eur. Conf. Comput. Vis., 2020. 2, 3

[^23]: Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph., 41(4):1–15, 2022. 3

[^24]: Thien-Minh Nguyen, Shenghai Yuan, Muqing Cao, Yang Lyu, Thien H Nguyen, and Lihua Xie. NTU VRAL: A Visual-Inertial-Ranging-LiDAR Dataset, from an Aerial Vehicle Viewpoint. Int. J. Rob. Res., 41(3):270–280, 2022. 6, 7

[^25]: Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, Andrew Y Ng, et al. ROS: An Open-source Robot Operating System. In IEEE Int. Conf. Robot. Autom. Worksh., page 5. Kobe, Japan, 2009. 7

[^26]: Carl Edward Rasmussen. Gaussian Processes in Machine Learning. In Summer School on Machine Learning, pages 63–71. Springer, 2003. 4

[^27]: Antoni Rosinol, John J Leonard, and Luca Carlone. NeRF-SLAM: Real-time Dense Monocular SLAM with nNeural Radiance Fields. In IEEE Int. Conf. Intell. Rob. Syst., pages 3437–3444, 2023. 2, 3, 6, 7

---

[^28]: Jianyuan Ruan, Bo Li, Yinqiang Wang, and Zhou Fang. GP-SLAM+: Real-time 3D LiDAR SLAM based on Improved Regionalized Gaussian Process Map Reconstruction. In IEEE Int. Conf. Intell. Rob. Syst., pages 5171–5178, 2020. 2, 4

[^29]: Erik Sandström, Yue Li, Luc Van Gool, and Martin R Oswald. Point-SLAM: Dense Neural Point Cloud-based SLAM. In Int. Conf. Comput. Vis., pages 18433–18444, 2023. 3

[^30]: Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-Motion Revisited. In IEEE Conf. Comput. Vis. Pattern Recog., 2016. 5, 7

[^31]: Matthias Seeger. Gaussian Processes for Machine Learning. Int. J. Neural. Syst., 14(02):69–106, 2004. 2

[^32]: Tixiao Shan, Brendan Englot, Drew Meyers, Wei Wang, Carlo Ratti, and Daniela Rus. LIO-SAM: Tightly-coupled LiDAR Inertial Odometry via Smoothing and Mapping. In IEEE Int. Conf. Intell. Rob. Syst., pages 5135–5142, 2020. 2

[^33]: Tixiao Shan, Brendan Englot, Carlo Ratti, and Daniela Rus. LVI-SAM: Tightly-coupled LiDAR-Visual-Inertial Odometry via Smoothing and Mapping. In IEEE Int. Conf. Robot. Autom., pages 5692–5698, 2021. 2, 4

[^34]: Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A Benchmark for the Evaluation of RGB-D SLAM Systems. In IEEE Int. Conf. Intell. Rob. Syst., pages 573–580, 2012. 2

[^35]: Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew J Davison. iMAP: Implicit Mapping and Positioning in Real-time. In Int. Conf. Comput. Vis., pages 6229–6238, 2021. 2, 6

[^36]: Lisong C Sun, Neel P Bhatt, Jonathan C Liu, Zhiwen Fan, Zhangyang Wang, Todd E Humphreys, and Ufuk Topcu. MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements. arXiv preprint arXiv:2404.00923, 2024. 3

[^37]: Zachary Teed and Jia Deng. DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras. Adv. Neural Inform. Process. Syst., 34:16558–16569, 2021. 3

[^38]: Hengyi Wang, Jingwen Wang, and Lourdes Agapito. Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-time SLAM. In IEEE Conf. Comput. Vis. Pattern Recog., pages 13293–13302, 2023. 3

[^39]: Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image Quality Assessment: from Error Visibility to Structural Similarity. IEEE Trans. Image Process., 13(4): 600–612, 2004. 6

[^40]: Chenyang Wu, Yifan Duan, Xinran Zhang, Yu Sheng, Jianmin Ji, and Yanyong Zhang. MM-Gaussian: 3D Gaussian-based Multi-modal Fusion for Localization and Reconstruction in Unbounded Scenes. arXiv preprint arXiv:2404.04026, 2024. 3

[^41]: Ke Wu, Kaizhao Zhang, Zhiwei Zhang, Shanshuai Yuan, Muer Tie, Julong Wei, Zijun Xu, Jieru Zhao, Zhongxue Gan, and Wenchao Ding. HGS-Mapping: Online Dense Mapping Using Hybrid Gaussian Representation in Urban Scenes. arXiv preprint arXiv:2403.20159, 2024. 3

[^42]: Wei Xu and Fu Zhang. Fast-LIO: A Fast, Robust LiDAR-Inertial Odometry Package by Tightly-coupled Iterated

Kalman Filter. IEEE Robot. Autom. Lett., 6(2):3317–3324, 2021. 1, 2

[^43]: Wei Xu, Yixi Cai, Dongjiao He, Jiarong Lin, and Fu Zhang. Fast-LIO2: Fast Direct LiDAR-Inertial Odometry. IEEE Trans. Robot., 38(4):2053–2073, 2022. 1, 2, 4

[^44]: Chi Yan, Delin Qu, Dan Xu, Bin Zhao, Zhigang Wang, Dong Wang, and Xuelong Li. GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting. In IEEE Conf. Comput. Vis. Pattern Recog., pages 19595–19604, 2024. 2, 3, 6

[^45]: Dongyu Yan, Xiaoyang Lyu, Jieqi Shi, and Yi Lin. Efficient Implicit Neural Reconstruction using LiDAR. In IEEE Int. Conf. Robot. Autom., pages 8407–8414. IEEE, 2023. 3

[^46]: Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street Gaussians for Modeling Dynamic Urban Scenes. arXiv preprint arXiv:2401.01339, 2024. 2, 3

[^47]: Xingrui Yang, Hai Li, Hongjia Zhai, Yuhang Ming, Yuqian Liu, and Guofeng Zhang. Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation. In IEEE Int. Symp. on Mix. and Augm. Real., pages 499–507, 2022. 3

[^48]: Zikang Yuan, Jie Deng, Ruiye Ming, Fengtian Lang, and Xin Yang. SR-LIVO: LiDAR-Inertial-Visual Odometry and Mapping With Sweep Reconstruction. IEEE Robot. Autom. Lett., 2024. 1, 2, 3

[^49]: Vladimir Yugay, Yue Li, Theo Gevers, and Martin R Oswald. Gaussian-SLAM: Photo-Realistic Dense SLAM with Gaussian Splatting. arXiv preprint arXiv:2312.10070, 2023. 2, 3, 6

[^50]: Ji Zhang and Sanjiv Singh. Low-drift and Real-time LiDAR Odometry and Mapping. Autonomous Robots, 41(2):401–416, 2017. 2

[^51]: Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE Conf. Comput. Vis. Pattern Recog., pages 586–595, 2018. 6

[^52]: Chunran Zheng, Qingyan Zhu, Wei Xu, Xiyuan Liu, Qizhi Guo, and Fu Zhang. Fast-LIVO: Fast and Tightly-coupled Sparse-direct LiDAR-Inertial-Visual Odometry. In IEEE Int. Conf. Intell. Rob. Syst., pages 4003–4009, 2022. 1, 2, 6, 7

[^53]: Chunran Zheng, Wei Xu, Zuhao Zou, Tong Hua, Chongjian Yuan, Dongjiao He, Bingyang Zhou, Zheng Liu, Jiarong Lin, Fangcheng Zhu, et al. Fast-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometry. arXiv preprint arXiv:2408.14035, 2024. 2, 3

[^54]: Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Driving Gaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes. In IEEE Conf. Comput. Vis. Pattern Recog., pages 21634–21643, 2024. 2, 3

[^55]: Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R Oswald, and Marc Pollefeys. NICE-SLAM: Neural Implicit Scalable Encoding for SLAM. In IEEE Conf. Comput. Vis. Pattern Recog., pages 12786–12796, 2022. 3

---

# GS-LIVM: Real-Time Photo-Realistic LiDAR-Inertial-Visual Mapping with Gaussian Splatting

Supplementary Material

6. Pseudocode for Preprocessing Algorithm ALG.STORE & CUDA Batched Algorithm ALG.SOLVE in Voxel-GPR

Preprocessing algorithm for latest scanned point cloud $P_{f}$ is shown in Algorithm 2. Each point in $P_{f}$ is stored into a voxel structure, and the visited voxel information in this frame is recoeded simultaneously. Utilizing the cuBLAS scientific computing library within the GPU, the pseudocode for batch-solving (3) for all voxels is shown as Algorithm 3.

```pseudo
// Algorithm 2: Preprocessing for single-frame collected point clouds

Input:  $P_f$,  $\mathcal{C}$ (Global Stored Point Cloud in Voxels)

Output:  $\mathcal{S}_{\text{updated}}$ (Store Visited Voxels in This Frame)

```

1 $\mathcal{S}_{\text{updated}} \leftarrow [\cdot]$

2 for p in $P_f$ do

3 $\quad H_p = \text{ALG.HASH}(p^x, p^y, p^z)$

4 $\quad \text{push } p \text{ to } \mathcal{C}[H_p] \rightarrow \text{points}$

5 $\quad \text{push } H_p \text{ to } \mathcal{S}_{\text{updated}}$

6 end

```pseudo
// Algorithm 3: Batch solving $\mu_*$ and $\Sigma_-$ in Voxel-GPR based on cuBLAS

Input: f, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K, K,

```

### 7. Supplementary Experiments

#### 7.1. Comparison with Fully Optimized Offline 3DGS

In the rendering comparison presented in Figure 5, to ensure a fair comparison, we limit the training of the original 3DGS implementation to 3000 iterations. Interestingly, we discover that with sufficient optimization time, 3DGS indeed achieves better metrics than our method (Table 4). This is because 3DGS involves offline optimization over an extended period to overfit the training data. Nonetheless, limitations are encountered with NVS, as demonstrated in Figure 8. Overfitting to the training dataset causes 3D Gaussians to extend beyond the observation point, which is the primary issue with offline methods. It can be observed that our method can overcome the drawbacks of overfitting.

<div style="text-align: center;">Table 4. Comparison with the fully optimized offline 3DGS [^13].</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td rowspan="2">Sequence</td><td rowspan="2">PSNR $\uparrow$</td><td colspan="2">3DGS</td><td rowspan="2">PSNR $\uparrow$</td><td colspan="2">Ours</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>SSIM $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $\downarrow$</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hku_seq_00</td><td style='text-align: center; word-wrap: break-word;'>27.827</td><td style='text-align: center; word-wrap: break-word;'>0.861</td><td style='text-align: center; word-wrap: break-word;'>0.204</td><td style='text-align: center; word-wrap: break-word;'>22.430</td><td style='text-align: center; word-wrap: break-word;'>0.719</td><td style='text-align: center; word-wrap: break-word;'>0.247</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>1005_00</td><td style='text-align: center; word-wrap: break-word;'>26.196</td><td style='text-align: center; word-wrap: break-word;'>0.812</td><td style='text-align: center; word-wrap: break-word;'>0.233</td><td style='text-align: center; word-wrap: break-word;'>21.123</td><td style='text-align: center; word-wrap: break-word;'>0.679</td><td style='text-align: center; word-wrap: break-word;'>0.258</td></tr></table>

![Image](images/5e5c59516b6a2de12e4e3b7aceabdff25f172bf63a8697adecd6b6a71c7f7c9b.jpg)

![Image](images/d79c31e9f8f6645589834e4632c43395b859766872eeb138fcf9e3f2842af61b.jpg)

<div style="text-align: center;">(c) 3DGS [^13]</div>

![Image](images/f32d13587b8bfb649000493f24862e0f2e52e2890c009d3c41b055a78b79ba48.jpg)

![Image](images/5bbff11a9551460dd004374059314ead116d05f06ca98d584f9c15f5b34a2621.jpg)

<div style="text-align: center;">(d) Ours</div>

<div style="text-align: center;">Figure 8. In all subfigures, the top images represent a supervised perspective with image information, while the bottom images depict an unsupervised perspective with synthesized new viewpoints. Results in (a)(c) are obtained using the offline 3DGS method, while results in (b)(d) are obtained using our method.</div>

---

<div style="text-align: center;">Table 5. Quantitative rendering performance comparison on more datasets. The results ranked from best to worst are highlighted as first, second, and third.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td rowspan="2">Sequence</td><td style='text-align: center; word-wrap: break-word;'>LiDAR</td><td colspan="3">Nerf-SLAM [^27]</td><td colspan="3">MonoGS [^21]</td><td colspan="3">3DGS [^13]</td><td colspan="3">Ours</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Type</td><td style='text-align: center; word-wrap: break-word;'>PSNR $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $^{\downarrow}$</td><td style='text-align: center; word-wrap: break-word;'>PSNR $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $^{\downarrow}$</td><td style='text-align: center; word-wrap: break-word;'>PSNR $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $^{\downarrow}$</td><td style='text-align: center; word-wrap: break-word;'>PSNR $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>SSIM $^{\uparrow}$</td><td style='text-align: center; word-wrap: break-word;'>LPIPS $^{\downarrow}$</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hkust_campus_02</td><td style='text-align: center; word-wrap: break-word;'>Livox Avia</td><td style='text-align: center; word-wrap: break-word;'>10.728</td><td style='text-align: center; word-wrap: break-word;'>0.389</td><td style='text-align: center; word-wrap: break-word;'>0.716</td><td style='text-align: center; word-wrap: break-word;'>12.408</td><td style='text-align: center; word-wrap: break-word;'>0.426</td><td style='text-align: center; word-wrap: break-word;'>0.582</td><td style='text-align: center; word-wrap: break-word;'>17.629</td><td style='text-align: center; word-wrap: break-word;'>0.692</td><td style='text-align: center; word-wrap: break-word;'>0.412</td><td style='text-align: center; word-wrap: break-word;'>15.745</td><td style='text-align: center; word-wrap: break-word;'>0.634</td><td style='text-align: center; word-wrap: break-word;'>0.382</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hkul</td><td style='text-align: center; word-wrap: break-word;'>Livox Avia</td><td style='text-align: center; word-wrap: break-word;'>9.232</td><td style='text-align: center; word-wrap: break-word;'>0.384</td><td style='text-align: center; word-wrap: break-word;'>0.716</td><td style='text-align: center; word-wrap: break-word;'>8.928</td><td style='text-align: center; word-wrap: break-word;'>0.286</td><td style='text-align: center; word-wrap: break-word;'>0.750</td><td style='text-align: center; word-wrap: break-word;'>19.548</td><td style='text-align: center; word-wrap: break-word;'>0.708</td><td style='text-align: center; word-wrap: break-word;'>0.328</td><td style='text-align: center; word-wrap: break-word;'>14.990</td><td style='text-align: center; word-wrap: break-word;'>0.600</td><td style='text-align: center; word-wrap: break-word;'>0.424</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>1018.00</td><td style='text-align: center; word-wrap: break-word;'>Velodyne VLP-16</td><td style='text-align: center; word-wrap: break-word;'>12.186</td><td style='text-align: center; word-wrap: break-word;'>0.387</td><td style='text-align: center; word-wrap: break-word;'>0.699</td><td style='text-align: center; word-wrap: break-word;'>12.721</td><td style='text-align: center; word-wrap: break-word;'>0.634</td><td style='text-align: center; word-wrap: break-word;'>0.417</td><td style='text-align: center; word-wrap: break-word;'>23.452</td><td style='text-align: center; word-wrap: break-word;'>0.720</td><td style='text-align: center; word-wrap: break-word;'>0.253</td><td style='text-align: center; word-wrap: break-word;'>16.971</td><td style='text-align: center; word-wrap: break-word;'>0.688</td><td style='text-align: center; word-wrap: break-word;'>0.338</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>private-360</td><td style='text-align: center; word-wrap: break-word;'>Livox MID-360</td><td style='text-align: center; word-wrap: break-word;'>13.681</td><td style='text-align: center; word-wrap: break-word;'>0.407</td><td style='text-align: center; word-wrap: break-word;'>0.598</td><td style='text-align: center; word-wrap: break-word;'>8.912</td><td style='text-align: center; word-wrap: break-word;'>0.263</td><td style='text-align: center; word-wrap: break-word;'>0.778</td><td style='text-align: center; word-wrap: break-word;'>22.428</td><td style='text-align: center; word-wrap: break-word;'>0.693</td><td style='text-align: center; word-wrap: break-word;'>0.302</td><td style='text-align: center; word-wrap: break-word;'>17.777</td><td style='text-align: center; word-wrap: break-word;'>0.618</td><td style='text-align: center; word-wrap: break-word;'>0.439</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>private-pandar</td><td style='text-align: center; word-wrap: break-word;'>Pandar XT-32</td><td style='text-align: center; word-wrap: break-word;'>9.492</td><td style='text-align: center; word-wrap: break-word;'>0.328</td><td style='text-align: center; word-wrap: break-word;'>0.610</td><td style='text-align: center; word-wrap: break-word;'>7.414</td><td style='text-align: center; word-wrap: break-word;'>0.289</td><td style='text-align: center; word-wrap: break-word;'>0.623</td><td style='text-align: center; word-wrap: break-word;'>18.647</td><td style='text-align: center; word-wrap: break-word;'>0.658</td><td style='text-align: center; word-wrap: break-word;'>0.369</td><td style='text-align: center; word-wrap: break-word;'>16.234</td><td style='text-align: center; word-wrap: break-word;'>0.600</td><td style='text-align: center; word-wrap: break-word;'>0.330</td></tr></table>

#### 7.2. More Rendering and Mapping Results

We collect a Livox MID-360 (private-360) and a HESAI Pandar XT-32 (private-pandar) sequence with image resolution $640 \times 512$ to verify the performance of our algorithm in different LiDAR types additionally. The expanded comparison of rendering performance are shown in Table 5.

It is worth noting that our method can reconstruct the entire scene with limited GPU resources (8GB), which is a feature unattainable by other methods [^7] [^16]. In Figure 9, we present snapshots of rendered images in corresponding whole sequences. It can be observed that our method maintains excellent rendering quality even in large outdoor scenes under the real-time mapping requirement.

Since our 3D Gaussians information is initialized from point clouds, the rendering performance for sparse multiline spinning LiDAR is inferior to that of the Livox LiDAR due to the difference in point cloud density. This difference can be observed in the Botanic Garden sequence in Figure 9.

The reconstructed 3D Gaussian models of sequences hku_main_building and 1005_01 are also presented in Figure 10. Sequence hku_main_building has a duration of 20 minutes and a trajectory length of approximately 900 meters, yet we are still able to achieve real-time map expansion. Outdoor park (sequence 1005_01) typically contains rich structural and textural features, making photo-realistic mapping highly challenging. Using Velodyne VLP-16 LiDAR in the outdoor park, our proposed algorithm is able to achieve a good dense reconstruction model.

#### 7.3. More Ablation Experiments

Expanded ablation experiments on additional sequences are presented in Table 6. In challenging environments, such as sensor degradation (Visual_Challenge) or very sparse point clouds (eee_02), the experiments continue to show that our module design is robust, enhancing photorealistic mapping effects in these scenarios.

#### 7.4. Tracking Performance

Our photo-realistic mapping method utilizes a general SLAM framework to obtain precise camera poses. In theory, this approach can be integrated into any general LIVO framework. In our experiments, we adopt the framework from [^48]. Compared to the original [^48], we introduce the following improvements: 1) We replace the time sweep

<div style="text-align: center;">Table 6. Quantitative ablation comparison of hyperparameters in various sequences, such as duration (DT), mapping time (MT), interval of 3D Gaussians in voxel side $n_{s}$, convergence variance of Voxel-GPR $\eta$, structure similarity loss (SS), and delta depth similarity loss (DDS).</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Sequence</td><td style='text-align: center; word-wrap: break-word;'>DT (s)</td><td style='text-align: center; word-wrap: break-word;'>MT (s)</td><td style='text-align: center; word-wrap: break-word;'>$n_s$</td><td style='text-align: center; word-wrap: break-word;'>$\eta$</td><td style='text-align: center; word-wrap: break-word;'>SS</td><td style='text-align: center; word-wrap: break-word;'>DDS</td><td style='text-align: center; word-wrap: break-word;'>PSNR</td><td style='text-align: center; word-wrap: break-word;'>SSIM</td></tr><tr><td rowspan="5">Visual_Challenge</td><td rowspan="5">162</td><td style='text-align: center; word-wrap: break-word;'>166</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.1</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>18.479</td><td style='text-align: center; word-wrap: break-word;'>0.608</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>162</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>17.326</td><td style='text-align: center; word-wrap: break-word;'>0.617</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>162</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>17.599</td><td style='text-align: center; word-wrap: break-word;'>0.626</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>162</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>18.127</td><td style='text-align: center; word-wrap: break-word;'>0.613</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>162</td><td style='text-align: center; word-wrap: break-word;'>4</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>18.422</td><td style='text-align: center; word-wrap: break-word;'>0.659</td></tr><tr><td rowspan="5">eee_02</td><td rowspan="5">321</td><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.2</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>12.766</td><td style='text-align: center; word-wrap: break-word;'>0.436</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>12.136</td><td style='text-align: center; word-wrap: break-word;'>0.430</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✗</td><td style='text-align: center; word-wrap: break-word;'>12.395</td><td style='text-align: center; word-wrap: break-word;'>0.434</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>12.512</td><td style='text-align: center; word-wrap: break-word;'>0.441</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>321</td><td style='text-align: center; word-wrap: break-word;'>4</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>14.328</td><td style='text-align: center; word-wrap: break-word;'>0.473</td></tr></table>

based refinement method with the sensor's original times-tamps at a finer granularity, resolving the crash issue when using multi-line spinning LiDAR. 2) We apply CUDA encapsulation for certain large matrix multiplications to accelerate optimization. To ensure the fairness of comparisons, we conduct tracking experiments on the improved version of [^48].

We compare improved method with filter-based method R $^{3}$LIVE [^18], FAST-LIO2 [^43], and graph optimization based method LVI-SAM [^33]. In tracking performance evaluation, we focus on key performance metrics Relative Pose Error (RPE) and Absolute Trajectory Error (ATE) [^34], considering full transformations that encompass both rotation and translation.

To assess the adaptability of our dataset for tracking, we meticulously select 7 representative sequences from the Botanic dataset [^20] and conduct comprehensive evaluations on cutting-edge algorithms, including those presented in [^18] [^33] [^43], by comparing them against ground truth data. The data used for our tracking algorithm comes from a Velodyne VLP-16 multi-line spinning LiDAR. Table 7 presents the evaluation metrics on 7 sequences from the Botanic Garden [^20] dataset. Our method, lacking a loop closure detection module, exhibits lower accuracy on some sequences compared to LVI-SAM [^33]. Overall, comparisons with above state-of-the-art methods reveal that our approach demonstrates clear advantages on certain sequences. Figure 11 illustrates the trajectory comparison for the sequence 1006_01 within the Botanic Garden [^20] dataset, featuring a trajectory length of approximately 730 meters.

---

![Image](images/bc9bca39aaf58e7b15ed2846748f087646663100902ba2d7226aa5bb534c4e2a.jpg)

<div style="text-align: center;">Figure 9. Illustration of the rendering performance of the entire sequence from a dataset. It is noteworthy that these illustrations are all rendered based on the real-time mapping results. These scenes are hkust_campus_seq_00 (Livox AVIA), Visual_Challenge (Livox AVIA), eee_03 (Ouster 16), 1005_00 (Livox AVIA), hku_park_00 (Livox AVIA), hku_main_building (Livox AVIA), 1006_01 (Livox AVIA), 1006_01 (Velodyne VLP-16), 1005_01 (Velodyne VLP-16), and eee_03 (Ouster 16), respectively.</div>

---

![Image](images/959d050d81dad5d233a21e87b7a9496a0e306e8206fd2703d1db454695d54b44.jpg)

<div style="text-align: center;">Figure 10. More illustrations of the 3D Gaussians model of the entire sequence. These two scenes are hku_main_building ((Livox AVIA)) and 1005_01 (Velodyne VLP-16), respectively.</div>

<div style="text-align: center;">Table 7. Quantitative tracking performance comparison of LiDAR-IMU SLAM method [^43] and LiDAR-Visual-IMU fusion SLAM method [^18] [^33] on BotanicGarden dataset [^20].</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'></td><td colspan="2">1005.00</td><td colspan="2">1005.01</td><td colspan="2">1005.07</td><td colspan="2">1006.01</td><td colspan="2">1008.03</td><td colspan="2">1018.00</td><td colspan="2">1018.13</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>RPE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>ATE $\downarrow$</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>R3LIVE [^18]</td><td style='text-align: center; word-wrap: break-word;'>1.165</td><td style='text-align: center; word-wrap: break-word;'>3.153</td><td style='text-align: center; word-wrap: break-word;'>1.151</td><td style='text-align: center; word-wrap: break-word;'>1.451</td><td style='text-align: center; word-wrap: break-word;'>2.112</td><td style='text-align: center; word-wrap: break-word;'>3.893</td><td style='text-align: center; word-wrap: break-word;'>0.934</td><td style='text-align: center; word-wrap: break-word;'>3.505</td><td style='text-align: center; word-wrap: break-word;'>2.050</td><td style='text-align: center; word-wrap: break-word;'>3.383</td><td style='text-align: center; word-wrap: break-word;'>0.165</td><td style='text-align: center; word-wrap: break-word;'>0.378</td><td style='text-align: center; word-wrap: break-word;'>0.133</td><td style='text-align: center; word-wrap: break-word;'>0.366</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>FAST-LIO2 [^43]</td><td style='text-align: center; word-wrap: break-word;'>1.048</td><td style='text-align: center; word-wrap: break-word;'>2.665</td><td style='text-align: center; word-wrap: break-word;'>0.652</td><td style='text-align: center; word-wrap: break-word;'>0.483</td><td style='text-align: center; word-wrap: break-word;'>0.947</td><td style='text-align: center; word-wrap: break-word;'>0.751</td><td style='text-align: center; word-wrap: break-word;'>1.047</td><td style='text-align: center; word-wrap: break-word;'>1.521</td><td style='text-align: center; word-wrap: break-word;'>0.852</td><td style='text-align: center; word-wrap: break-word;'>0.798</td><td style='text-align: center; word-wrap: break-word;'>0.241</td><td style='text-align: center; word-wrap: break-word;'>0.187</td><td style='text-align: center; word-wrap: break-word;'>0.245</td><td style='text-align: center; word-wrap: break-word;'>0.308</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>LVI-SAM [^33]</td><td style='text-align: center; word-wrap: break-word;'>0.347</td><td style='text-align: center; word-wrap: break-word;'>0.312</td><td style='text-align: center; word-wrap: break-word;'>0.147</td><td style='text-align: center; word-wrap: break-word;'>0.129</td><td style='text-align: center; word-wrap: break-word;'>0.127</td><td style='text-align: center; word-wrap: break-word;'>0.257</td><td style='text-align: center; word-wrap: break-word;'>0.462</td><td style='text-align: center; word-wrap: break-word;'>0.410</td><td style='text-align: center; word-wrap: break-word;'>0.272</td><td style='text-align: center; word-wrap: break-word;'>0.252</td><td style='text-align: center; word-wrap: break-word;'>0.139</td><td style='text-align: center; word-wrap: break-word;'>0.044</td><td style='text-align: center; word-wrap: break-word;'>0.152</td><td style='text-align: center; word-wrap: break-word;'>0.051</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>OURS</td><td style='text-align: center; word-wrap: break-word;'>0.174</td><td style='text-align: center; word-wrap: break-word;'>0.291</td><td style='text-align: center; word-wrap: break-word;'>0.058</td><td style='text-align: center; word-wrap: break-word;'>0.073</td><td style='text-align: center; word-wrap: break-word;'>0.061</td><td style='text-align: center; word-wrap: break-word;'>0.496</td><td style='text-align: center; word-wrap: break-word;'>0.202</td><td style='text-align: center; word-wrap: break-word;'>0.702</td><td style='text-align: center; word-wrap: break-word;'>0.068</td><td style='text-align: center; word-wrap: break-word;'>0.414</td><td style='text-align: center; word-wrap: break-word;'>0.055</td><td style='text-align: center; word-wrap: break-word;'>0.075</td><td style='text-align: center; word-wrap: break-word;'>0.050</td><td style='text-align: center; word-wrap: break-word;'>0.077</td></tr></table>

![Image](images/1990c63d473edd0716620ce68fd971d9369681715b99e8225ef40273dabb43c0.jpg)

<div style="text-align: center;">Figure 11. The trajectory comparison for the Botanic Garden sequence 1006_01 is illustrated, featuring a detailed illustration of a three-way intersection within the scene in the upper left, with the offset along the Z plane for this junction depicted below. It is evident from this comparison that our trajectory aligns most closely with the actual trajectory, demonstrating a high level of tracking precision.</div>

In the top right corner of Figure 11, an enlarged view of a three-way junction on the X-Y plane is presented, while the bottom right corner displays an elevation map of the same junction on the Z plane. From the visual representation, it is evident that our trajectory aligns most closely with the ground truth, showcasing the highest qualitative tracking precision. More comparison results are available on Figure 12.

<div style="text-align: center;">Table 8. Duration (DT), mapping time (MT), count of 3D Gaussians, maximum memory cost (Mem), and rendering metrics on ultra long sequence.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Sequence</td><td style='text-align: center; word-wrap: break-word;'>DT (s)</td><td style='text-align: center; word-wrap: break-word;'>MT (s)</td><td style='text-align: center; word-wrap: break-word;'>Count</td><td style='text-align: center; word-wrap: break-word;'>Mem (Mb)</td><td style='text-align: center; word-wrap: break-word;'>PSNR</td><td style='text-align: center; word-wrap: break-word;'>SSIM</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hku_main_building</td><td style='text-align: center; word-wrap: break-word;'>1170</td><td style='text-align: center; word-wrap: break-word;'>1170</td><td style='text-align: center; word-wrap: break-word;'>2,674,234</td><td style='text-align: center; word-wrap: break-word;'>4489</td><td style='text-align: center; word-wrap: break-word;'>15.234</td><td style='text-align: center; word-wrap: break-word;'>0.538</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>hkust_campus_00</td><td style='text-align: center; word-wrap: break-word;'>1073</td><td style='text-align: center; word-wrap: break-word;'>1090</td><td style='text-align: center; word-wrap: break-word;'>3,302,675</td><td style='text-align: center; word-wrap: break-word;'>5812</td><td style='text-align: center; word-wrap: break-word;'>15.124</td><td style='text-align: center; word-wrap: break-word;'>0.496</td></tr></table>

#### 7.5. Time Consumption and Memory Usage on Ultra-Long Time Sequence

Addressing SLAM and photo-realistic mapping in ultralong time sequence is a challenging issue. As the scale of the map increases, insufficient available storage space leads to a slowdown or interruption in the mapping process. To deal with these problems, we conduct tests on two ultralong sequences in test dataset. With the limited GPU resources, we set $n_{r}=2$ and obtain the mapping time, render

---

![Image](images/3386c5e87c782c73003e899bdf7b758e30d56a9a588c1844dc114c83eea0b4af.jpg)

![Image](images/c6d6a2ceb4d725d38acf337edb1e186709a661b5ce93d9de8acb9acae5934cea.jpg)

![Image](images/f96725a8360590b5967cc640c95dd5b27bf20ec8edb4152764562ed444f6b2e8.jpg)

![Image](images/aca2c403c1b31737e2f517fab647c7673b729bb54f57738dc059b2423ddbf654.jpg)

![Image](images/0b819c425dd4bf94d4f974d469822aa66aeaf3ff1ff082392f2fc66ee1f01c8b.jpg)

![Image](images/32374daa907a6d0d55f4c5d1aca529b8ee499368f2a6cc24ad840a0b230b9dc2.jpg)

![Image](images/a46b258790a63a324e7477740a20f27fb6a113698902010d90a5af16098810d2.jpg)

![Image](images/df66b39d194afdcf2c62defe2304f9f78492e4ee347a5fbe8d1536f8cd56d9d4.jpg)

<div style="text-align: center;">Figure 12. More trajectory comparisons. The first row of images shows the trajectory of the entire scene, while the second row of images displays the enlarged results within the corresponding blue boxes. The four scenes are 1005_00, 1005_07, 1008_03, 1008_13, respectively.</div>

![Image](images/96dd215d93bebfbd57104a119f224913afd4e678fe0d3152219f949271f50db3.jpg)

<div style="text-align: center;">Figure 13. Time consumption and GPU memory consumption on ultra long scequence hku_main_building (Duration: 1170 s).</div>

<div style="text-align: center;">Table 9. Hyper parameters used in all datasets.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Name</td><td style='text-align: center; word-wrap: break-word;'>Value</td><td style='text-align: center; word-wrap: break-word;'>Description</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>size</td><td style='text-align: center; word-wrap: break-word;'>0.2</td><td style='text-align: center; word-wrap: break-word;'>Side length of each voxel.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$\tau$</td><td style='text-align: center; word-wrap: break-word;'>10</td><td style='text-align: center; word-wrap: break-word;'>Point count threshold for voxel processing.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$n_s$</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>Interval between each side within the voxel.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$n_r$</td><td style='text-align: center; word-wrap: break-word;'>3</td><td style='text-align: center; word-wrap: break-word;'>Number of neighboring Gaussians per side.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$\eta$</td><td style='text-align: center; word-wrap: break-word;'>0.3</td><td style='text-align: center; word-wrap: break-word;'>Convergence threshold for Voxel-GPR.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$\lambda_{SSIM}$</td><td style='text-align: center; word-wrap: break-word;'>0.2</td><td style='text-align: center; word-wrap: break-word;'>Weight for the photometric SSIM loss.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$\lambda_p$</td><td style='text-align: center; word-wrap: break-word;'>0.1</td><td style='text-align: center; word-wrap: break-word;'>Weight for the structural similarity loss.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$\lambda_d$</td><td style='text-align: center; word-wrap: break-word;'>0.1</td><td style='text-align: center; word-wrap: break-word;'>Weight for delta depth similarity loss.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$\mathcal{T}$</td><td style='text-align: center; word-wrap: break-word;'>50</td><td style='text-align: center; word-wrap: break-word;'>Size of the sliding window for recent images.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$k_{curr}$</td><td style='text-align: center; word-wrap: break-word;'>1</td><td style='text-align: center; word-wrap: break-word;'>Number of frames selected from the current window.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$k_{his}$</td><td style='text-align: center; word-wrap: break-word;'>1</td><td style='text-align: center; word-wrap: break-word;'>Number of frames selected from historical data.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$lr_{position}$</td><td style='text-align: center; word-wrap: break-word;'>0.0005</td><td style='text-align: center; word-wrap: break-word;'>Learning rate for position parameters.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$lr_{color}$</td><td style='text-align: center; word-wrap: break-word;'>0.0025</td><td style='text-align: center; word-wrap: break-word;'>Learning rate for color parameters.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$lr_{opacity}$</td><td style='text-align: center; word-wrap: break-word;'>0.025</td><td style='text-align: center; word-wrap: break-word;'>Learning rate for opacity parameters.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$lr_{scale}$</td><td style='text-align: center; word-wrap: break-word;'>0.0025</td><td style='text-align: center; word-wrap: break-word;'>Learning rate for scale parameters.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>$lr_{rotation}$</td><td style='text-align: center; word-wrap: break-word;'>0.0025</td><td style='text-align: center; word-wrap: break-word;'>Learning rate for rotation parameters.</td></tr></table>

ing metrics, as shown in Table 8. The time and GPU memory consumption of each algorithm module over time are depicted in Figure 13. In the later stages of the long time series, most of the time is spent on rendering the ultra-large 3DGS map, map feedback, and optimization. It can be seen that our method is still able to achieve real-time map expansion in ultra-long sequences, with resource consumption in ultra-large scenes remaining within acceptable levels.

#### 7.6. Hyper Parameters in Our Experiments

The hyper parameters used in all our sequences are listed in Table 9. For more configuration details, please refer to the config file in our GitHub repository.

### 8. Limitations

In this paper, real-time performance is our primary focus, with all modules specifically designed to minimize the time complexity of the algorithm, thus ultimately enabling real-time photo-realistic mapping in large-scale open environments. However, the study does have some limitations: The initialization of the 3DGS model is solely reliant on the point cloud data, which results in initialization failure in areas lacking point cloud coverage, leading to missing regions. Future research will explore integrating image data to aid in the initialization process. Moreover, due to the real-time constraints, the reconstruction quality may not match that of images obtained through actual data acquisition. Improving the reconstruction quality will be part of our future research endeavors.