# Omni-LIVO: Robust RGB-Colored Multi-Camera Visual-Inertial-LiDAR Odometry via Photometric Migration and ESIKF Fusion Yinong Cao$^{1}$, Chenyang Zhang$^{1}$, Xin He$^{1,*}$, Yuwei Chen$^{1}$, Chengyu Pu$^{1}$, Bingtao Wang$^{1}$, Kaile Wu$^{1}$, Shouzheng Zhu$^{1}$, Fei Han$^{2}$, Shijie Liu$^{1}$, Chunlai Li$^{1}$ and Jianyu Wang$^{1,3}$ Abstract—Wide field-of-view (FoV) LiDAR sensors provide dense geometry across large environments, but existing LiDAR-inertial-visual odometry (LIVO) systems generally rely on a single camera, limiting their ability to fully exploit LiDAR-derived depth for photometric alignment and scene colorization. We present Omni-LIVO, a tightly coupled multi-camera LIVO system that leverages multi-view observations to comprehensively utilize LiDAR geometric information across extended spatial regions. Omni-LIVO introduces a Cross-View direct alignment strategy that maintains photometric consistency across non-overlapping views, and extends the Error-State Iterated Kalman Filter (ESIKEF) with multi-view updates and adaptive covariance. The system is evaluated on public benchmarks and our custom dataset, showing improved accuracy and robustness over state-of-the-art LIVO, LIO, and visual-inertial SLAM baselines. Index Terms—SLAM, multi-camera system, tightly coupled, visual-inertial-LiDAR fusion, direct visual methods, voxel map. ### I. INTRODUCTION SIMULTANEOUS Localization and Mapping (SLAM) has become essential for autonomous systems operating in unknown environments. Traditional methods are categorized by sensor types: vision-based SLAM [^1], [^2] provides rich visual information but struggles in low-light or texture-poor scenes, while LiDAR-based SLAM [^3] offers robust geometry but degrades in repetitive environments. IMUs, though environment-independent, suffer from long-term drift. To address single-sensor limitations, multi-sensor fusion systems like visual-inertial SLAM (VI-SLAM) and LiDAR-inertial SLAM (LI-SLAM) enhance robustness through complementary sensor properties [^4], [^5]. Recent LIVO systems combine dense LiDAR geometry, visual measurements, and high-frequency IMU measurements. LVI-SAM [^6] employs feature-based visual tracking, while FAST-LIVO [^7] and FAST-LIVO2 [^8] achieve high accuracy through photometric patch alignment on LiDAR-derived map points. Multi-camera configurations offer significant advantages for LIVO systems through extended spatial coverage, enabling comprehensive utilization of LiDAR-derived depth for photometric alignment, richer geometric constraints, improved observability, and enhanced robustness under occlusion and illumination changes. While multi-camera systems have been explored in visual SLAM [^9], [^10], their tight integration with LiDAR and inertial data in LIVO systems remains underexplored. Realizing these advantages requires addressing key technical challenges. Unlike stereo systems that use overlapping views for feature matching, non-overlapping multi-camera configurations require different approaches. First, photometric continuity must be maintained through temporal patch migration as patches transition between cameras. Second, adaptive covariance modeling is needed to handle heterogeneous measurement reliability across different views [^11], [^12]. Directly leveraging LiDAR-derived depth for multi-camera sparse direct alignment maintains efficient performance while enabling accurate visual point selection and extended spatial coverage beyond single-camera constraints. To address these challenges, we present Omni-LIVO $^{1}$, a tightly coupled multi-camera LIVO system that extends FAST-LIVO2 with multi-view photometric constraints. The main contributions are: 1) Cross-View temporal migration: A direct alignment strategy preserving photometric consistency across non-overlapping views. 2) Adaptive multi-view ESIKF: Enhanced ESIKF with adaptive covariance for improved robustness in adverse conditions. 3) Extensive experimental evaluation: Comprehensive evaluation of the tightly coupled multi-camera LIVO framework on public benchmarks and custom datasets. ### II. RELATED WORKS ### A. LiDAR-Inertial Odometry Evolution LiDAR-inertial odometry (LIO) systems evolved from feature-based LiDAR odometry like LOAM [^3] and LeGO-LOAM [^13] to direct methods. FAST-LIO [^14] eliminated --- feature extraction and directly registered raw points using an iterated Kalman filter. FAST-LIO2 [^15] further improved efficiency through incremental map updates with ikd-Tree. However, these systems lack visual information for operation in geometrically degraded environments. ### B. LiDAR-Visual-Inertial Integration LiDAR-visual-inertial fusion addresses pure LIO limitations. R²LIVE [^16] employs sparse visual features with reprojection error minimization. R³LIVE [^17] advanced to direct photometric alignment with RGB-color consistency. FAST-LIVO [^7] pioneered tight coupling by attaching image patches to LiDAR map points. FAST-LIVO2 [^8] refined this through sequential ESIKF updates with direct methods. However, existing LIVO systems predominantly employ single-camera configurations, limiting comprehensive exploitation of wide-FoV LiDAR geometry across extended spatial coverage. ### C. Multi-Camera Visual-Inertial Systems Multi-camera visual-inertial SLAM has gained attention for extended spatial coverage. MIMC-VINS [^18] supports arbitrary camera numbers using multi-state constraint Kalman filtering. BAMF-SLAM [^19] integrates multiple fisheye cameras with IMU through bundle adjustment. For large FoV coverage, LF-VIO [^20] and LF-VISLAM [^21] employ negative imaging plane models for single wide-FoV cameras. MAVIS [^10] extends to partially overlapped multi-camera systems. These systems operate in the visual-inertial domain without leveraging LiDAR geometric structure. ### D. Direct Visual Methods Direct visual methods minimize photometric error without feature extraction. DTAM [^22], LSD-SLAM [^1], and DSO [^23] established foundations for dense and sparse tracking. MultiCol-SLAM [^24] extended direct methods to multi-camera systems. This suits multi-camera LIVO by avoiding correspondence complexity in non-overlapping views. ### III. SYSTEM OVERVIEW AND METHODOLOGY The Omni-LIVO system comprises four modules: Sensor Acquisition, Preprocessing, Multi-Camera ESIKF, and Unified Mapping, as illustrated in Fig. 1. The processing pipeline begins with IMU propagation, followed by sequential ESIKF updates that integrate LiDAR point-to-plane residuals with multi-view photometric constraints. Cross-View temporal migration and adaptive covariance preserve photometric consistency across non-overlapping views. ### A. Notational Conventions Table I summarizes the key mathematical notation used throughout this paper.
TABLE I
NOTATIONAL CONVENTIONS
SymbolDescription
$\mathbf{T}_{c}^{w}$Transformation from world to camera frame
$\mathbf{R}_{c}$, tRotation matrix and translation vector
p3D point in world coordinates
$\tau_{c}$Inverse exposure time parameter for camera c
$N_{c}$Number of cameras
$\mathcal{P}_{k}^{\text{seg}}$Temporally segmented LiDAR point cloud
$\mathcal{C}_{k}^{(i)}$Image frame from camera i at time k
$\mathbf{V}_{k}$Voxel containing geometric and visual data
$\mathbf{n}_{k}$Plane normal vector
$\mathbf{r}_{VIO}$Visual-inertial odometry residual vector
$\mathbf{r}_{LIO}$LiDAR-inertial odometry residual
$\hat{\mathbf{x}}^{-}$, $\mathbf{P}^{-}$Prior state estimate and covariance
$\hat{\mathbf{x}}_{k|k}$, $\mathbf{P}_{k|k}$Posterior state estimate and covariance
$\mathbf{H}$Jacobian matrix
$\mathbf{K}$Kalman gain
$\ell$Iteration index
$\boxplus$Manifold addition operator
### B. Multi-Sensor Data Alignment Building upon FAST-LIVO2 [^8], we extend temporal alignment to multi-camera configurations [^25]. The system forms temporally coherent measurement packets: $$ \mathcal{M}_{\mathrm{k}}=\{\mathcal{P}_{\mathrm{k}}^{\mathrm{s e g}},\{\mathcal{J}_{\mathrm{j}}\}_{\mathrm{j}=\mathrm{k}-1}^{\mathrm{k}},\{\mathcal{C}_{\mathrm{k}}^{(\mathrm{i})}\}_{\mathrm{i}=1}^{N_{c}}\} $$ where $\mathcal{M}_k$ denotes the multi-sensor measurement packet at time k, and $\{\mathcal{J}_j\}_{j=k-1}^k$ represents the set of high-frequency IMU measurements spanning the inter-frame interval. Although LiDAR and cameras may operate at the same nominal rate (e.g., 10 Hz), their trigger events are not always aligned. To align data across modalities, each LiDAR point $p_i$ with timestamp $\tau_i$ is assigned to the temporally nearest camera frame: $$ p_{i}\rightarrow frame_{k},\quad\mathrm{w i t h}k=\underset{j}{\arg\min}|\tau_{i}-t_{\mathrm{s y n c}}^{(j)}| $$ where $t_{\mathrm{sync}}^{(j)}$ is the sync trigger time. Within each frame k, all camera channels are externally triggered to ensure simultaneous exposure: $$ t_{\mathrm{c a m}}^{(i,k)}=t_{\mathrm{s y n c}}^{(k)},\quad\forall i\in[1,N_{c}] $$ where $t_{\mathrm{cam}}^{(i,k)}$ is the trigger time for camera i at frame k. ### C. Cross-View Direct Method We formalize Cross-View temporal migration to maintain photometric continuity across non-overlapping views (Fig. 3). The migration indicator $M(\mathbf{p}, i \to j)$ determines whether a map point transitions from camera i to j: $$ M(\mathbf{p},\mathrm{i}\rightarrow\mathrm{j})=1\Leftrightarrow\left\{\begin{aligned}&\mathrm{cam}_{\mathrm{ref}}(\mathbf{p})=\mathrm{i}\\&\mathrm{cam}_{\mathrm{cur}}(\mathbf{p})=\mathrm{j}\\&\mathrm{i}\neq\mathrm{j}\\&\text{exclusive}_{\text{visible}}(\mathbf{p},\mathrm{j})=1\\&\text{temporal}_{\text{migrated}}(\mathbf{p},\mathrm{i}\rightarrow\mathrm{j})=1\end{aligned}\right. $$ where $\mathrm{cam}_{\mathrm{ref}}(\mathbf{p})$ and $\mathrm{cam}_{\mathrm{cur}}(\mathbf{p})$ return the camera IDs observing $\mathbf{p}$ in reference and current frames, exclusive $\mathrm{visible}(\mathbf{p}, \mathrm{j}) = 1$ --- ![Image](images/f8414a1a129bbd96d1c26ede8d616ca98e445cee3c4ca2d10af2c06c9041f3be.jpg)
Fig. 1. Overview of the Omni-LIVO architecture. Light yellow, light blue, and light green modules represent visual, LiDAR, and IMU data respectively. Orange borders indicate extensions to baseline methods; blue borders indicate novel contributions.
![Image](images/0af7573c812d98c4dd9260cacc806ffc5d4083f5bc900422cdd6e21981cb1f2e.jpg)
Fig. 2. Multi-sensor temporal alignment. LiDAR points are associated with the nearest camera frame, while IMU measurements provide continuous motion priors for temporally consistent fusion.
![Image](images/034956e7a1d30bd77d0265f7063a00ffb28e776627ec4fa8d275edada10ff798.jpg)
Fig. 3. Cross-View temporal migration. At $t_1$, map points $\mathbf{p}_1$, $\mathbf{p}_2$, $\mathbf{p}_3$ are observed by Left $(C_L)$, Front $(C_F)$, and Right $(C_R)$ cameras respectively. As the platform moves, points migrate between camera views $(\mathbf{p}_2$ migrates from $C_F$ to $C_L$ at $t_2)$, maintaining photometric continuity across non-overlapping $90^\circ$ FoVs through Cross-View residuals.
ensures $\mathbf{p}$ is visible only in camera j (i.e., $\pi_j(\mathbf{T}_j^w \cdot \mathbf{p}) \in \Omega_j$ with $\Omega_j$ being the valid image region, and invisible in other cameras), and $\text{temporal}_{\text{migrated}}(\mathbf{p}, \mathbf{i} \to \mathbf{j}) = 1$ verifies $\mathbf{p}$ transitioned from camera i to j across successive frames. When camera migration occurs, the reference patch is dynamically switched by prioritizing observations from the current camera, or selecting the patch with minimum photometric error: $$ \mathbf{p}_{ref}^{*}=\underset{\mathbf{p}_{k}\in\mathcal{O}(\mathbf{p})}{arg min}\frac{1}{|\mathcal{O}(\mathbf{p})|-1}\sum_{\mathbf{p}_{m}\in\mathcal{O}(\mathbf{p})\backslash\{\mathbf{p}_{k}\}}\|\mathbf{I}_{k}-\mathbf{I}_{m}\|^{2} $$ where $\mathcal{O}(\mathbf{p})$ denotes the set of all observations of point $\mathbf{p}$, $\mathbf{I}_k \in \mathbb{R}^P$ is the vectorized patch intensity, and P is the patch size. To ensure consistent intensity comparison across cameras, image measurements are normalized by vignetting: $$ I_{k}^{\prime}(u,v)=V_{k}(r^{2})\cdot I_{k}(u,v) $$ where $I_{k}(u,v)$ denotes the raw image intensity at pixel coordinates $(u,v)$, $I_{k}^{\prime}(u,v)$ is the vignetting-normalized intensity, r is the radial distance from the image center, and $V_{k}(r^{2})$ is a vignetting function modeled as: $$ V_{k}(r^{2})=1+\sum_{n=1}^{3}\beta_{k,n}\cdot r^{2n} $$ where $\beta_{k,n}$ are the vignetting polynomial coefficients calibrated offline for each camera. Exposure variations across cameras are handled through per-camera inverse exposure time parameters $\tau_{c}$ jointly optimized in the ESIKF framework, ensuring photometric consistency without requiring relative exposure factor estimation. The relative transformation and induced homography under local planar assumption are formulated to warp patches between cameras. The relative transformation from camera i to camera j is: $$ \mathbf{T}_{\mathrm{i}\rightarrow\mathrm{j}}(t)=\mathbf{T}_{\mathrm{j}}^{\mathrm{w}}(t_{\mathrm{j}})\cdot(\mathbf{T}_{\mathrm{i}}^{\mathrm{w}}(t_{\mathrm{i}}))^{-1} $$ Under the local planar assumption, the homography matrix $\mathbf{H}_{i\to j}(\mathbf{p})$ that maps points from camera i to camera j is [^26]: $$ \mathbf{H}_{\mathrm{i}\rightarrow\mathrm{j}}(\mathbf{p})=\mathbf{R}_{\mathrm{i}\rightarrow\mathrm{j}}+\frac{\mathbf{t}_{\mathrm{i}\rightarrow\mathrm{j}}\mathbf{n}_{\mathrm{i}}^{\mathrm{T}}}{d_{\mathrm{i}}} $$ --- ![Image](images/b2de486cf1d4b59500907e964e3ec68efc02fc3c04c39a87d9dc0ba82dc73dfd.jpg)
Fig. 4. Photometric patch associations during turning: intra-camera patches (green) remain within the same view, while migrated patches (red) form Cross-View associations across different cameras.
where $\mathbf{R}_{i\to j}$ and $\mathbf{t}_{i\to j}$ are the rotation and translation from $\mathbf{T}_{i\to j}$, $\mathbf{n}_i$ is the unit surface normal at the patch in camera i frame, and $d_i$ is the point depth in camera i frame. This homography enables direct patch warping across non-overlapping camera views for photometric alignment. Based on these definitions, we formulate both intra-camera and Cross-View residuals, with the patch association process visualized in Fig. 4. For intra-camera alignment, the direct photometric residual $r_{direct}$ is: $$ \begin{array}{r}{r_{\mathrm{d i r e c t}}^{\mathrm{c}}(\mathbf{p})=\tau_{c}\cdot I_{c}^{\prime}(\boldsymbol{\pi}_{c}(\mathbf{T}_{c}^{w}\cdot\mathbf{p}))-\tau_{\mathrm{r e f}}\cdot I_{\mathrm{r e f}}^{\prime}(\boldsymbol{\pi}_{\mathrm{r e f}}(\mathbf{p}_{\mathrm{r e f}}))}\end{array} $$ where $\pi_{c}(\cdot)$ denotes the camera projection function from 3D to 2D image coordinates, and subscript “ref” denotes the reference frame. Cross-View temporal migration residuals $r_{migration}$ establish constraints between different cameras across time: $$ r_{\mathrm{m i g r a t i o n}}^{i\rightarrow j}(p)=\tau_{i}\cdot W_{H}\left[I_{i}^{\prime},\pi_{i}(p,t_{i})\right]-\tau_{j}\cdot I_{j}^{\prime}(\pi_{j}(p,t_{j})) $$ where camera i is the source (reference) and camera j is the target (current), $W_{H}$ represents warping using homography H, and $t_{i}$, $t_{j}$ are the respective camera timestamps. $$ \mathbf{J}_{\mathbf{R}}^{\mathrm{c o u p l e d}}=\mathbf{J}_{\mathbf{R}}^{i}-\mathbf{J}_{\mathbf{R}}^{j},\quad\mathbf{J}_{\mathbf{t}}^{\mathrm{c o u p l e d}}=\mathbf{J}_{\mathbf{t}}^{i}-\mathbf{J}_{\mathbf{t}}^{j} $$ For cross-camera constraints, we formulate bidirectional coupled Jacobians that account for geometric contributions from both cameras: where $\mathbf{J}_{\mathbf{R}}^{k}=\tau_{k}\frac{\partial I_{k}}{\partial\mathbf{R}}$ and $\mathbf{J}_{\mathbf{t}}^{k}=\tau_{k}\frac{\partial I_{k}}{\partial\mathbf{t}}$ are exposure-weighted Jacobians with respect to rotation and translation for camera k. Similarly, exposure Jacobians are computed as $\frac{\partial r^{i\rightarrow j}}{\partial\tau_{i}}=I_{i}$ and $\frac{\partial r^{i\rightarrow j}}{\partial\tau_{j}}=-I_{j}$. This bidirectional formulation enforces both geometric and photometric consistency between camera pairs. The complete residual vector $\mathbf{r}_{VIO}$ stacks all intra-camera and Cross-View residuals for joint optimization: $$ \mathbf{r}_{V I O}=\left[\mathbf{r}_{\mathrm{d i r e c t}}^{1},\ldots,\mathbf{r}_{\mathrm{d i r e c t}}^{N_{c}},\mathbf{r}_{\mathrm{m i g r a t i o n}}^{1\to2},\ldots,\mathbf{r}_{\mathrm{m i g r a t i o n}}^{i\to j}\right]_{(12)}^{T} $$ Temporal migration constraints preserve photometric continuity across cameras, enhancing geometric diversity and pose estimation robustness. Multi-camera configurations enable stricter photometric thresholds while maintaining sufficient observations, improving both efficiency and accuracy. ### D. LiDAR Geometric Constraint Processing LiDAR measurements provide structural constraints that complement photometric observations in our multi-camera framework. We employ voxel-based plane extraction to efficiently process point clouds and generate geometric residuals. The raw LiDAR point cloud is discretized into voxels of resolution $v_{\text{size}}$. For each voxel $\mathcal{V}_k$ containing $N_k$ points $\mathcal{P}_k = \{p_i\}_{i=1}^{N_k}$, we compute the voxel centroid $\bar{p}_k = \frac{1}{N_k} \sum_{i=1}^{N_k} p_i$ and the sample covariance matrix $\mathbf{C}_k$: $$ \mathbf{C}_{k}=\frac{1}{N_{k}}\sum_{i=1}^{N_{k}}(p_{i}-\bar{p}_{k})(p_{i}-\bar{p}_{k})^{\mathrm{T}} $$ Planar voxels are identified via eigen-decomposition of $\mathbf{C}_k$. If the smallest eigenvalue $\lambda_{\min} < \epsilon_{\text{plane}}$ (where $\epsilon_{\text{plane}}$ is the planarity threshold), the corresponding eigenvector defines the plane normal $\mathbf{n}_k$, which satisfies the plane equation for any point p on the plane: $$ \mathbf{n}_{k}^{\mathrm{T}}(p-\bar{p}_{k})=0 $$ LiDAR geometric residuals are constructed by measuring point-to-plane distances. Let $\mathbf{p}^{\mathrm{w}}$ denote a LiDAR point in world frame and $\mathbf{T}_{\mathrm{L}}^{\mathrm{w}}$ the world-to-LiDAR transformation. The point in LiDAR frame is $\mathbf{p}^{\mathrm{L}} = \mathbf{R}_{\mathrm{L}}^{\mathrm{w}}\mathbf{p}^{\mathrm{w}} + \mathbf{t}_{\mathrm{L}}^{\mathrm{w}}$, and the LiDAR-inertial residual $\mathbf{r}_{LIO}$ measuring point-to-plane distance is: $$ \mathbf{r}_{LIO}(\mathbf{p}^{\mathrm{w}})=\mathbf{n}_{k}^{\mathrm{T}}(\mathbf{R}_{\mathrm{L}}^{\mathrm{w}}\mathbf{p}^{\mathrm{w}}+\mathbf{t}_{\mathrm{L}}^{\mathrm{w}}-\bar{\mathbf{p}}_{k}) $$ where $\bar{p}_{k}$ is the plane centroid in LiDAR frame. ### E. Iterative Multi-Camera State Estimation with ESIKF Measurement quality varies across cameras due to heterogeneous exposure and environments. We extend ESIKF to integrate multi-view photometric constraints with LiDAR measurements through sequential update and adaptive covariance, as shown in Fig. 5. The state is first updated with LiDAR point-to-plane residuals, then sequentially updated with multi-camera photometric residuals. Multi-camera residuals and Jacobians are stacked across all cameras and processed through iterative ESIKF updates until convergence. For heterogeneous exposure settings, independent inverse exposure time parameters $\tau_c$ are maintained for each camera $c \in [1, N_c]$ and jointly optimized within the ESIKF framework. The Jacobian matrix is extended to $\mathbb{R}^{m \times (6 + N_c)}$, where m is the total number of measurements, 6 represents pose parameters (rotation and translation), and $N_c$ represents per-camera exposure parameters. The photometric covariance matrix $R_{img}$ is adaptively scaled based on real-time photometric error to account for varying measurement quality. The covariance scaling factor $\sigma^{(t)}$ at time t is computed via linear mapping: $$ \sigma^{(t)}=\sigma_{\min}+\frac{\bar{e}^{(t-1)}-1}{e_{\max}-1}\cdot\left(\sigma_{\max}-\sigma_{\min}\right) $$ where $\bar{e}^{(t-1)}$ is the mean squared photometric error from the previous frame, $\sigma_{\min}$ and $\sigma_{\max}$ are the minimum and maximum covariance scaling bounds (set to 100.0 and 1000.0). --- ![Image](images/63e8511731e91fc2e8a679e44934685c725c1e3d469fefd41f14a55e4aa4725c.jpg)
Fig. 5. Iterative Multi-Camera ESIKF Framework. The pipeline performs Synchronize, IMU Propagate, LiDAR Update, Select Patches, and Adaptive Cov, followed by Init VIO and iterative updates of the photometric VIO loop until convergence.
respectively), and $e_{max}$ is the maximum error threshold for scaling (typically set to 100.0). Temporal smoothing ensures stability: $$ \tilde{\boldsymbol{\sigma}}^{(t)}=\lambda\cdot\boldsymbol{\sigma}^{(t)}+\left(1-\lambda\right)\cdot\tilde{\boldsymbol{\sigma}}^{(t-1)} $$ where $\tilde{\sigma}^{(t)}$ is the smoothed covariance scaling factor and $\lambda$ is the smoothing parameter (set to 0.3). This adaptive weighting down-weights cameras with high photometric errors (e.g., motion blur, occlusion) and up-weights reliable observations, improving robustness in adverse conditions. ```pseudo // Algorithm 1: summarizes the complete implementation. Algorithm-specific notation: $\tilde{\mathbf{x}}^{(\ell)}$ is the linearization point; $\mathbf{r}_{c}^{(\ell)}$, $\mathbf{H}_{c}^{(\ell)}$ are per-camera residuals and Jacobians stacked into $\mathbf{r}_{VIO}^{(\ell)}$, $\mathbf{H}_{VIO}^{(\ell)}$; $\delta\hat{\mathbf{x}}^{(\ell)}$ is the state increment; $e^{(\ell)}$ is the mean squared error; $\ell_{\max} = 10$ is the maximum iteration count. ``` ```pseudo // Algorithm 1: Multi-Camera ESIKF Update ``` 1 IMU propagation: $\hat{\mathbf{x}}^{-}$, $\mathbf{P}^{-}$; LiDAR plane extraction 2 LiDAR update: Iterative ESIKF with $\mathbf{r}_{LIO}$, $\mathbf{H}_{LIO}$ 3 Select patches from cameras $c \in [1, N_c]$; Compute $\tilde{\boldsymbol{\sigma}}^{(t)}$ via Eq. (17)-(18) 4 VIO update: $\ell \leftarrow 0$, $\delta \hat{\mathbf{x}}^{(0)} \leftarrow \mathbf{0}$, $e_{last} \leftarrow \infty$ 5 repeat 6 $\ell \leftarrow \ell + 1$; Update $\tilde{\mathbf{x}}^{(\ell)} = \hat{\mathbf{x}}^{-}$ $\oplus \delta \hat{\mathbf{x}}^{(\ell-1)}$ 7 for camera $c = 1, \ldots, N_c$ do 8 $\lfloor \text{Compute } \mathbf{r}_c^{(\ell)}, \mathbf{H}_c^{(\ell)} \text{ with cross-camera constraints}$ 9 Stack $\mathbf{r}_{VIO}^{(\ell)} = [\mathbf{r}_1^{(\ell)}; \ldots; \mathbf{r}_{N_c}^{(\ell)}]$, $\mathbf{H}_{VIO}^{(\ell)} = [\mathbf{H}_1^{(\ell)}; \ldots; \mathbf{H}_{N_c}^{(\ell)}]$ 10 Compute $e^{(\ell)} = \|\mathbf{r}_{VIO}^{(\ell)}\|^2 / n_{\text{meas}}$ 11 if $e^{(\ell)} \leq e_{last}$ then 12 $\left| e_{last} \leftarrow e^{(\ell)} \right|$ 13 $\mathbf{K} = ((\mathbf{H}_{VIO}^{(\ell)})^{\mathrm{T}} \mathbf{H}_{VIO}^{(\ell)} + \mathbf{P}^{-1} / \tilde{\boldsymbol{\sigma}}^{(t)})^{-1}$ 14 $\delta \hat{\mathbf{x}}^{(\ell)} = \mathbf{K}((\mathbf{H}_{VIO}^{(\ell)})^{\mathrm{T}} \mathbf{r}_{VIO}^{(\ell)} - \mathbf{P}^{-1} \delta \hat{\mathbf{x}}^{(\ell-1)} / \tilde{\boldsymbol{\sigma}}^{(t)})$ 15 $\hat{\mathbf{x}}^{-} \leftarrow \hat{\mathbf{x}}^{-}$ $\oplus \delta \hat{\mathbf{x}}^{(\ell)}$ 16 else 17 $\lfloor \text{break}\rfloor$ 18 until $\|\delta \mathbf{R}^{(\ell)}\| < 0.001^\circ$ and $\|\delta \mathbf{t}^{(\ell)}\| < 0.01mm$ or $\ell \geq \ell_{\text{max}}$ 19 $\mathbf{P}_{k|k} = (\mathbf{I} - \mathbf{K} \mathbf{H}_{VIO}^{(\ell)})^{\mathbf{P}^{-}}$ 20 return $\hat{\mathbf{x}}_{k|k}$, $\mathbf{P}_{k|k}$ ### F. Unified Voxel Mapping with Multi-View Integration We integrate LiDAR geometry and multi-view photometric patches within a single adaptive volumetric structure employing density-driven octree refinement [^27]. Each voxel $V_{k}$ encapsulates three components: where $N_{obs}$ is the number of observations, $\mathcal{P}_{j}^{(k)}$ is the patch organized in multi-level pyramid structure, $T_{wc}^{(k)}$ is the camera pose, $\tau^{(k)}$ is the exposure parameter, and $id_{cam}^{(k)}$ is the camera identifier. $$ \mathcal{V}_{k}=\{\mathcal{G}_{k},\mathcal{W}_{k},\mathcal{S}_{k}\} $$ where $\mathcal{G}_k$ represents geometric plane parameters from LiDAR, $\mathcal{W}_k$ contains visual map points with multi-view observations, and $\mathcal{S}_k$ stores octree subdivision structure. Visual map points are generated by partitioning images into regular grids and selecting LiDAR points with highest Shi-Tomasi corner response. Each visual point j maintains a feature structure $\mathcal{F}_j$ containing $N_{\text{obs}}$ observations across multiple frames: $$ \mathcal{F}_{j}=\{\mathcal{P}_{j}^{(k)},\mathrm{T}_{w c}^{(k)},\tau^{(k)},i d_{\mathrm{c a m}}^{(k)}\}_{k=1}^{N_{\mathrm{o b s}}} $$ ### IV. EXPERIMENTAL EVALUATION We evaluate the proposed system on public benchmarks and custom datasets, and compare with state-of-the-art methods. ### A. Datasets and Baselines To demonstrate the effectiveness of the proposed multicamera LIVO framework, we conduct comprehensive experiments evaluating trajectory accuracy, ablation of key components, computational efficiency, and qualitative mapping performance across diverse environments. Evaluation is conducted on Hilti SLAM Challenge 2022/2023 and Newer College Dataset. We additionally collected a custom dataset comprising multiple sequences using LIVOX MID360 LiDAR (360° horizontal FoV, 10Hz) and four cameras (1024×768, 10Hz) in cross-pattern configuration with handheld and robotic platforms (Fig. 6). We compare against five methods: FAST-LIVO2, FAST-LIVO, R³LIVE (LiDAR-inertial-visual), FAST-LIO2 (LiDAR-inertial), and OpenMAVIS (multi-camera visual-inertial). All LIVO systems use 10Hz for both camera and LiDAR data acquisition. FAST-LIVO2 and our system use identical LIO and IMU parameters. Ablation studies are conducted using five system variants: w/o ac (Adaptive Covariance disabled), w/o cvtm (Cross-View Temporal Migration disabled), 1-Cam, 2-Cam, and Ours (complete system with three cameras). ---
TABLE II
ABSOLUTE TRAJECTORY ERRORS (RMSE IN METERS) ON MULTIPLE BENCHMARK DATASETS
DatasetSequenceBaseline MethodsOmni-LIVO Variants
FAST-LIVOFAST-LIVO2$R^{3}$LIVEFAST-LIO2OpenMAVISw/o acw/o cvtm1-Cam2-CamOurs
Hiti 2022Attic to Upper Gallery1.6500.150$\times$$\times$0.1030.0820.0890.1760.0850.069
Attic to Upper Gallery 21.1650.123$\times$3.5910.2550.1290.1410.1220.1340.120
Basement 20.1290.0430.0970.1360.0820.0380.0360.0490.0310.031
Construction Ground0.0160.0100.0150.0180.0940.0100.0110.0110.0100.011
Construction Multilevel0.1550.0200.0210.0380.9330.0210.0200.0220.0200.020
Construction Stairs0.2910.0390.6110.6250.0540.0280.0320.0420.0310.030
Construction Upper 10.0320.0300.0280.1240.0830.0290.0290.0290.0290.027
Construction Upper 20.0890.0410.0410.1000.0400.0190.0200.0370.0220.009
Construction Upper 30.0910.0450.049$\times$0.0750.0270.0250.0470.0250.011
Cupola0.2700.130$\times$$\times$0.1400.1330.1290.1440.1310.126
Cupola 2$\times$0.185$\times$$\times$0.1360.1270.1460.1870.1210.098
Long Corridor0.0720.0560.0690.0770.0980.0520.0550.0560.0530.051
Lower Gallery0.0650.0330.0100.0560.0640.0090.0110.0210.0110.006
Lower Gallery 20.3870.155$\times$$\times$0.1780.1090.1190.1580.1080.083
Outside Building0.0470.0310.0330.0290.4250.0200.0180.0220.0160.016
Hiti 2023Floor 00.0300.0230.0450.0680.0600.0210.0200.0300.0240.020
Floor 10.0290.0200.0450.0550.1270.0180.0190.0200.0190.017
Floor 20.0930.0220.1210.1210.1780.0120.0220.0200.0180.012
Basement0.0720.0190.0700.1900.2460.0210.0200.0200.0160.016
Stairs0.2300.0200.2020.1540.2250.0100.0130.0190.0130.008
Newer CollegeQuad easy0.0990.0820.0860.0920.0840.0770.0810.0800.0770.071
Quad medium1.5300.0730.0610.1010.1130.0690.0740.0750.0700.065
Quad hard1.7570.0830.0710.1460.1240.0730.0750.0750.0720.057
Stairs0.1240.0570.224$\times$$\times$0.0490.0510.0460.0610.048
Underground easy0.1250.0490.0370.1560.1200.0510.0500.0490.0510.049
Underground medium0.0880.0430.0470.1530.1300.0370.0410.0450.0390.034
Underground hard0.1570.2410.0690.1410.2550.0680.0720.0960.0610.055
![Image](images/7a3b4016acc0d26ce52c5bc9df36af336b22de911fe110b66ed2c43a8b92388b.jpg)
(a) Handheld device setup
![Image](images/6ae93d0ab275c7375709022ffe0d2fd80a840b879b5ac7a4de84ab965003c0f6.jpg)
(b) Robotic platform setup
Fig. 6. SLAM data acquisition platforms used in our custom dataset. (a) Handheld configuration. (b) Robotic configuration.
### B. Quantitative Results Table II summarizes absolute trajectory errors (bold: best, underline: second-best) across all sequences. Omni-LIVO variants achieve the lowest error in 25 out of 27 sequences and successfully complete all evaluated tracks, outperforming FAST-LIVO2, R $^{3}$LIVE, FAST-LIO2, and OpenMAVIS across diverse environments including construction sites, indoor corridors, and underground scenarios. Compared to FAST-LIVO2, our system achieves an average 30% error reduction across all 27 sequences, with improvements up to 82% on Lower Gallery. Ablation analysis demonstrates that Cross-View Temporal Migration improves accuracy in sequences with large viewpoint changes (Cupola 2: 0.098 m vs. 0.146 m w/o cvtm), while Adaptive Covariance enhances robustness under illumination variations (Lower Gallery 2: 0.083 m vs. 0.109 m w/o ac). Camera number comparison shows progressive gains from 1-Cam to 3-Cam (Ours), with the largest improvements in complex environments (Underground hard: 0.096 m to 0.055 m). This trend indicates that multi-camera fusion provides greater benefits in geometrically challenging scenarios. ### C. Computational Performance Computational complexity is $\mathcal{O}(N_p \cdot P \cdot L)$ for single-camera and $\mathcal{O}((N_p + N_{\text{cross}}) \cdot P \cdot L)$ for multi-camera, where $N_p$ is visual points, P is patch size, L is pyramid levels, and $N_{\text{cross}}$ is cross-camera observations. We use NVIDIA Jetson AGX Orin with 12-core ARM CPU (2.2GHz) and 64GB RAM running Ubuntu 20.04. System parameters: LiDAR voxel size 1.0 m, surface filtering threshold 0.5 m, maximum 150 visual points per frame. Table III presents processing time and memory consumption. Compared to FAST-LIVO2, Omni-LIVO introduces $1.16\times$ overhead on Hilti (60.3 ms vs. 52.1 ms, 3 cameras) and $1.21\times$ on custom sequences (40.6 ms vs. 33.6 ms, 4 cameras), primarily due to cross-camera constraint computation, while maintaining real-time 10Hz performance with 2906 MB (Hilti) and 2359 MB (Custom) memory consumption, demonstrating practical viability for embedded platforms. ### D. Qualitative Results Fig. 7 demonstrates Omni-LIVO's mapping quality on 2 sequences with multi-camera configurations. Fig. 8 validates the system's ability to maintain global consistency and achieve precise loop closure, with Omni-LIVO showing superior alignment compared to FAST-LIVO2's visible drift. The improved consistency is attributed to enhanced observability from multiview constraints. Table IV shows that Omni-LIVO produces --- ![Image](images/909fe3b9745d916bdf8c775d8e21353739ef3a12009044c1861bfdd1be923ae3.jpg) ![Image](images/44812fd3abb9fa90eb8e34a281cde0160cc34a68bf18b511c21176d0e033ab4b.jpg) ![Image](images/d1f344b6f67f627dccf2faeb94b72e8f256850df65585333cba633cd35271628.jpg)
(a) Cupola 2
![Image](images/810cdc8e7f975497402ebff0df5768f52f772b25451a2533f12b2a0ddaa739e6.jpg) ![Image](images/cf8e44c9fdf5af7f5a8c0a3824744887d9386593c722c62b24e15f846c7fba55.jpg)
(b) Underground Hard
![Image](images/077a5bc5016ba869da9be3c1c61a1385082d96f06a6007ea68e036c1f99b61e0.jpg)
Fig. 7. Mapping quality comparison on two challenging sequences. (a) Cupola 2 scene with overall map and detailed regions. (b) Underground Hard scene with overall map and detailed regions. The colored regions are grayscale LiDAR point clouds colorized using multi-camera gray images.
![Image](images/e4889a32a05bb8787f1099e1dfb17443d84121f47949249b1d704522dd9d0fde.jpg)
(a) Overall map
![Image](images/cc1baeeebde70d2664445b9984c9c7d7768e2cb8ae5dae2115095cb37f25cebe.jpg)
(b) Omni-LIVO: Start/End alignment
![Image](images/e7b2b08273e4524646f29ea43f6528221f5cab916467655af814afbbe79b302b.jpg)
(c) FAST-LIVO2: Start/End misalignment
Fig. 8. Loop closure comparison on staircase sequence. (a) Overall map. (b) Omni-LIVO: precise start/end alignment. (c) FAST-LIVO2: visible drift.
more RGB-colored points than competing methods, with particularly significant improvements in complex environments (3.5× more than FAST-LIVO2 in basement3). ### E. Discussion Experimental results show that Omni-LIVO achieves improved accuracy over baseline methods across evaluated sequences with moderate computational overhead. The multicamera configuration provides extended spatial coverage and denser scene colorization. Ablation studies demonstrate that Cross-View constraints improve performance in sequences
TABLE III
COMPUTATIONAL PERFORMANCE COMPARISON
Sequence#CamFAST-LIVO2Omni-LIVO
Time (ms)Mem (MB)Time (ms)Mem (MB)
Construction Upper350.7190158.12017
Cupola 2351.2347757.03498
Outside Building354.5326165.93203
Avg. (Hilt)352.1288060.32906
Corridor417.3105421.01197
Basement3449.9347360.13520
Avg. (Custom)433.6226440.62359
---
TABLE IV
RGB POINT CLOUD STATISTICS (NUMBER OF COLORED POINTS)
Methodstairsclassroombasement3corridor
Omni-LIVO14,184,5004,484,88933,513,6088,715,790
FAST-LIVO25,539,7003,111,5049,519,1254,863,001
FAST-LIVO3,827,9551,611,5395,019,2002,543,585
R $^{3}$LIVE393,242619,3101,213,9291,007,194
with large viewpoint changes, while adaptive covariance enhances robustness under illumination variations. ### V. CONCLUSION This letter presents Omni-LIVO, a tightly coupled multi-camera LiDAR-inertial-visual odometry system that extends direct visual methods to multi-view configurations through Cross-View temporal migration and adaptive ESIKF integration. Evaluation on public benchmarks and custom datasets shows that the system achieves lower trajectory errors compared to state-of-the-art LIVO, LIO, and visual-inertial methods. The multi-camera configuration provides extended spatial coverage and produces denser RGB-colored point clouds. The quantitative and qualitative improvements demonstrate the effectiveness of the proposed multi-camera fusion approach. Future work will explore online loop closure and investigate multi-sensor calibration refinement. ## REFERENCES [^1]: J. Engel, T. Schöps, and D. Cremers, "LSD-SLAM: Large-scale direct monocular SLAM," in Proc. Eur. Conf. Comput. Vis. (ECCV), Zurich, Switzerland, Sep. 2014, pp. 834–849, doi: 10.1007/978-3-319-10605-2_54. [^2]: R. Mur-Artal and J. D. Tardós, “ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras,” IEEE Trans. Robot., vol. 33, no. 5, pp. 1255–1262, Oct. 2017, doi: 10.1109/TRO.2017.2705103. [^3]: J. Zhang and S. Singh, "LOAM: LiDAR odometry and mapping in real-time," in Proc. Robot.: Sci. Syst., Berkeley, CA, USA, Jul. 2014, pp. 1–9, doi: 10.15607/RSS.2014.X.007. [^4]: T. Qin, P. Li, and S. Shen, "VINS-Mono: A robust and versatile monocular visual-inertial state estimator," IEEE Trans. Robot., vol. 34, no. 4, pp. 1004–1020, Aug. 2018, doi: 10.1109/TRO.2018.2853729. [^5]: C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. M. Montiel, and J. D. Tardós, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM,” IEEE Trans. Robot., vol. 37, no. 6, pp. 1874–1890, Dec. 2021, doi: 10.1109/TRO.2021.3075644. [^6]: T. Shan, B. Englot, C. Ratti, and D. Rus, "LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), Xi'an, China, May 2021, pp. 5692–5698. [^7]: C. Zheng, Q. Zhu, W. Xu, X. Liu, Q. Guo, and F. Zhang, "FAST-LIVO: Fast and tightly-coupled sparse-direct LiDAR-inertial-visual odometry," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), Kyoto, Japan, Oct. 2022, pp. 4003–4009, doi: 10.1109/IROS47612.2022.9981107. [^8]: C. Zheng, W. Xu, Q. Guo, and F. Zhang, "FAST-LIVO2: Fast, direct LiDAR-inertial-visual odometry," IEEE Trans. Robot., vol. 41, pp. 326–346, 2025, doi: 10.1109/TRO.2024.3502198. [^9]: J. Kuo, M. Muglikar, Z. Zhang, and D. Scaramuzza, “Redesigning SLAM for arbitrary multi-camera systems,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), Paris, France, May 2020, pp. 2116–2122, doi: 10.1109/ICRA40945.2020.9196730. [^10]: Y. Wang, Y. Ng, I. Sa, A. Parra, C. Rodriguez-Opazo, T. Lin, and H. Li, "MAVIS: Multi-camera augmented visual-inertial SLAM using SE $_{2}$(3)-based exact IMU pre-integration," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), Yokohama, Japan, May 2024, pp. 1694–1700, doi: 10.1109/ICRA57147.2024.10609982. [^11]: L. T. Zhang, D. Wisth, M. Camurri, and M. Fallon, “Balancing the budget: Feature selection and tracking for multi-camera visual-inertial odometry,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 1182–1189, Apr. 2022, doi: 10.1109/LRA.2021.3137910. [^12]: A. Fontan, L. Oliva, J. Civera, and R. Triebel, “Model for multi-view residual covariances based on perspective deformation,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 1960–1967, Apr. 2022, doi:10.1109/LRA.2022.3142905. '13] T. Shan and P rightweight and ground-optimized "terrain," in Proc. IEFR Oct. 2018, [^14]: W. Xu, Y. Cai, D. He, J. Lin, and F. Zhang, "FAST-LIO2: Fast direct LiDAR-inertial odometry," IEEE Trans. Robot., vol. 38, no. 4, pp. 2053–2073, Aug. 2022, doi: 10.1109/TRO.2022.3141876. [^15]: W. Xu and F. Zhang, "FAST-LIO: A fast, robust LiDAR-inertial odometry package by tightly-coupled iterated Kalman filter," IEEE Robot. Autom. Lett., vol. 6, no. 2, pp. 3317–3324, Apr. 2021, doi: 10.1109/LRA.2021.3064227. [^16]: J. Lin, C. Zheng, W. Xu, and F. Zhang, “R²LIVE: A robust, real-time, LiDAR-inertial-visual tightly-coupled state estimator and mapping,” IEEE Robot. Autom. Lett., vol. 6, no. 4, pp. 7469–7476, Oct. 2021, doi: 10.1109/LRA.2021.3095515. [^17]: J. Lin and F. Zhang, "R $^{3}$LIVE: A robust, real-time, RGB-colored, LiDAR-inertial-visual tightly-coupled state estimation and mapping package," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), Philadelphia, PA, USA, May 2022, pp. 10672–10678, doi:10.1109/ICRA46639.2022.9811935. [^18]: K. Eckenhoff, P. Geneva, and G. Huang, “MIMC-VINS: A versatile and resilient multi-IMU multi-camera visual-inertial navigation system,” IEEE Trans. Robot., vol. 37, no. 5, pp. 1360–1380, Oct. 2021, doi: 10.1109/TRO.2021.3049445. [^19]: W. Zhang, S. Wang, X. Dong, R. Guo, and N. Haala, “BAMF-SLAM: Bundle adjusted multi-fisheye visual-inertial SLAM using recurrent field transforms,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), London, UK, May 2023, pp. 6232–6238, doi: 10.1109/ICRA48891.2023.10160905. [^20]: Z. Wang, K. Yang, H. Shi, P. Li, F. Gao, and K. Wang, "LF-VIO: A visual-inertial-odometry framework for large field-of-view cameras with negative plane," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2022, pp. 4423–4430. [^21]: Z. Wang, K. Yang, H. Shi, P. Li, F. Gao, J. Bai, and K. Wang, "LF-VISLAM: A SLAM framework for large field-of-view cameras with negative imaging plane on mobile agents," IEEE Trans. Autom. Sci. Eng., vol. 21, no. 4, pp. 6321–6335, Oct. 2024. [^22]: R. A. Newcombe, S. J. Lovegrove, and A. J. Davison, “DTAM: Dense tracking and mapping in real-time,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Barcelona, Spain, Nov. 2011, pp. 2320–2327, doi:10.1109/ICCV.2011.6126513. [^23]: R. Wang, M. Schworer, and D. Cremers, “Stereo DSO: Large-scale direct sparse visual odometry with stereo cameras,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Venice, Italy, Oct. 2017, pp. 3903–3911, doi:10.1109/ICCV.2017.421. [^24]: S. Urban and S. Hinz, “MultiCol-SLAM: A modular real-time multi-camera SLAM system,” arXiv preprint, Oct. 2016. [Online]. Available: [^25]: T.-M. Nguyen, S. Yuan, M. Cao, Y. Lyu, T. H. Nguyen, and L. Xie, "MILIOM: Tightly coupled multi-input lidar-inertia odometry and mapping," IEEE Robot. Autom. Lett., vol. 6, no. 3, pp. 5573–5580, Jul. 2021, doi: 10.1109/LRA.2021.3080633. [^26]: R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge, UK: Cambridge University Press, 2003. [^27]: C. Yuan, W. Xu, X. Liu, X. Hong, and F. Zhang, “Efficient and probabilistic adaptive voxel mapping for accurate online LiDAR odometry,” IEEE Robot. Autom. Lett., vol. 7, no. 3, pp. 8518–8525, Jul. 2022, doi: 10.1109/LRA.2022.3187250. [^28]: L. Zhang, M. Helmberger, L. F. T. Fu, D. Wisth, M. Camurri, D. Scaramuzza, and M. Fallon, “Hilti Oxford dataset: A millimetre accurate benchmark for simultaneous localization and mapping,” IEEE Robot. Autom. Lett., vol. 8, no. 1, pp. 408–415, Jan. 2023, doi: 10.1109/LRA.2022.3226077. [^29] L. Zhang, M. Camurri, D. Wisth, and M. Fallon, “Multi-Camera LiDAR Inertial Extension to the Newer College Dataset,” arXiv preprint, Dec. 2021. [Online]. Available: