# MFINet: Multi-view Fusion and 2D-3D Interaction Enhancement for Real-Time LiDAR Semantic Segmentation Nan Ma, Zhijie Liu, Yiheng Han $^{*}$ Beijing University of Technology, Beijing, China , , ## Abstract LiDAR semantic segmentation is a key task in advanced autonomous driving systems. Projection-based methods exhibit real-time potential due to their efficiency, but suffer from inevitable 3D information loss and rely on time-consuming post-processing, limiting overall performance. To address this, we propose MFINet, a real-time semantic segmentation network based on multi-view fusion and 2D-3D interaction enhancement. It adopts a three-branch architecture that integrates 3D Point View (3D-PV), 2D Bird's Eye View (2D-BEV) and 2D Range View (2D-RV) to make full use of 2D and 3D representation. From 3D to 2D, we design a 3D Point Feature Projector (3DPFP), which injects 3D features into the 2D BEV and RV pseudo-images to retain effective 3D information. From 2D to 3D, a Feature Enhancement (FE) module is designed to leverage the advantages of 2D information in extracting geometric and semantic features. We also introduce a 2D-3D Fusion Head (FH) to aggregate point features from multiple views. Besides, we incorporate a MultiScale Dilated Attention (MSDA) module with a sliding window strategy to enhance feature discrimination. Extensive experiments on the SemanticKITTI and NuScenes benchmarks demonstrate that MFINet outperforms existing methods on the SemanticKITTI, NuScenes val set and achieves competitive results on the NuScenes test set. Code — ## Introduction In autonomous driving and robotic perception systems, LiDAR has become a crucial sensor for 3D environmental perception due to its ability to provide high-precision, structured three-dimensional spatial information. Compared to 2D images, point cloud data offers more accurate representation of positions and distances, enabling more precise characterization of object shapes and spatial relationships. Among these, point cloud semantic segmentation aims to assign precise semantic labels to each point, serving as a key step for environment modeling and scene understanding. In recent years, deep learning has driven rapid advances in point cloud semantic segmentation. Depending on the form ![Image](images/b68f72e75a5aa72ef409eabed1f8307e2fdfd0507e1d145576b36d2172605329.jpg)
Figure 1: LIDAR semantic segmentation accuracy vs speed on SemanticKITTI val set.
of data representation, existing methods can be broadly categorized into three types: point-based (Zhao et al. 2021; Hui et al. 2023; Han et al. 2024), voxel-based (Cheng et al. 2021; Yan et al. 2021; Li, Dai, and Ding 2022; Ibrahim et al. 2023), and projection-based (Zhang et al. 2020; Cortinhal, Tzelepis, and Erdal Aksoy 2020; Zhao, Bai, and Huang 2021; Li et al. 2022). Point-based methods operate directly on raw point clouds, preserving 3D structure to the greatest extent. However, their difficulty in efficiently modeling irregular data leads to high computational costs, making real-time inference challenging. Voxel-based methods discretize point clouds into regular grids to improve computational efficiency but face trade-offs between accuracy and memory consumption. In contrast, projection-based methods map 3D point clouds into 2D pseudo-images, allowing efficient modeling using mature 2D CNN architectures. These methods offer a good balance between real-time performance and deployment flexibility, making them a mainstream choice for lightweight applications. However, the projection process inherently involves dimensionality reduction, inevitably causing 3D information loss. For example, the 2D Range View (2D-RV) has advantages in preserving depth information but loses part of the spatial structural information. --- 2D Bird's Eye View (2D-BEV) provides a clear planar geometric layout and is well-suited for modeling road topology, but it fails to retain feature information along the vertical direction. Additionally, due to the sparsity of LiDAR data, the resulting pseudo-images contain a lot of empty pixels, which can introduce noise during feature learning and degrade the model's accuracy and stability. Motivated by the above findings, We propose MFINet, a real-time semantic segmentation network based on multiview fusion and 2D-3D interactive enhancement, which adopts a three-branch architecture comprising representations of 3D Point View (3D-PV), 2D Bird's Eye View (2D-BEV) and Range View (2D-RV). This design fully leverages the complementary information across different views to alleviate the loss of 3D information caused by the projection process. From 3D to 2D, we design a 3D Point Feature Projector (3DPFP) to inject 3D features into the 2D-BEV and 2D-RV pseudo-images, effectively enhancing geometric fidelity during the projection process. From 2D to 3D, a Feature Enhancement (FE) module is employed to leverage the advantages of 2D information in extracting geometric and semantic features. We also introduce a 2D-3D Fusion Head (FH) to aggregate point features from multiple views and semantic levels to enable end-to-end and real-time semantic prediction. In addition, we incorporate Multi-Scale Dilated Attention (MSDA) module into the shallow layers of the 2D-BEV and 2D-RV encoder branches. This module leverages a sliding window strategy to efficiently model sparse features and enhance feature discrimination. To ensure real-time performance, the key of our pipeline is to combine a brief 3D-PV branch and optimal 2D branches. The main contributions of this paper are summarized as follows: - We propose MFINet, a real-time semantic segmentation network based on multi-view fusion and 2D-3D interactive enhancement, which adopts a three-branch architecture comprising representations of 3D Point View (3DPV), 2D Bird's Eye View (2D-BEV) and Range View (2D-RV). - We design a novel framework incorporating a 3D Point Feature Projector (3DPFP), Feature Enhancement module (FE), 2D-3D Fusion Head (FH), and Multi-Scale Dilated Attention (MSDA), which significantly enhances the performance of LiDAR semantic segmentation through 2D-3D interactive feature enhancement. Meanwhile, the 3D-PV branch maintains maximal simplicity, thereby ensuring real-time efficiency. - We conducted extensive experiments on the SemanticKITTI and NuScenes benchmarks, demonstrating that MFINet outperforms existing methods on the SemanticKITTI and NuScenes validation sets and achieves competitive performance on the NuScenes test set with faster inference speed. ## Related Works Point-based methods take raw point clouds as input, offering excellent geometric fidelity. Representative works include PointNet (Qi et al. 2017a), which introduces a shared MLP and global max pooling for point-wise encoding; PointNet++ (Qi et al. 2017b), which adopts a hierarchical structure to capture local and multi-scale contextual features; KPConv (Thomas et al. 2019), which performs continuous deformable convolution in Euclidean space; Point Transformer (Zhao et al. 2021), which leverages self-attention to model long-range dependencies; PACConv (Xu et al. 2021), which dynamically generates convolution kernels based on point-wise positional relationships to handle irregular distributions; SuperLiDAR (Hui et al. 2023), which learns “superpoint” representations via Euclidean distance-based aggregation; and PCB-RandNet (Han et al. 2024) introduces a cylindrical partitioning strategy in the polar coordinate system for balanced sampling across distance ranges, enhancing distant object recognition and improving segmentation accuracy with efficient computation. Voxel-based methods discretize point clouds into regular grids, enabling efficient modeling with 3D convolutional networks. Cylinder3D (Zhu et al. 2021) adopts cylindrical partitioning and asymmetric 3D convolutions to balance point distribution and enhance structure representation, while introducing a point-wise refinement module to mitigate semantic loss from voxel label encoding. PVKD (Hou et al. 2022) combines point-voxel dual-branch outputs with knowledge distillation and leverages supervoxel partitioning and difficulty-aware sampling to improve sparsity handling and semantic supervision. SphereFormer (Lai et al. 2023) proposes a radial-window self-attention mechanism that divides 3D space into elongated regions, facilitating fusion between sparse distant and dense nearby points. 3D-ARSS (Wang et al. 2023) integrates spatial and channel attention modules and adopts sparse tensors to reduce memory and computation, enabling plug-and-play structural enhancement. SFPNet (Wang et al. 2024) presents a voxel-based framework that builds a sparse 3D grid via sparse voxelization and adopts an SSCN-based encoder-decoder to balance receptive field and boundary detail. Projection-based methods map sparse 3D point clouds into dense 2D pseudo-images, enabling efficient modeling with mature 2D semantic segmentation architectures and offering fast deployment and inference. SqueezeSeg (Wu et al. 2018) pioneered spherical projection with fully convolutional networks for per-pixel prediction. Subsequent works address geometric distortion and information loss: GFNet (Qiu, Yu, and Tao 2022) introduces a geometric flow module for alignment and bidirectional information fusion between views; CENet (Cheng, Han, and Xiao 2022) employs large kernels and auxiliary supervision to improve nonlinear modeling and efficiency. Recently, Transformer-based architectures have been increasingly adopted: RangeFormer (Kong et al. 2023) uses a Transformer framework to capture global context in RV images, addressing issues such as semantic inconsistency and shape distortion from many-to-one projection; TransRVNet (Cheng, Han, and Xiao 2023) combines Transformer and CNN to enhance multi-scale fusion; RangeViT (Ando et al. 2023) applies Vision Transformer to further boost performance. WaffleIron (Puy, Boulch, and Marlet 2023) proposes a lightweight framework based on MLPs and dense 2D convolutions. --- ![Image](images/066eaac4cc0a3693c8b4e31e1e7ee70af838013c4f8c639269e28077c935e518.jpg)
Figure 2: Overview of MFINet, which employs a three-branch architecture based on 3D Point View (3D-PV), 2D Bird's Eye View (2D-BEV), and Range View (2D-RV). Its core components include the 3D Point Feature Projector (3DPFP), Multi-Scale Dilated Attention (MSDA), 2D-3D Feature Enhancement (FE), and 2D-3D Fusion Head (FH).
## Methodology The overall architecture of MFINet is shown in Fig. 2. The 3D Point Feature Projector (3DPFP) extracts 3D information and injects it into the 2D-RV and 2D-BEV branches by projection to construct pseudo-images. In the shallow layers of the 2D branches, the Multi-Scale Dilated Attention (MSDA) is used to efficiently model sparse features and enhance discrimination. During decoding, the 2D-3D Feature Enhancement (FE) is used to extract geometric and semantic features. The 2D-3D Fusion Head (FH) aggregates multiview point-level features for semantic prediction. Details are described in the following subsections. ## 2 D Projection 2D-RV Image. Spherical projection transforms the unstructured 3D point cloud from the cartesian coordinate system $(x, y, z)$ into a structured 2D image coordinate system $(u, v)$ to generate a pseudo-image, referred to as the RV. Specifically, each point is projected based on its azimuth angle $\theta = \arctan(y/x)$, elevation angle $\phi = \arctan\left(z/\sqrt{x^2 + y^2 + z^2}\right)$, and depth $d = \sqrt{x^2 + y^2 + z^2}$, as follows: $$ \begin{pmatrix}u\\ v\end{pmatrix}=\begin{pmatrix}\frac{1}{2}\left(1-\frac{\theta}{\pi}\right)W_{\mathrm{rv}}\\ \left(1-\frac{\phi+f_{\mathrm{up}}}{f_{\mathrm{up}}+f_{\mathrm{down}}}\right)H_{\mathrm{rv}}\end{pmatrix} $$ where $W_{rv}$ and $H_{rv}$ represent the width and height of the RV pseudo-image, and $f_{up}$ and $f_{down}$ represent the upper and lower field of view angles of the LiDAR. 2D-BEV Image. The BEV projection maps the unstructured 3D point cloud onto the horizontal plane to generate a pseudo-image from the BEV perspective. Each point's plane coordinates $(x, y)$ are mapped to pixel positions $(u, v)$ in the BEV image, as calculated below: $$ (u,v)=\left(\frac{x-x_{\min}}{x_{\max}-x_{\min}}\cdot W_{\mathrm{b e v}},\frac{y-y_{\min}}{y_{\max}-y_{\min}}\cdot H_{\mathrm{b e v}}\right) $$ where $W_{bev}$ and $H_{bev}$ represent the width and height of the BEV image, and $(x_{\min}, x_{\max})$ and $(y_{\min}, y_{\max})$ represent the range of the point cloud in the x and y directions. ## 3 D Point Feature Projector (3DPFP) Accurate semantic prediction relies on the preservation of fine-grained 3D details during the projection process. However, as a dimensionality reduction operation, projection inevitably leads to 3D spatial information loss. Furthermore, the unordered nature of point clouds leads to degradation of geometric information in projection. To alleviate this issue, we design a 3DPFP that extracts 3D features before projection, enhancing the geometric fidelity of the pseudo-images in both 2D-BEV and 2D-RV. Specifically, given a set of 3D points $P = \{p_i\}_{i=1}^N$, where each $p_i \in \mathbb{R}^d$ denotes the 3D attributes (e.g., x, y, z, r, d), we apply multilayer perceptron (MLP) to obtain point-wise --- 3D features $f_i$. Since multiple 3D features may be projected to the same 2D position, we apply max pooling across all 3D features sharing the same pixel to obtain a unique descriptor per location. Then, based on the spatial locations $(u_i^{2D-RV}, v_i^{2D-RV})$ and $(u_i^{2D-BEV}, v_i^{2D-BEV})$, the pooled features are projected onto the 2D RV and BEV planes. The per-point features are embedded as follows: $$ I_{\mathrm{2D-BEV/RV}}=\mathrm{Proj}\left(\mathrm{Pool}(\mathrm{MLP}_{3}(P))\right) $$ where Proj( $\cdot$) denotes spherical or BEV projection. The resulting pseudo-images enriched with 3D features are used as inputs to the 2D-BEV and 2D-RV branches. ## Multi-Scale Dilated Attention (MSDA) Convolutional neural networks excel at capturing local features, but their limited receptive fields restrict multi-scale global modeling. Traditional vision Transformers (Dosovitskiy et al. 2020) excel at modeling long-range dependencies between arbitrary image patches. However, for shallow 2D pseudo-image feature maps that exhibit sparsity and locality, the global attention mechanism not only incurs quadratic computational overhead, but also introduces redundant dependencies and noise, thereby undermining the model's efficiency and representation capability. To handle these issues, inspired by (Jiao et al. 2023), we introduce an MSDA module into the shallow encoders of both the 2D-BEV and 2D-RV branches, while deep stages adopt standard Multi-Head Self-Attention (MHSA). As shown in Fig. 3, MSDA enhances noise feature discrimination and efficiently models semantic contextual dependencies through a multi-scale sliding window strategy, achieving a good balance between representational power and computational efficiency. The process is as follows: Given an input feature map $\mathbf{X} \in \mathbb{R}^{H \times W \times C}$, we obtain the corresponding queries, keys, and values by linear projection: $$ \mathbf{Q}=W_{Q}\mathbf{X},\quad\mathbf{K}=W_{K}\mathbf{X},\quad\mathbf{V}=W_{V}\mathbf{X} $$ denote the projection matrices. After that, we divide the channels of the feature map to n subspaces and perform Sliding Window Dilated Attention (SWDA) in different subspaces with different dilation rates. For each subspace i, we define the sparse receptive field centered at position p with dilation $r_i$ and kernel size $k \times k$ as: $$ \mathcal{N}_{i}(p)=\{j\in\Omega\mid j\in\mathbf{W i n d o w}(p,r_{i},k)\} $$ where $\Omega$ denotes valid positions in the feature map. For each subspace i, we compute the attention output using SWDA mechanism: $$ \mathbf{S}_{i}(p)=\sum_{j\in\mathcal{N}_{i}(p)}\operatorname{Softmax}_{j}\left(\frac{\mathbf{Q}_{i}(p)\cdot\mathbf{K}_{i}(j)}{\sqrt{d}}\right)\mathbf{V}_{i}(j) $$ Finally, outputs from all subspaces are concatenated and linearly projected back to the original channel dimension: $$ \mathbf{Z}=Concat\left(\mathbf{S}_{1}(\mathbf{X}_{1}),\ldots,\mathbf{S}_{n}(\mathbf{X}_{n})\right) $$ ![Image](images/273000146be0c8757aa82f7a5529cb7cac10544d68d57bb23c01a274f9ea5d03.jpg)
Figure 3: Multi-Scale Dilated Attention(MSDA).By default, we use a $3 \times 3$ kernel size with dilation rates r = 1, 2 and 3, and the sizes of attended receptive fields in different heads are $3 \times 3$, $5 \times 5$ and $7 \times 7$.
$$ \mathrm{MSDA}(\mathbf{X})=\mathrm{Linear}(\mathbf{Z}) $$ where Concat( $\cdot$) concatenates attention outputs along the channel dimension to form $\mathbf{Z}$; Linear( $\cdot$) is a learnable projection mapping $\mathbf{Z}$ back to the original channel dimension. ## 2 D-3D Feature Enhancement (FE) To fully exploit 2D information for geometric and semantic feature extraction, we introduce FE into the decoders of the 2D-RV and 2D-BEV branches. The FE jointly optimizes image-level semantics, boundary details, and point-level features. As shown in Fig. 4, the FE is activated only during the training phase. In the decoders of the 2D-RV and 2D-BEV branches, the intermediate feature maps $\{F^i\}_{i=1}^N$ from multiple scales are first upsampled to match the original input resolution and then fed in parallel into the FE. The detailed procedure is as follows: First, a segmentation head is applied to each upsampled feature map $\tilde{F}_{up}^i$ to generate a semantic segmentation map $\tilde{S}_i$. Then, based on $\tilde{S}_i$, multi-scale boundary detail maps are extracted using $3 \times 3$ Laplacian convolutions with 1, 2, 4 strides. These detail maps are upsampled to the original resolution and fused via a learnable $1 \times 1$ convolution. A threshold of 0.1 is applied to binarize the fused map, resulting in a binary detail map $\tilde{P}_i$ that captures both boundary and corner information. Meanwhile, each feature map $\tilde{F}_{up}^i$ is back-projected into the 3D space to obtain the corresponding point-wise semantic prediction $\tilde{P}_i$. Finally, we compute the losses for the 2D semantic segmentation, boundary detail map, and 3D point-wise semantic prediction, and formulate the overall loss as: $$ \mathcal{L}_{i}^{\mathrm{FE}}=\lambda_{1}\mathcal{L}_{i}^{\mathrm{img}}+\lambda_{2}\mathcal{L}_{i}^{\mathrm{bdry}}+\lambda_{3}\mathcal{L}_{i}^{\mathrm{point}} $$ --- ![Image](images/80ba3253dd4de84b9a6d4f7a30537c91ea4ac3a9279e65a7003e59e29f8f5da8.jpg)
Figure 4: 2D-3D Feature Enhancement (FE). Achieves joint enhancement of image-level, boundary-level, and point-level features.
where $\mathcal{L}_{i}^{\mathrm{img}}$ denotes the 2D segmentation loss combining weighted cross-entropy (WCE) and Lovasz-Softmax. $\mathcal{L}_{i}^{\mathrm{bdry}}$ is a binary cross-entropy (BCE) loss to guide the network to learn details of features such as boundaries and corners. $\mathcal{L}_{i}^{\mathrm{point}}$ denotes the point-wise loss, also based on WCE. The weights are set as $\lambda_{1}=0.4$, $\lambda_{2}=0.2$, and $\lambda_{3}=0.4$. ## 2 D-3D Fusion Head (FH) We observe that fusing the final outputs of the 3D-PV, 2D-BEV, and 2D-RV branches yields satisfactory results but is not optimal. Features from different semantic levels are complementary: shallow features emphasize 3D structures, while deeper features are richer in semantic context. To exploit these multi-level features effectively, we design an FH to integrate representations from the 3D-PV, 2D-BEV, and 2D-RV branches at different semantic levels. Specifically, we first back-project the multi-scale 2D feature maps $F_{2D-BEV}^{l}$ from the decoders of the 2D-BEV and 2D-RV branches into 3D space and concatenate them. Then, the fused and compressed features $F_{2D-BEV}^{point}$ are integrated with the 3D features $F_{point}$ extracted by the 3DPFP. The overall operation can be formulated as: $$ F_{2\mathrm{D-BEV/RV}}^{\mathrm{point}}=\mathrm{MLP}\left(\mathrm{Concat}\left(\prod_{l=1}^{3}F_{2\mathrm{D-BEV/RV}}^{l}\right)\right) $$ $$ F_{\mathrm{f u s e d}}=\mathrm{C o n c a t}\left(F_{\mathrm{2D-B E V}}^{\mathrm{p o i n t}},F_{\mathrm{2D-R V}}^{\mathrm{p o i n t}},F_{\mathrm{p o i n t}}\right) $$ where l indexes the feature maps from three different levels of the decoder, $\Pi(\cdot)$ denotes the back-projection, and $F_{\mathrm{fused}}$ is used to generate the final semantic scores with a linear head for the point over the entire point cloud. The above fusion process fully exploits the complementarity among features from different views and semantic levels. The 3D-PV, 2D-BEV, and 2D-RV branches provide diverse spatial attentions from distinct perspectives, yielding a more comprehensive semantic representation. Hierarchical fusion progressively enhances features—from geometric details to semantic abstraction, and from local to global understanding—thereby improving prediction accuracy. ## Loss Function The overall loss function consists of a main loss and an auxiliary loss, formulated as: $$ \mathcal{L}_{\mathrm{t o t a l}}=\mathcal{L}_{\mathrm{m a i n}}+\mathcal{L}_{\mathrm{a u x}} $$ where $\mathcal{L}_{\text{main}}$ is the point-wise semantic prediction loss from the 2D-3D FH, computed using cross-entropy, and $\mathcal{L}_{\text{aux}}$ is the auxiliary loss from the 2D-3D FE.
Methodsmean-IoUFPS (Hz)carbicyclemotorcycletruckother-vehiclepersonbicyclistmotorcyclistroadparkingsidewalkother-groundbuildingfencevegetationtrunkterrainpoletraffic-sign
RangeNet++52.81291.025.047.140.725.545.262.90.093.846.581.90.285.854.284.252.972.753.240.0
PolarNet53.61691.530.738.846.424.054.162.20.092.447.178.01.889.145.585.459.672.358.142.2
SalsaNext59.42490.544.649.686.354.674.081.40.093.440.669.10.084.653.083.664.364.254.439.8
TransRVNet58.12694.544.248.181.734.849.873.70.094.848.281.30.783.251.685.459.073.262.737.2
PCB-RandNet56.5-94.56.835.576.737.648.379.00.093.442.480.81.189.056.987.668.474.461.040.8
SPVNAS61.93.296.220.160.479.759.065.380.60.093.346.980.30.090.662.388.166.174.663.948.3
Cylinder3D63.26.296.541.665.887.755.971.789.30.093.537.178.31.188.449.287.966.872.364.952.4
MinkowskiNet63.13.496.820.463.685.563.068.684.10.093.547.381.10.090.661.788.066.574.164.649.5
GFNet63.01094.249.763.274.932.169.383.20.095.753.883.80.291.262.988.566.176.264.148.3
AMVNet65.211.895.648.865.488.754.870.886.20.095.553.983.20.390.962.187.966.874.264.749.3
PC-BEV66.4-94.367.671.475.467.577.481.30.195.239.783.00.186.349.785.170.268.270.264.2
MFINet(Ours)68.128.196.956.079.092.468.075.594.40.093.649.681.30.091.063.889.164.377.264.749.6
Table 1: Quantitative results on the SemanticKITTI val set, with the best and second-best bolded and underlined.
---
Methodsmean-IoUBarrierBicycleBusCarConstructionMotorcyclePedestrianTraffic-coneTrailerTruckDriveableOther-flatSidewalkTerrainManmadeVegetation
PolarNet71.074.728.285.390.935.177.571.358.857.476.196.571.174.774.087.385.7
Salsanext72.274.834.185.988.442.272.472.263.161.376.596.070.871.271.586.784.4
PCSCNet72.073.342.287.886.144.982.276.162.949.349.395.266.969.572.383.782.5
Cylinder3D76.176.840.391.393.851.378.078.964.962.184.496.871.676.475.490.587.4
SVASeg74.773.144.588.486.648.280.577.765.657.582.196.570.574.774.687.386.9
PVKD76.076.240.090.294.050.977.478.864.762.084.196.671.476.476.390.386.9
AMVNet76.179.832.482.286.462.581.975.372.383.565.197.467.078.874.690.887.9
RangeViT75.275.540.788.390.149.379.377.266.365.280.096.471.473.873.889.987.2
RPVNet77.678.243.492.793.249.085.780.566.066.984.096.973.575.976.090.688.9
PRNet78.078.040.992.593.454.185.480.663.269.884.797.375.177.776.191.089.3
RangeFormer78.178.045.294.092.958.783.977.969.163.785.696.774.575.175.389.187.5
PC-BEV78.878.246.392.593.455.087.181.065.469.285.797.176.877.076.390.688.5
SDSeg3D78.778.252.894.593.154.588.182.269.467.386.696.474.575.275.387.184.1
WaffleIron79.179.853.894.387.649.689.183.870.672.784.997.175.876.575.987.886.3
SphereFormer79.578.746.795.293.754.088.981.168.074.286.297.274.376.375.891.489.7
SFPNet80.178.849.795.393.563.186.482.968.672.886.797.074.776.075.391.289.5
MFINet(Ours)80.379.549.293.993.958.687.082.568.979.885.997.275.077.576.091.289.3
Table 2: Quantitative results on the NuScenes val set, with the best and second-best bolded and underlined.
## Experiment ## Dataset SemanticKITTI (Behley et al. 2019) is a large-scale LiDAR point cloud dataset from the University of Bonn, Germany, comprising over 43,000 scans with $360^{\circ}$ coverage and 19 semantic categories. Sequences 00–10 are for training (with 08 as validation), and 11–21 for testing. NuScenes (Caesar et al. 2020) is a LiDAR semantic segmentation dataset comprising 1,000 scenes collected from diverse urban areas in Boston and Singapore. The dataset contains 28,130 training samples, 6,019 validation samples, and 6,008 test samples. ## Implementation Details MFINet is implemented using the PyTorch framework. The model is trained using an NVIDIA RTX 4090 GPU, and for fairness, evaluated on a single NVIDIA RTX 2080Ti GPU. The model is trained with the Adam optimizer using an initial learning rate of 0.006 and a momentum of 0.9. A OneCycle learning rate policy is adopted to dynamically adjust the learning rate. The input pseudo-image resolutions are set to $64 \times 2048$ for the 2D-RV and $600 \times 600$ for the 2D-BEV. ## Results and Discussion Table 1 presents the experimental results on the SemanticKITTI val set. MFINet achieves the highest mIoU of 68.1% while maintaining a real-time inference speed of 28.1 FPS. This validates the effectiveness of our proposed multi-view fusion, 2D-3D interactive enhancement strategy, and lightweight architecture in balancing accuracy and efficiency. Table 2 reports the performance comparison in the NuScenes val set. MFINet achieves the highest mIoU of 80.3% among all methods, outperforming AMVNet (Liong et al. 2020), RangeFormer (Kong et al. 2023), and PC-BEV (Qiu et al. 2025) by 4.2%, 2.2%, and 1.5%, further validating its robustness and broad applicability. Table 3 presents the results on the NuScenes test set. MFINet achieves an mIoU of 79.7%, surpassing all compared methods except RangeFormer (Kong et al. 2023). It is worth noting that all the compared methods mentioned above are LiDAR-only approaches. As shown in Table 4, MFINet achieves competitive performance on both SemanticKITTI val and NuScenes val/test using only LiDAR input. While multi-modal methods LCPs (Zhang et al. 2023), EPMF (Tan et al. 2024) and 2DPASS (Yan et al. 2022) rely on additional RGB information, our model attains comparable or better mIoU with significantly fewer parameters and FLOPs. Compared with state-of-the-art LiDAR-only model LSK3DNet (Feng et al. 2024), MFINet further reduces computational cost while maintaining strong accuracy. Meanwhile, we conducted FPS tests on the Nvidia Jetson TX2, where LSK3DNet, LSK3DNetS (kernel size is 7), and MFINet achieved 1.2, 3.8, and 5.1 FPS. These results indicate that despite adopting a simple projection-based design, our method exhibits strong generalization capability and practical deployment value. ## Ablation Studies Table 5 presents an ablation study on the three branches of MFINet: 3D Point View (3D-PV), 2D Range View (2D-RV), and Bird's Eye View (2D-BEV), aiming to verify the effectiveness of the multi-view fusion architecture. We evaluated two dual-branch combinations, 3D-PV + 2D-RV and 3D-PV + 2D-BEV, and compared them against the full three-branch configuration. The results show that the three-branch fusion significantly outperforms all dual-branch combinations in ---
Methodsmean-IoUBarrierBicycleBusCarConstructionMotorcyclePedestrianTraffic-coneTrailerTruckDriveableOther-flatSidewalkTerrainManmadeVegetation
PolarNet69.472.216.877.086.551.169.764.854.169.763.496.667.177.772.187.184.4
JS3C-Net73.680.126.287.884.555.272.671.366.376.871.296.864.576.974.187.586.1
AMVNet76.179.832.482.286.462.581.975.372.383.565.197.467.078.874.690.887.9
3D-ARSS75.883.641.590.253.170.176.774.961.081.580.995.869.477.178.589.288.4
GFNet76.181.131.676.090.560.280.775.371.882.565.197.867.080.476.291.888.9
SAT3D76.774.343.991.392.853.077.879.764.863.385.796.573.474.777.389.988.0
Cylinder3D77.282.829.884.389.463.079.377.273.484.669.197.770.280.375.590.487.6
SPVNAS77.480.030.091.990.864.779.075.670.981.074.697.469.280.076.189.387.1
SphereFormer78.181.539.793.487.566.475.777.270.685.673.697.664.879.875.092.289.0
(AF)-S3Net78.378.952.289.984.277.474.377.372.083.973.897.166.577.574.087.786.8
RangeFormer80.185.647.491.290.970.784.777.174.183.272.697.570.779.275.491.388.9
MFINet(Ours)79.782.244.589.791.669.182.279.275.186.673.297.967.080.876.092.089.0
Table 3: Quantitative results on the NuScenes test set, with the best and second-best bolded and underlined.
MethodsModalityParamsFLOPsSK-VNS-VNS-T
LCPsL+C77.7/63.280.578.9
EPMFL+C34.2418.065.980.679.2
2DPASSL+C45.6/69.379.480.8
LSK3DNetL28.8763.670.280.179.6
LSK3DNet-SL13.6249.468.178.376.9
MFINet(Ours)L10.1151.968.180.379.7
Table 4: Comparison on SemanticKITTI val and NuScenes val/test with Params, FLOPs, and mIoU. L+C indicates multi-modal methods. L indicates LiDAR-only methods.
Row3D-PV2D-RV2D-BEVFPSParamsFLOPsmIoU
1✓✓✗46.86.864.964.1
2✓✗✓40.96.888.262.6
3✓✓✓28.110.1151.968.1
Table 5: Ablation study on different view branch combinations on the SemanticKITTI validation set.
segmentation accuracy. This demonstrates the complementary nature of the three views in capturing 3D structure, contextual semantics, and spatial distribution, all of which play a crucial role in improving segmentation performance. We further conduct ablation studies on the key components of MFINet to evaluate their individual contributions to overall model performance, as shown in Table 6. The model without 3DPFP, MSDA, FE and FH serves as the baseline upon which each component is introduced. Experimental results show that incorporating the 3DPFP brings a 0.6% improvement in mIoU, indicating that 3D feature injection enhances the geometric representation of pseudo-images. Adding the MSDA yields an additional 0.9% gain, demonstrating its effectiveness in modeling sparse features. Introducing the FE further improves mIoU by 1.5%, validating the effectiveness of jointly optimizing semantic information at the image, boundary, and point levels. The in-
Row3DPFPMSDAFEFHFPSParamsFLOPsmIoU
1✗✗✗✗42.34.973.763.5
2✓✗✗✗32.98.4126.364.1
3✓✓✗✗30.99.4141.365.0
4✓✓✓✗29.09.9148.866.5
5✓✓✓✓28.110.1151.968.1
6✗✓✓✓34.36.6114.566.1
7✓✗✓✓30.89.1138.066.7
Table 6: Ablation study on the effectiveness of core components in MFINet.
clusion of the FH brings an additional 1.6% gain, further enhancing the integration of multi-level semantic features. Compared with the full MFINet, removing only the 3DPFP results in a 2% drop in accuracy, while removing only the MSDA leads to a 1.4% decrease, indicating that both modules not only contribute positively to the overall performance of the model but also enhance the effectiveness of the other components. These results confirm the effectiveness of each module and the scalability of MFINet's modular design in enhancing segmentation accuracy. ## Conclusion This paper proposes MFINet, a real-time semantic segmentation network based on multi-view fusion and 2D-3D interactive enhancement. We design a novel framework incorporating a 3D Point Feature Projector (3DPFP), Feature Enhancement module (FE), 2D-3D Fusion Head (FH), and Multi-Scale Dilated Attention (MSDA), which significantly enhances the performance of LiDAR semantic segmentation through 2D-3D interactive feature enhancement. Meanwhile, the 3D-PV branch maintains maximal simplicity, thereby ensuring real-time efficiency. Extensive experiments on SemanticKITTI and NuScenes demonstrate that MFINet surpasses state-of-the-art methods. Moreover, ablation studies prove the effectiveness of each module. --- ## Acknowledgments This work was supported by the National Key R&D Program of China (No. 2023YFF0615800), the National Natural Science Foundation of China (Nos. 62371013, , or N_HKU705/24, 62503024), the Beijing Natural Science Foundation (No. L247007), and in part by the Beijing Postdoctoral Research Foundation (No. 2024-ZZ-23). ## References Ando, A.; Gidaris, S.; Bursuc, A.; Puy, G.; Boulch, A.; and Marlet, R. 2023. Rangevit: Towards vision transformers for 3d semantic segmentation in autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5240–5250. Behley, J.; Garbade, M.; Milioto, A.; Quenzel, J.; Behnke, S.; Stachniss, C.; and Gall, J. 2019. Semantickitti: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE/CVF international conference on computer vision, 9297–9307. Caesar, H.; Bankiti, V.; Lang, A. H.; Vora, S.; Liong, V. E.; Xu, Q.; Krishnan, A.; Pan, Y.; Baldan, G.; and Beijbom, O. 2020. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11621–11631. Cheng, H.-X.; Han, X.-F.; and Xiao, G.-Q. 2022. Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving. In 2022 IEEE international conference on multimedia and expo (ICME), 01–06. IEEE. Cheng, H.-X.; Han, X.-F.; and Xiao, G.-Q. 2023. TransRVNet: LiDAR semantic segmentation with transformer. IEEE Transactions on Intelligent Transportation Systems, 24(6): 5895–5907. Cheng, R.; Razani, R.; Taghavi, E.; Li, E.; and Liu, B. 2021. 2-s3net: Attentive feature fusion with adaptive feature selection for sparse semantic segmentation network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12547–12556. Cortinhal, T.; Tzelepis, G.; and Erdal Aksoy, E. 2020. Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds. In International Symposium on Visual Computing, 207–222. Springer. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth $16 \times 16$ words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. Feng, T.; Wang, W.; Ma, F.; and Yang, Y. 2024. Lsk3dnet: Towards effective and efficient 3d perception with large sparse kernels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14916–14927. Han, X.-F.; Cheng, H.; Jiang, H.; He, D.; and Xiao, G. 2024. Pcb-randnet: Rethinking random sampling for lidar semantic segmentation in autonomous driving scene. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 4435–4441. IEEE. Hou, Y.; Zhu, X.; Ma, Y.; Loy, C. C.; and Li, Y. 2022. Point-to-voxel knowledge distillation for lidar semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8479–8488. Hui, L.; Tang, L.; Dai, Y.; Xie, J.; and Yang, J. 2023. Efficient lidar point cloud oversegmentation network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 18003–18012. Ibrahim, M.; Akhtar, N.; Anwar, S.; and Mian, A. 2023. SAT3D: Slot attention transformer for 3D point cloud semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 24(5): 5456–5466. Jiao, J.; Tang, Y.-M.; Lin, K.-Y.; Gao, Y.; Ma, A. J.; Wang, Y.; and Zheng, W.-S. 2023. Dilateformer: Multi-scale dilated transformer for visual recognition. IEEE transactions on multimedia, 25: 8906–8919. Kong, L.; Liu, Y.; Chen, R.; Ma, Y.; Zhu, X.; Li, Y.; Hou, Y.; Qiao, Y.; and Liu, Z. 2023. Rethinking range view representation for lidar segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 228–240. Lai, X.; Chen, Y.; Lu, F.; Liu, J.; and Jia, J. 2023. Spherical transformer for lidar-based 3d recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 17545–17555. Li, J.; Dai, H.; and Ding, Y. 2022. Self-distillation for robust LiDAR semantic segmentation in autonomous driving. In European conference on computer vision, 659–676. Springer. Li, X.; Zhang, G.; Jiang, T.; Cai, X.; and Wang, Z. 2022. PRNet: Point-Range Fusion Network for Real-Time LiDAR Semantic Segmentation. In IJCAI, 1116–1122. Liong, V. E.; Nguyen, T. N. T.; Widjaja, S.; Sharma, D.; and Chong, Z. J. 2020. Amvnet: Assertion-based multi-view fusion network for lidar semantic segmentation. arXiv preprint arXiv:2012.04934. Puy, G.; Boulch, A.; and Marlet, R. 2023. Using a waffle iron for automotive point cloud semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3379–3389. Qi, C. R.; Su, H.; Mo, K.; and Guibas, L. J. 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 652–660. Qi, C. R.; Yi, L.; Su, H.; and Guibas, L. J. 2017b. Point-net++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30. Qiu, H.; Yu, B.; and Tao, D. 2022. Gfnet: Geometric flow network for 3d point cloud semantic segmentation. arXiv preprint arXiv:2207.02605. Qiu, S.; Li, X.; Xue, X.; and Pu, J. 2025. PC-BEV: An Efficient Polar-Cartesian BEV Fusion Framework for LiDAR Semantic Segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 6612–6620. --- Tan, M.; Zhuang, Z.; Chen, S.; Li, R.; Jia, K.; Wang, Q.; and Li, Y. 2024. EPMF: Efficient perception-aware multi-sensor fusion for 3D semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12): 8258–8273. Thomas, H.; Qi, C. R.; Deschaud, J.-E.; Marcotegui, B.; Goulette, F.; and Guibas, L. J. 2019. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, 6411–6420. Wang, F.; Wu, Z.; Yang, Y.; Li, W.; Liu, Y.; and Zhuang, Y. 2023. Real-time semantic segmentation of LiDAR point clouds on edge devices for unmanned systems. IEEE Transactions on Instrumentation and Measurement, 72: 1–11. Wang, Y.; Zhao, W.; Cao, C.; Deng, T.; Wang, J.; and Chen, W. 2024. Sfpnet: Sparse focal point network for semantic segmentation on general lidar point clouds. In European Conference on Computer Vision, 403–421. Springer. Wu, B.; Wan, A.; Yue, X.; and Keutzer, K. 2018. Squeeze-seg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud. In 2018 IEEE international conference on robotics and automation (ICRA), 1887–1893. IEEE. Xu, M.; Ding, R.; Zhao, H.; and Qi, X. 2021. Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3173–3182. Yan, X.; Gao, J.; Li, J.; Zhang, R.; Li, Z.; Huang, R.; and Cui, S. 2021. Sparse single sweep lidar point cloud segmentation via learning contextual shape priors from scene completion. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 3101–3109. Yan, X.; Gao, J.; Zheng, C.; Zheng, C.; Zhang, R.; Cui, S.; and Li, Z. 2022. 2d pass: 2d priors assisted semantic segmentation on lidar point clouds. In European conference on computer vision, 677–695. Springer. Zhang, Y.; Zhou, Z.; David, P.; Yue, X.; Xi, Z.; Gong, B.; and Foroosh, H. 2020. Polarnet: An improved grid representation for online lidar point clouds semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9601–9610. Zhang, Z.; Zhang, Z.; Yu, Q.; Yi, R.; Xie, Y.; and Ma, L. 2023. Lidar-camera panoptic segmentation via geometry-consistent and semantic-aware alignment. In Proceedings of the IEEE/CVF international conference on computer vision, 3662–3671. Zhao, H.; Jiang, L.; Jia, J.; Torr, P. H.; and Koltun, V. 2021. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, 16259–16268. Zhao, Y.; Bai, L.; and Huang, X. 2021. Fidnet: Lidar point cloud semantic segmentation with fully interpolation decoding. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 4453–4458. IEEE. Zhu, X.; Zhou, H.; Wang, T.; Hong, F.; Ma, Y.; Li, W.; Li, H.; and Lin, D. 2021. Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9939–9948.