# FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation Xiang Xu $^{®}$, Lingdong Kong $^{®}$, Student Member, IEEE, Hui Shuai $^{®}$, and Qingshan Liu $^{®}$, Senior Member, IEEE Abstract—LiDAR segmentation has become a crucial component of advanced autonomous driving systems. Recent range-view LiDAR segmentation approaches show promise for real-time processing. However, they inevitably suffer from corrupted contextual information and rely heavily on post-processing techniques for prediction refinement. In this work, we propose FRNet, a simple yet powerful method aimed at restoring the contextual information of range image pixels using corresponding frustum LiDAR points. First, a frustum feature encoder module is used to extract per-point features within the frustum region, which preserves scene consistency and is critical for point-level predictions. Next, a frustum-point fusion module is introduced to update per-point features hierarchically, enabling each point to extract more surrounding information through the frustum features. Finally, a head fusion module is used to fuse features at different levels for final semantic predictions. Extensive experiments conducted on four popular LiDAR segmentation benchmarks under various task setups demonstrate the superiority of FRNet. Notably, FRNet achieves 73.3% and 82.5% mIoU scores on the testing sets of SemanticKITTI and nuScenes. While achieving competitive performance, FRNet operates 5 times faster than state-of-the-art approaches. Such high efficiency opens up new possibilities for more scalable LiDAR segmentation. The code has been made publicly available at . Index Terms—LiDAR semantic segmentation, autonomous driving, real-time processing, frustum-range representation. ### I. INTRODUCTION LIDAR segmentation, a crucial and indispensable component in modern autonomous driving, robotics, and other safety-critical applications, has witnessed substantial advancements recently [^1], [^2], [^3], [^4]. The principal challenge lies in devising a scalable LiDAR segmentation system that strikes a delicate balance between efficiency and accuracy, especially in resource-constrained operational scenarios [^5], [^6]. Here, “scalable” refers to the system’s ability to maintain high performance while managing increased data loads without compromising real-time processing capabilities. Existing LiDAR segmentation methods adopt various data representation perspectives, each with inherent trade-offs in terms of accuracy and computational efficiency. Point-based methods [^7], [^8], [^9], [^10], [^11], [^12] manipulate the original point cloud to preserve spatial granularity but entail computationally expensive neighborhood searches to construct local structures. This demand results in significant computational overhead and limits their capability to deal with large-scale point clouds [^10], [^13], [^14]. Sparse-voxel-based methods [^15], [^16], [^17], [^18], [^19] transform scattered LiDAR points into regular voxel grids and leverage popular sparse convolutions to extract features. However, this process also demands heavy computations, especially for applications that require high voxel resolutions [^15], [^16], [^20]. Multi-view methods [^21], [^22], [^23], [^24] extract features from various representations to enhance prediction accuracy [^25], [^26], [^27]. While this strategy can improve performance, it aggregates the computation required for all representations. Albeit these representation methods achieved satisfactory performance, their computation overhead hinders them from real-time online applications in real life [^24], [^28], [^29], [^30], [^31]. Recently, pseudo-image-based methods [^33], [^34], [^35], [^36], [^37], [^38], [^39], [^40] have appeared as a simple yet efficient intermediate representation for LiDAR processing, such as bird's-eye-view [^37], [^38], [^39] and range-view [^33], [^34], [^35], [^36], which are more computationally tractable. These representations enable the direct application of popular 2D segmentation approaches [^41], [^42], [^43] to the pseudo-images, offering a promising solution for real-time LiDAR segmentation. However, the 3D-to-2D projection inevitably introduces corrupted neighborhood contextual information, which poses limitations on the further development of pseudo-image approaches [^44]. The detailed analysis of this problem can be further summarized into two aspects as follows. Firstly, a fundamental concern with pseudo-image methods is the inadvertent exclusion of certain 3D points during the conversion to 2D projections [^45]. This leads to an incomplete representation where only the points that are projected are involved in subsequent convolutional operations. Such a scenario disregards the contextual information of the unprojected points, causing the pseudo-images to lose crucial high-fidelity 3D geometric context [^46]. Furthermore, the omitted points, which still necessitate 3D label predictions, lack associated feature information, thereby necessitating post-processing to --- infer these labels. Unfortunately, current post-processing methods struggle to incorporate local features of these omitted points effectively, capping the potential advancements in pseudo-image approaches. Secondly, the challenge with pseudo-images is exacerbated by the significant presence of empty pixels due to the inherent sparsity of LiDAR data. For instance, in the nuScenes dataset, nearly 60% of range image pixels are found to be void [^36], [^47]. This vacancy not only skews the scene representation but also compromises the quality of semantic segmentation. As the network processes deeper layers, these sparse pseudo-images can become riddled with noise, obstructing the extraction of meaningful patterns. In essence, although pseudo-image methods excel in terms of real-time processing capabilities, their reduced dimensional perspective can overlook crucial 3D details, leading to degraded scalability. Observing the above issues, in this work, we propose a Frustum-Range Network (FRNet) for scalable LiDAR segmentation, which incorporates points into the range image, achieving a superior balance between efficiency and accuracy. FRNet consists of three main components. Firstly, a Frustum Feature Encoder (FFE) is utilized to group all points with the same frustum region into corresponding range-view pixels in relation to the range image using multiple multi-layer perceptrons (MLPs). This allows for the preservation of all points and the prediction of semantic labels for these points in an end-to-end manner. Subsequently, the point features are pooled to represent the frustum region and formatted into a 2D representation, which is then subjected to traditional convolutions. Secondly, a Frustum-Point (FP) fusion module is employed to efficiently update the hierarchical features of each point during each convolutional stage. This module includes frustum-to-point fusion to update per-point features and point-to-frustum fusion to enhance frustum features. As a result, all points extract larger local features based on the frustum region. Finally, a Fusion Head (FH) module is designed to leverage features from different levels to generate individual features for each point, facilitating end-to-end prediction without the need for post-processing techniques. As shown in Fig. 1, the proposed FRNet achieves great improvement among range-view methods while still maintaining high efficiency. Data augmentation plays an important role in training robust and generalizable models. Inspired by the success of mixing strategies [^48], [^49], [^50], we introduce a fine-grained mixing method dubbed FrustumMix, which operates based on frustum units. Specifically, we randomly divide point cloud scenes into several frustum regions along inclination and azimuth directions and swap the corresponding regions from different scenes, with the neighborhood of each frustum region coming from the other scene. FrustumMix provides a straightforward and lightweight method to adapt FRNet by focusing on the frustum region. Furthermore, to address the issue of empty pixels in the 2D representation, we propose RangeInterpolation to reconstruct the semantic surface based on surrounding point information. It first transforms the LiDAR point cloud into a range image, as done in previous works [^33], [^34], [^35], [^36]. For empty pixels in the range image, we then aggregate surrounding range information within a pre-defined ![Image](images/1e5dc4ac4f3e68731dea8f349c23d384f898df7a1d1913ae9fd451f9af8bd347.jpg)
Fig. 1. A study on the scalability of state-of-the-art LiDAR segmentation models on the SemanticKITTI [^32] leaderboard. The size of the circular representation corresponds to the number of model parameters. FRNet achieves competitive performance with current state-of-the-art models while still maintaining satisfactory efficiency for real-time processing.
window using an average pooling operation to generate a new point. RangeInterpolation is able to assist in creating a more compact 2D representation with fewer empty pixels, enabling convolutions to learn more semantically coherent information. The key contributions of this work are summarized as: - We introduce a novel Frustum-Range representation for scalable LiDAR segmentation. In our framework, the point-level geometric information is integrated into the pseudo-image representation, which not only retains the complete geometric structure of the point cloud, but also takes advantage of the efficiency of the 2D network. - We propose two 3D data augmentation techniques to assist in training robust and generalizable models. FrustumMix generates more complex scenes by mixing two different scans, while RangeInterpolation reconstructs the semantic surfaces based on surrounding range information. These techniques help generate a compact 2D representation and learn more context-aware features. - Extensive experiments across four prevailing LiDAR segmentation benchmarks demonstrate our superiority. FRNet achieves 73.3% and 82.5% mIoU scores, respectively, on SemanticKITTI and nuScenes, while still maintaining promising scalability for real-time LiDAR segmentation. ### II. RELATED WORK ### A. Scalable LiDAR Segmentation Although point-view [^7], [^8], [^10], [^11], [^12], sparse-voxel-view [^15], [^16], [^17], [^18], and multi-view [^21], [^22], [^23], [^24], [^51], [^52] methods have achieved significant success in LiDAR segmentation, they often suffer from high computational costs and are unable to achieve real-time processing. Recently, various methods have been proposed to project LiDAR points into range images along inclination and azimuth directions to balance the efficiency and accuracy of segmentation [^53]. SqueezeSeg [^33] and SqueezeSegV2 [^54] --- use the lightweight model SqueezeNet [^55] to retain information. SqueezeSegV3 [^56] introduces a Spatially-Adaptive Convolution module to adjust kernel weights based on the locations of range images. RangeNet++ [^34] integrates DarkNet into SqueezeSeg [^33] and proposes an efficient KNN post-processing for segmentation. RangeViT [^35] and RangeFormer [^36] introduce transformer blocks to extract both local and global information from the range image. However, these methods only involve projected points and fail to extract contextual information for discarded points. To address this problem, WaffleIron [^11] proposes to mix point and image features with residual connections. Although achieving promising performance, it requires downsampling points based on voxel grid and space neighborhood search for point feature embedding, which hinders it from real-time processing and end-to-end predictions over the entire point cloud. In this work, we propose an efficient integration of point and image features that preserves both the 3D geometric information from raw point features and the efficiency of 2D convolutions, eliminating the need for pre- or post-processing techniques and yielding superior scalability for real-time LiDAR segmentation. ### B. LiDAR Data Augmentation Recent works have explored various data augmentation techniques for point clouds [^57], [^58], which have shown promise in improving performance for indoor scene understanding but exhibit limited generalization to outdoor scenes. Mix3D [^50] creates new training samples by combining two out-of-context scenes, which incurs higher memory costs. PolarMix [^49] proposes to mix two LiDAR scenes based on azimuth angles and crop instances from one scene to another, while LaserMix [^48] suggests exchanging labeled scans with unlabeled scans along the inclination direction for efficient semi-supervised learning. RangeFormer [^36] presents a lightweight augmentation method that operates only on projected points in range images, disregarding occluded points. In this work, we introduce augmentation techniques aimed at training a robust and scalable model for LiDAR segmentation, including FrustumMix and RangeInterpolation. ### C. LiDAR Post-Processing Prior works have proposed various post-processing techniques to reconstruct the semantic labels of occluded points based on 2D results. SqueezeSeg [^33] employs the traditional conditional random field (CRF) as a recurrent neural network (RNN). RangeNet++ [^34] introduces a fast K-Nearest-Neighbor (KNN) search to aggregate final results across the entire point cloud. FIDNet [^59] uses a Nearest Label Assignment (NLA) method for label smoothing. RangeFormer [^36] divides full point clouds into sub-clouds and infers each subset in a supervised manner. However, these methods heavily rely on 2D results and fail to extract point features over the entire point cloud in a supervised manner. In contrast, KPRNet [^60] and RangeViT [^35] incorporate a KPConv [^7] block to extract point features in an end-to-end manner. Nevertheless, this approach requires substantial computational resources to ![Image](images/0e0f0d579c1d2b08b50d842b316b68c4005505a99fe59b9750e1c6a310b21855.jpg)
Fig. 2. Pilot study on the performance degradation of post-processing in existing range-view methods [^34], [^39], [^59], [^61] on the val set of SemanticKITTI [^32]. We choose various K values as hyperparameters in KNN post-processing. Compared to their performance at 2D (i.e., the range image), a severe drop in performance occurs with different K values.
search neighborhoods in 3D space, which hinders them from real-time processing. In this work, we partition points into frustum regions and extract local features of points in a 2D manner, avoiding time-consuming neighbor searches. Based on the learned point features, FRNet predicts results over the entire point cloud in an end-to-end manner with high efficiency. ### III. FRNET: A SCALABLE LIDAR SEGMENTOR In this section, we first conduct a pilot study on the most popular range-view LiDAR segmentation methods, wherein we observe a significant impact of KNN post-processing on performance (Sec. III-A). We then elaborate on the technical details of frustum-range architecture (Sec. III-B) and frustum-range operators (Sec. III-C). The overall pipeline of the proposed FRNet framework is depicted in Fig. 3. ### A. Pilot Study LiDAR segmentation aims to assign semantic labels to each point in the point cloud [^62]. A common approach is to operate directly on the LiDAR points without any preprocessing, following the paradigm established by PointNet++ [^63]. However, point-based operations often require extensive computations and can yield sub-optimal performance due to the sparsity of outdoor scenes. To address this problem, many recent works have proposed projecting LiDAR points onto a regular intermediate representation, known as range images, based on spherical coordinates to enhance efficiency. Specifically, given a point cloud $\mathcal{P} = \{\mathbf{p}_i\}_{i=1}^N$ with N points, where $\mathbf{p}_i \in \mathbb{R}^{3+L}$ includes the coordinate $(x_i, y_i, z_i)$ and additional L-dimensional features (intensity, elongation, etc.), we map the point to its corresponding 2D position $(u_i, v_i)$ in the range image via the following transformation: $$ \left(\begin{matrix}{u_{i}}\\ {v_{i}}\\ \end{matrix}\right)=\left(\begin{matrix}{\frac{1}{2}[1-\operatorname{a r c t a n}(y_{i},x_{i})\pi^{-1}]W}\\ {[1-(\operatorname{a r c s i n}(z_{i}d_{i}^{-1})+\phi_{\mathrm{d o w n}})\phi^{-1}]H}\\ \end{matrix}\right), $$ --- ![Image](images/7453562905f78f5aba41b4d5a8af05aa6a3bd6c216f4199a32bef340711d42c9.jpg)
Fig. 3. Architecture overview. The proposed FRNet comprises three main components: 1) Frustum Feature Encoder is used to embed per-point features within the frustum region. 2) Frustum-Point (FP) Fusion Module updates per-point features hierarchically at each stage of the 2D backbone. 3) Fusion Head fuses different levels of features to predict final results.
where $d_i = \sqrt{x_i^2 + y_i^2 + z_i^2}$ is the depth of the point; $\phi = |\phi_{up}| + |\phi_{down}|$ represents the vertical field-of-view (FOV) of the sensor, with $\phi_{up}$ and $\phi_{down}$ being the inclination angles in the upward and downward directions, respectively; H and W are the height and width of the range image. However, such projection often results in the loss of occluded points and requires post-processing techniques to reconstruct the entire semantic information [^36]. We conducted a pilot study on various range-view methods, including RangeNet++ [^34], SalsaNext [^64], FIDNet [^59], and CENet [^61]. We compared their performances on the original points (i.e., after the 2D-to-3D projection) using the widely-used KNN post-processing with different nearest neighbor parameters, as well as their performance on the 2D range images. As illustrated in Fig. 2, the post-processing procedures are often unsupervised, and the choice of hyperparameters can significantly impact the final performance. To address this issue, we propose a scalable Frustum-Range Network (FRNet) capable of directly predicting semantic labels for each point while maintaining high efficiency. ### B. Frustum-Range Architecture 1) Problem Definition: Given a point cloud $\mathcal{P}$ acquired by the LiDAR sensor, the objective of FRNet is to employ a feed-forward network $\mathcal{G}$ to predict semantic labels $\widehat{\mathcal{Y}}$ for each individual point as follows: $$ \widehat{\mathcal{Y}}=\mathcal{G}(\mathcal{P},\theta), $$ where $\theta$ represents the learnable parameters within the network. As depicted in Fig. 3, FRNet consists of three key components: 1) a frustum feature encoder responsible for per-point feature extraction; 2) a frustum-point fusion module that efficiently fuses hierarchical point features and frustum features; 3) a fusion head module that integrates point features at different levels for accurate semantic prediction. 2) Operator Representation: Let $\text{Flat}(\cdot): \mathbb{R}^{N \times C} \to \mathbb{R}^{H \times W \times C}$ denotes the function that projects point features onto the frustum image plane with resolution $(H, W)$. Within each frustum region, we apply a max-pooling function to obtain the frustum features. Let $\text{MLP}(\cdot): \mathbb{R}^{N \times C} \to \mathbb{R}^{N \times D}$ represents the MLPs that take C-dimensional point features as input and output D-dimensional point features. Let $\text{Infl}(\cdot): \mathbb{R}^{H \times W \times C} \to \mathbb{R}^{N \times C}$ be the operation that back-projects frustum features to point features. Notably, points falling into the same frustum region share the same features under this operation. 3) Frustum Feature Encoder: Accurate prediction of semantic labels for each point requires the extraction of individual features. Before converting the point cloud into a 2D frustum representation, the frustum feature encoder plays a pivotal role in extracting per-point features through a series of MLPs, which is crucial for accurate label prediction. The frustum feature encoder follows three steps. ① We divide the point cloud into H and W frustum regions along the vertical and horizontal directions, respectively. According to Eq. (1), for each point $\mathbf{p}_i$ in the point cloud $\mathcal{P}$, we calculate its frustum position $(u_i, v_i)$. Points sharing the same $(u, v)$ coordinates are grouped into the same frustum region. This results in a set of M frustum regions, defined as $\mathcal{P} = \{\mathcal{P}_1, \mathcal{P}_2, \ldots, \mathcal{P}_M\}$, where $M = H \times W$ and $\mathcal{P}_i \in \mathbb{R}^{N_i \times (3+L)}$ consists of $N_i$ points within the i-th frustum region. ② To explicitly embed the frustum structure, we leverage an average pooling function within each frustum region to obtain the cluster points $\widehat{\mathcal{P}} = \{\widehat{\mathcal{P}}_1, \widehat{\mathcal{P}}_2, \ldots, \widehat{\mathcal{P}}_M\}$, where $\widehat{\mathcal{P}}_i \in \mathbb{R}^{1 \times (3+L)}$. Subsequently, we construct a frustum graph between the point cloud $\mathcal{P}$ and the cluster points $\widehat{\mathcal{P}}$ within each frustum region. The per-point features are embedded as follows: $$ \begin{array}{r}{\mathcal{F}_{p}^{0}=\mathrm{M L P}([\mathcal{P};\mathcal{P}-\widetilde{\mathcal{P}}]),}\end{array} $$ where $[\cdot, \cdot]$ denotes feature concatenation. A max-pooling function is then applied within the frustum region to yield the initialized frustum feature. $\textcircled{3}$ The frustum feature is transformed into a 2D frustum representation based on its frustum position $(u, v)$: $\mathcal{F}_f^0 = \text{Flat}(\mathcal{F}_p^0)$. Both $\mathcal{F}_p^0$ and $\mathcal{F}_f^0$ serve as inputs to our backbone for the efficient hierarchical updating of point and frustum features. --- ![Image](images/aedbd4092a8f0e60f0d3a434351c008fa8788ccff5d330868afb1f0a9f699bba.jpg)
Fig. 4. Frustum-point fusion module comprises two steps: 1) A Frustum-to-Point fusion to update per-point features. 2) A Point-to-Frustum fusion to update frustum features.
4) Frustum-Point Fusion Module: Building on prior works [^36], [^59], [^61], we utilize a 2D backbone composed of multiple convolution blocks to efficiently extract hierarchical frustum features. Each stage block is accompanied by a frustum-point fusion module, facilitating the incremental update of point and frustum features. As illustrated in Fig. 4, the frustum-point fusion module consists of two essential components: 1) a frustum-to-point fusion that enables the hierarchical update of per-point features, and 2) a point-to-frustum fusion that fuses individual features of each point within its corresponding frustum region. The i-th frustum-point fusion module takes $\mathcal{F}_p^{i-1}$ and $\widehat{\mathcal{F}}_f^{i-1}$ as inputs, where $\widehat{\mathcal{F}}_f^{i-1}$ is the output of the 2D convolution block that takes $\mathcal{F}_f^{i-1}$ as input. To efficiently extract context information for the points, we employ a frustum-to-point fusion block to integrate local features from the frustum region into the points. First, $\widetilde{\mathcal{F}}_{f}^{i-1}$ is back-projected to the point-level features according to the projection index. Subsequently, the back-projected features are concatenated with $\mathcal{F}_{p}^{i-1}$ and passed through an MLP to update the per-point features, avoiding high computational costs from neighborhood searches: $$ \mathcal{F}_{p}^{i}=\mathbf{M L P}([\mathtt{I n f l a t}(\widehat{\mathcal{F}}_{f}^{i-1});\mathcal{F}_{p}^{i-1}])~. $$ As discussed above, the 2D representation is non-compact due to the sparsity of the point cloud. The convolution block inevitably introduces noisy information for empty frustums, leading to compact feature representations. To preserve the sparsity attributes of the 2D representation, we design a point-to-frustum fusion module to fuse the sparse and dense frustum features. Specifically, the updated point features $\mathcal{F}_p^i$ are first pooled into the corresponding frustum region to yield a non-compact 2D representation. Subsequently, the non-compact frustum feature is concatenated with the compact frustum feature $\widehat{\mathcal{F}}_f^{i-1}$ and passed through a simple convolutional layer to reduce the number of channels: $$ \begin{array}{r}{\mathcal{F}_{\mathrm{f u s e}}^{i}=\mathrm{C o n v}([\mathrm{F l a t}(\mathcal{F}_{p}^{i});\widehat{\mathcal{F}}_{f}^{i-1}])~,}\end{array} $$ where Conv(·) represents a convolution layer with batch normalization and an activation function. Finally, the fused features are utilized to further enhance the frustum features in a residual-attentive manner: $$ \mathcal{F}_{f}^{i}=\widehat{\mathcal{F}}_{f}^{i-1}+\sigma(h(\mathcal{F}_{\mathrm{f u s e}}^{i},\theta))\odot\mathcal{F}_{\mathrm{f u s e}}^{i}~, $$ where $\sigma$ represents the sigmoid function, $h(\cdot)$ is a linear function with trainable parameters $\theta$, and $\odot$ indicates elementwise multiplication. 5) Fusion Head Module: FRNet aims to assign a category to each point in the point cloud, which requires point-level features for end-to-end label prediction. Although the output point features from the last layer of the frustum-point fusion module can yield promising performance, we observe this is not the optimal solution. Features from different levels have different context-aware information. Lower-level features are geometric while higher-level features are semantically rich. Leveraging these different level features can enhance performance. Drawing inspiration from FIDNet [^59] and CENet [^61], we first combine all point and frustum features from each stage of the backbone and fuse them to reduce channel dimension: $$ \mathcal{F}_{p}^{\mathrm{o u t}}=\mathrm{M L P}([\mathcal{F}_{p}^{1};\ldots;\mathcal{F}_{p}^{V}])~, $$ $$ \mathcal{F}_{f}^{\mathrm{o u t}}=\mathrm{C o n v}([\mathcal{F}_{f}^{1};\ldots;\mathcal{F}_{f}^{V}])~, $$ where V is the number of stages in the backbone. Fusing backbone features with different receptive fields helps obtain more context-aware and semantic information. Additionally, point features $\mathcal{F}_{p}^{out}$ contains local information from neighborhood frustum regions, while frustum features $\mathcal{F}_{f}^{out}$ contains global information within the corresponding frustum region. Meanwhile, $\mathcal{F}_{p}^{0}$ from the frustum feature encoder module constructs graph structures within the frustum region and between neighborhood frustum regions. Progressively fusing these features helps extract features from local to global, and from geometric to semantic, resulting in more accurate predictions. To fuse $\mathcal{F}_{f}^{out}$ and $\mathcal{F}_{p}^{out}$, we first back-project the frustum features to point-level and use an MLP layer to ensure the same channels as $\mathcal{F}_{p}^{out}$. Then, an addition operation is applied to fuse the two features. The same operation is utilized to fuse the new features with $\mathcal{F}_{p}^{0}$. The overall operation can be formulated as: $$ \begin{array}{r}{\mathcal{F}_{\mathrm{l o g i t}}=\mathrm{M L P}(\mathrm{M L P}(\mathrm{I n f l a t}(\mathcal{F}_{f}^{\mathrm{o u t}}))+\mathcal{F}_{p}^{\mathrm{o u t}})+\mathcal{F}_{p}^{0}.}\end{array} $$ Here, $\mathcal{F}_{\mathrm{logit}}$ is used to generate the final semantic scores with a linear head for the point over the entire point cloud. 6) Optimization: In the conventional setup, a point-level loss function is commonly employed to supervise the final semantic scores. However, in this work, we propose optimizing frustum features with additional auxiliary heads, utilizing the frustum-level loss function. This approach requires generating pseudo frustum labels. Typically, a frustum region encompasses multiple points, which may not all belong to the same category. To address this, we adopt a statistical approach inspired by sparse-voxel methods [^16]. We generate a pseudo label for each frustum by first counting the frequency of each category existing in the given frustum region, and then choosing the category with the highest frequency as the pseudo label for the frustum. Finally, the frustum-level features are --- ![Image](images/0dd1e94e9274337451a83aaf16ab4ae153aa3b76fe6afea9c7ab95fe33bb1b15.jpg)
(a) Scene 1
![Image](images/60ca1015eb8a399070dc7dc7fdd2d853f35445cee245441582d42e99a32eba61.jpg)
(b) Scene 2
![Image](images/fabf897960181aa8359890b723cb6604bbef4fa121b5f2a19bfb71b3aa98e885.jpg)
(c) Mixed Scene
Fig. 5. FrustumMix illustration. (a) and (b) show the original two LiDAR scenes. (c) presents the mixed scenes generated by the FrustumMix strategy, where scene 1 is colored green and scene 2 is colored purple.
![Image](images/d13b87ee744528abde7aabaec490c4f38f5cad52315192ce008a8f70edfc9bbf.jpg)
(a) Before RangeInterpolation
![Image](images/dc2bbfc2c9fc04b0e7695aa7032f1fe67885b7c7eb872dab73b7e8326969d194.jpg)
(b) After RangeInterpolation
Fig. 6. RangeInterpolation illustration. (a) shows the raw range image with numerous empty pixels. (b) illustrates the range image after applying RangeInterpolation, resulting in a smoother and more compact representation.
supervised using the generated pseudo labels. The overall loss function can be formulated as follows: $$ \mathcal{L}=\mathcal{L}_{p}+\lambda\mathcal{L}_{f}, $$ where $\mathcal{L}_{p}$ and $\mathcal{L}_{f}$ represent the point-level and frustum-level loss functions, respectively, and $\lambda$ is a hyperparameter that controls the weight of frustum supervision. $\mathcal{L}_{p}$ is calculated by cross-entropy loss, while $\mathcal{L}_{f}$ consists of cross-entropy loss, Lovázs-Softmax loss [^65], and boundary loss [^66]. ### C. Frustum-Range Operators Data augmentation enables the model to learn more robust representations. Common augmentations in LiDAR segmentation typically operate from a global perspective, including rotation, flipping, scaling, jittering, etc. In this work, we introduce two novel augmentations based on our proposed frustum-range view representation. 1) FrustumMix: Previous mixing strategies [^48], [^49] perform point-level swapping between two LiDAR scans. While these methods achieve notable improvements in various LiDAR representations, they are not well aligned with the frustum representation, potentially causing two distinct scenes to mix within the same frustum region. This misalignment disrupts both the semantic coherence and geometric structure within the frustum region. To this end, we propose FrustumMix, a fine-grained extension of LaserMix [^48] that aligns better with the frustum representation. Specifically, given two LiDAR point clouds $\mathcal{P}_1$ and $\mathcal{P}_2$, we randomly split the point clouds into M non-overlapping frustum regions along the inclination or azimuth direction, denoted as $\mathcal{P}_1 = \{\mathcal{A}_1^1, \ldots, \mathcal{A}_1^M\}$ and $\mathcal{P}_2 = \{\mathcal{A}_2^1, \ldots, \mathcal{A}_2^M\}$. We then generate a new LiDAR scan $\hat{\mathcal{P}}$ by alternately swapping frustum regions from $\mathcal{P}_1$ and $\mathcal{P}_2$, as follows: $\hat{\mathcal{P}} = \mathcal{A}_1^1 \cup \mathcal{A}_2^2 \cup \mathcal{A}_1^3 \cup \mathcal{A}_2^4 \cup \cdots$. This approach ensures that mixing occurs at the level of frustum units, preserving semantic coherence and structural details entirely within each frustum region, thereby maintaining alignment with the frustum representation. Fig. 5 provides an illustration of FrustumMix. By enhancing context-awareness between frustums while preserving intra-frustum structural invariance, FrustumMix achieves improved segmentation performance and better representation learning. 2) RangeInterpolation: Previous range-view methods [^36], [^59], [^61] often encounter a large number of empty pixels due to the sparsity of LiDAR points, resulting in noisy information. To mitigate this problem, we introduce RangeInterpolation, a technique that utilizes the range image to reconstruct surfaces in the LiDAR scan. Specifically, the point cloud $\mathcal{P}$ is first projected onto the range image $\mathcal{R} \in \mathbb{R}^{H \times W \times (3+L)}$. For empty pixel positions $(u, v)$ in the range image $\mathcal{R}$, we establish a window of size $m \times n$ centered at $(u, v)$, where m and n are odd. We aim to interpolate the point falling into $(u, v)$ using the surrounding range information within the pre-defined window. Considering that not all pixels in the window contain valid values, we aggregate information from non-empty pixels within the window for point interpolation using an average operation. During the training procedure, the semantic label of the interpolated point is determined via threshold-guided voting within the window. Specifically, we identify the most frequent category within the window and its frequency. If the frequency is below a threshold, we classify the interpolated point as located at the boundary and assign the label as ignored, which will not contribute to loss calculation. Otherwise, the label is determined by the category with the highest frequency. As shown in Fig. 6, RangeInterpolation produces --- coherent semantic information in the range image, resulting in a more compact representation. ### IV. EXPERIMENTS In this section, we demonstrate the scalability and robustness of the proposed method. We first introduce the datasets, metrics, and detailed implementations we used. Subsequently, we show that FRNet achieves an optimal trade-off between efficiency and accuracy when compared to state-of-the-art methods across various task setups. Finally, we conduct a series of ablation studies to analyze each component in FRNet. ### A. Experimental Settings 1) Datasets: We conduct comprehensive experiments on four popular LiDAR segmentation datasets. SemanticKITTI [^32] is a large-scale outdoor dataset that comprises 22 sequences collected from various scenes in Karlsruhe, Germany, using a Velodyne HDL-64E LiDAR sensor. Officially, sequences 00 to 07 and 09 to 10 (19,130 scans) are used for training, sequence 08 (4,071 scans) for validation, and sequences 11 to 21 (20,351 scans) for online testing. The original dataset is annotated with 28 classes, though only 19 merged classes are used for single-scan evaluation. The vertical FOV is $[-25^{\circ}, 3^{\circ}]$. nuScenes [^47] is a multimodal dataset that is widely used in autonomous driving scenarios. It contains LiDAR point clouds captured around streets in Boston and Singapore, provided by 32-beam LiDAR sensors. The dataset includes 1,000 driving scenes with sparser points. It is annotated with 32 classes and only 16 semantic categories are used for evaluation. The vertical FOV is $[-30^{\circ}, 10^{\circ}]$. SemanticPOSS [^68] is a more challenging benchmark collected at Peking University and features much smaller and sparser point clouds. It consists of 2,988 LiDAR scenes with numerous dynamic instances. Officially, it is divided into 6 sequences, with sequence 2 used for evaluation with 13 merged categories. The vertical FOV is $[-16^{\circ}, 7^{\circ}]$. ScribbleKITTI [^67] shares the same scenes with SemanticKITTI [^32] but only provides weak annotations with line scribbles. It contains 189 million labeled points, with approximately 8.06% labeled points available during training. Additionally, we conduct a comprehensive out-of-distribution robustness evaluation experiment on SemanticKITTI-C and nuScenes-C from the Robo3D [^76] benchmark. Each dataset contains eight corruption types, including “fog”, “wet ground”, “snow”, “motion blur”, “beam missing”, “crosstalk”, “incomplete echo”, and “cross sensor”. 2) Evaluation Metrics: In line with standard protocols, we utilize Intersection-over-Union (IoU) and Accuracy (Acc) for each class i and compute the mean Intersection-over-Union (mIoU) and mean Accuracy (mAcc) to assess performance. IoU and Acc can be calculated as follows: $$ \mathrm{I o U}_{i}=\frac{\mathrm{T P}_{i}}{\mathrm{T P}_{i}+\mathrm{F P}_{i}+\mathrm{F N}_{i}},\quad\mathrm{A c c}_{i}=\frac{\mathrm{T P}_{i}}{\mathrm{T P}_{i}+\mathrm{F P}_{i}}, $$ where $TP_{i}$, $FP_{i}$, $FN_{i}$ are true-positives, false-positives, and false-negatives for class i, respectively. For robustness evaluation, we follow the practice of Robo3D [^76] to use Corruption Error (CE) and Resilience Rate (RR) to evaluate the robustness of FRNet. The CE and RR are calculated as follows: $$ \mathrm{C E}_{i}=\frac{\sum_{l=1}^{3}(1-\mathrm{I o U}_{i,l})}{\sum_{l=1}^{3}(1-\mathrm{I o U}_{i,l}^{\mathrm{b a s e}})},\quad\mathrm{R R}_{i}=\frac{\sum_{l=1}^{3}\mathrm{I o U}_{i,l}}{3\times\mathrm{I o U}_{\mathrm{c l e a n}}}, $$ where $\mathrm{IoU}_{i,l}$ represents the task-specific $\mathrm{IoU}$. $\mathrm{IoU}_{i,l}^{\mathrm{base}}$ and $\mathrm{IoU}_{\mathrm{clean}}$ denote scores of the baseline model and scores on the “clean” evaluation set, respectively. 3) Implementation Details: We implement FRNet using the popular MMDetection3D [^77] codebase and employ CENet [^61] as the 2D backbone to extract frustum features. For SemanticKITTI [^32] and ScribbelKITTI [^67], the point cloud is divided into $64 \times 512$ frustum regions, while for nuScenes [^47] and SemanticPOSS [^68], it is divided into $32 \times 480$. For optimization, we choose AdamW [^78] as our default optimizer with an initial learning rate of 0.01. We utilized the OneCycle scheduler [^79] to dynamically adjust the learning rate during training. All models are trained using four GPUs for 50 epochs on SemanticKITTI [^32] and ScribbelKITTI [^67], and for 80 epochs on nuScenes [^47] and SemanticPOSS [^68], respectively. The batch size is set to 4 for each GPU. Additionally, we implement a faster variant, dubbed FastFRNet, which features a larger frustum area and a scaled-down 2D backbone. Specifically, the point cloud is divided into $32 \times 360$ frustum regions for all datasets, and the 2D backbone is a variant of ResNet-18 [^80], following the design of CENet [^61]. ### B. Comparative Study 1) Comparisons to State of the Arts: We begin by comparing FRNet with state-of-the-art LiDAR segmentation methods across different representations, including sparse-voxel, point-view, multi-view, and range-view representations, on both fully-supervised and weakly-supervised benchmarks. Tab. I provides a summary of all the results. In this benchmark, both FrustumMix and RangeInterpolation are employed as data augmentation techniques to enhance model performance. To ensure a fair comparison, we evaluate the Frame Per Second (FPS) on a single GeForce RTX 2080Ti GPU for all models. For SemanticKITTI [^32] and nuScenes [^47], we report mIoU scores on both the validation and test sets. For ScribbleKITTI [^67] and SemanticPOSS [^68], which share the same data across the val and test sets, we report both mIoU and mAcc scores on the val set. Compared with recent multi-view methods [^24], although FRNet does not exhibit a significant advantage in performance, it shows notable superiority in terms of parameters and FPS. In comparison with range-view, sparse-voxel, and raw-points methods, FRNet achieves appealing performance while still maintaining high efficiency. Additionally, Fast-FRNet demonstrates the fastest inference speed with only a minor drop in performance. 2) Label-Efficient LiDAR Semantic Segmentation: Recently, semi-supervised learning has gained traction in LiDAR segmentation. In this work, we adopt the approach of LaserMix [^48] to apply FRNet to the semi-supervised segmentation task, as shown in Tab. II. For fair comparisons ---
TABLE I
SUPERVISED LIDAR SEGMENTATION RESULTS ON THE OFFICIAL SEMANTICKITTI [^32], NUSCENES [^47], SCRIBBLEKITTI [^67], AND SEMANTICPOSS [^68] BENCHMARKS. † DENOTES MODELS THAT USE PRETRAINED WEIGHTS. FPS DENOTES FRAME RATE PER SECOND (s). ALL MIOU AND MACC SCORES ARE GIVEN IN PERCENTAGE (%). THE BEST AND SECOND BEST SCORES FOR MODELS FROM EACH REPRESENTATION GROUP ARE HIGHLIGHTED IN BOLD AND UNDERLINE
MethodVenueRepresentationParamSemanticKITTInuScenesScribbleKITTISemanticPOSS
FPS $\uparrow$Val $\uparrow$Test $\uparrow$Val $\uparrow$Test $\uparrow$mIoU $\uparrow$mAcc $\uparrow$mIoU $\uparrow$mAcc $\uparrow$
MinkUNet [^17]CVPR'19Sparse Voxel ●21.7 M9.162.863.775.8-55.0-53.168.3
SPVNAS [^15]ECCV'20Sparse Voxel ●21.8 M8.962.566.474.4-56.9-48.461.5
Cylinder3D [^16]CVPR'21Sparse Voxel ●55.9 M6.265.967.876.177.957.0-52.964.9
PVKD [^69]CVPR'22Sparse Voxel ●14.1 M13.266.471.276.0-----
KPConv [^7]ICCV'19Raw Points ●18.3 M3.261.358.8------
RandLA-Net [^8]CVPR'20Raw Points ●1.2 M1.957.153.9------
PTv2 [^70]NeurIPS'22Raw Points ●12.8 M4.770.372.680.282.6----
WaffleIron [^11]ICCV'23Raw Points ●6.8 M7.468.070.879.1-----
PTv3 [^12]CVPR'24Raw Points ●46.2 M20.270.874.280.482.7----
RPVNet [^22]ICCV'21Multi-View ●24.8 M7.965.570.377.6-----
2DPASS [^71]ECCV'22Multi-View ●26.5 M8.469.372.279.480.8----
SphereFormer [^72]CVPR'23Multi-View ●32.3 M4.967.874.878.481.9----
UniSeg [^24]ICCV'23Multi-View ●147.6 M6.971.375.2-83.5----
TASeg [^73]CVPR'24Multi-View ●46.7 M6.172.776.5-84.6----
RangeNet++ [^34]IROS'19Range View ●50.4 M12.050.352.265.5-44.657.830.9-
FIDNet [^59]IROS'20Range View ●6.1 M31.858.959.571.4-54.165.446.4-
CENet [^61]ICME'22Range View ●6.8 M33.462.664.773.3-55.766.850.3-
RangeViT† [^35]CVPR'23Range View ●23.7 M10.060.764.075.2-53.666.5--
RangeFormer† [^36]ICCV'23Range View ●24.3 M6.267.673.378.180.163.0---
Fast-FRNetOursFrustum-Range ●7.5 M33.867.172.578.882.162.471.252.567.1
FRNetOursFrustum-Range ●10.0 M29.168.773.379.082.563.172.353.568.1
TABLE II
SEMI-SUPERVISED LIDAR SEGMENTATION RESULTS ON THE SEMANTICKITTI [^32], NUSCENES [^47], AND SCRIBBLEKITTI [^67] BENCHMARKS, UNDER AN ANNOTATION QUOTA OF 1%, 10%, 20%, AND 50%, RESPECTIVELY. ALL MIOU SCORES ARE GIVEN IN PERCENTAGE (%). THE BEST AND SECOND BEST SCORES FOR EACH DATA SPLIT ARE HIGHLIGHTED IN BOLD AND UNDERLINE
MethodVenueBackboneSemanticKITTInuScenesScribbleKITTI
1%10%20%50%1%10%20%50%1%10%20%50%
Sup.-only-FIDNet [^59]36.252.255.957.238.357.562.767.633.147.749.952.5
LaserMix [^48]CVPR'2343.458.859.461.449.568.270.673.038.354.455.658.7
Sup.-only-Cylinder3D [^16]45.456.157.858.750.965.966.671.239.248.052.153.8
CRB [^67]CVPR'22-58.759.160.9-----54.256.558.9
LaserMix [^48]CVPR'2350.660.061.962.355.369.971.873.244.253.755.156.8
LiM3D [^74]CVPR'23-61.662.662.8-----60.360.560.9
ImageTo360 [^75]ICCVW'2354.558.661.464.2--------
FrustumMixOurs55.762.563.064.960.070.072.674.145.655.758.260.8
Sup.-only-PTv3 [^12]42.760.962.664.148.067.871.575.940.754.656.258.9
PolarMix [^49]NeurIPS'2247.261.663.164.853.469.372.076.142.555.856.959.6
LaserMix [^48]CVPR'2351.763.364.865.357.471.874.277.445.056.958.360.7
FrustumMixOurs54.264.765.666.160.372.375.677.746.257.560.061.6
Sup.-only-FRNet44.960.461.863.151.968.170.974.642.453.555.157.0
PolarMix [^49]NeurIPS'2250.160.962.063.855.669.671.073.843.255.056.157.3
LaserMix [^48]CVPR'2352.962.963.265.058.771.572.375.045.856.857.759.0
FrustumMixOurs55.864.865.265.461.272.274.675.446.657.059.561.2
with other methods, FrustumMix and RangeInterpolation are not employed as data augmentation techniques in this benchmark, and TTA is excluded during inference. We first utilize Cylinder3D [^16] and PTv3 [^12] as backbones and leverage the proposed FrustumMix technique to mix labeled and unlabeled data for consistency learning, which aligns with the LaserMix framework [^48]. Notably, FrustumMix demonstrates superior performance compared to the original LaserMix [^48] strategy. Next, we employ FRNet as the backbone and evaluate various strategies for mixing labeled and unlabeled scans, including PolarMix [^49], LaserMix [^48], and our FrustumMix. The Sup-only baseline results are also obtained without FrustumMix and RangeInterpolation as data augmentation techniques. The results indicate that FrustumMix consistently achieves the best scores across all data splits when compared with other mixing strategies, particularly with limited annotations. 3) Out-of-Distribution Robustness: Robustness, which reflects a model's ability to generalize under different conditions, is crucial for automotive security systems, particularly ---
TABLE III
ROBUSTNESS PROBING ON SEMANTICKITTI-C AND NUSCENES-C BENCHMARKS [^76]. ALL MCE, MRR AND MIOU SCORES ARE GIVEN IN PERCENTAGE (%) . THE BEST AND SECOND BEST SCORES FOR EACH MODEL ARE HIGHLIGHTED IN BOLD AND UNDERLINE
#MethodVenuemCE $\downarrow$mRR $\uparrow$mIoU $\uparrow$FogWegSnowMotionBeamCrossEchoSensor
SemanticKITTI-CMinkUNet [^17]CVPR'19100.081.962.855.954.053.332.956.358.354.446.1
SPVCNN [^15]ECCV'20100.382.262.555.354.051.434.556.758.154.646.0
FIDNet [^59]IROS'20113.877.058.843.751.649.740.449.349.548.229.9
Cylinder3D [^16]CVPR'21103.380.163.437.157.546.952.557.656.052.546.2
2DPASS [^71]ECCV'22106.177.564.640.560.748.557.858.828.555.850.0
CENet [^61]ICME'22103.481.362.642.757.353.652.755.845.453.445.8
WaffleIron [^11]ICCV'23109.572.266.045.558.649.333.059.322.558.654.6
FRNetOurs96.880.067.647.662.257.156.862.540.958.147.3
nuScenes-CMinkUNet [^17]CVPR'19100.074.475.853.673.940.473.468.526.663.851.0
SPVCNN [^15]ECCV'20106.774.774.459.072.541.158.465.436.862.349.2
FIDNet [^59]IROS'20122.473.371.464.868.059.048.948.157.548.823.7
Cylinder3D [^16]CVPR'21111.872.976.259.972.758.142.164.544.460.542.2
2DPASS [^71]ECCV'2298.675.277.964.576.854.562.067.834.463.245.8
CENet [^61]ICME'22112.876.073.367.069.961.658.350.060.953.324.8
WaffleIron [^11]ICCV'23106.772.876.156.173.949.659.565.233.161.544.0
FRNetOurs98.677.577.769.176.669.554.568.341.458.743.1
![Image](images/dc9400d4a0fa91b9f992ec3a8a09befca65a07bb10c92a48b6ef45f4975f52c9.jpg)
Fig. 7. Class-wise LiDAR segmentation results of FRNet and the baseline model on the test set of SemanticKITTI [^32].
in extreme weather conditions [^81], [^82]. To evaluate the out-of-distribution robustness of FRNet, we utilize the noisy datasets SemanticKITTI-C and nuScenes-C introduced in Robo3D [^76]. In this benchmark, the models trained on SemanticKITTI [^32] and nuScenes [^47] are directly applied to evaluate the performance on SemanticKITTI-C and nuScenes-C [^76], respectively, without any fine-tuning. As shown in Tab. III, FRNet achieves promising performance across most corruption types and demonstrates superiority over recent works employing various LiDAR representations, including sparse voxel [^15], [^16], [^17], range view [^59], [^61], multi-view fusion [^71], and raw points [^11]. 4) Quantitative Assessments: We compare the class-wise IoU scores of FRNet with those of the baseline method [^61] in Fig. 7. Notably, FRNet demonstrates significant improvement across most semantic classes, particularly for dynamic classes with intricate structures, where it achieves mIoU gains ranging from 15% to 24%. In Fig. 8, we present the prediction results on SemanticKITTI [^32], compared with those of state-of-the-art range view methods, including FIDNet [^59], CENet [^61], and RangeViT [^35]. It is evident that FRNet delivers more accurate predictions, particularly on large planar objects such as “vegetation”, “terrain”, and “sidewalk”, which are often challenging to distinguish accurately. In this section, we will discuss the efficacy of each design in our FRNet architecture. Unless otherwise specified, all ### C. Ablation Study
TABLE IV
ABLATION STUDY OF EACH COMPONENT IN FRNET ON THE VAL SET OF SEMANTICKITTI [^32] AND NUSCENES [^47]. FFE: FRUSTUM FEATURE ENCODER. FP: FRUSTUM-POINT FUSION. FH: FUSION HEAD. FS: FRUSTUM-LEVEL SUPERVISION. RI: RANGEINTERPOLATION. TTA: TEST TIME AUGMENTATION. ALL MIOU AND MACC SCORES ARE GIVEN IN PERCENTAGE (%)
FFEFPFHFSRITTASemKITTI\nmIoUmAccnuScenes\nmIoUmAcc
✓62.772.372.882.7
✓✓64.373.573.683.4
✓✓✓66.273.976.484.6
✓✓✓✓67.074.177.284.8
✓✓✓✓✓67.674.277.785.2
✓✓✓✓✓✓68.774.979.085.0
experiments are conducted and reported on the val set of SemanticKITTI [^32] and nuScenes [^47], respectively. 1) Component Designs: Tab. IV summarizes the ablation results of each component in the FRNet architecture. Notably, FrustumMix is employed across all setups in this ablation study. Firstly, we directly apply the frustum feature encoder to CENet [^61], incorporating KNN post-processing to predict semantic labels over the entire point cloud, achieving 62.7% and 72.8% mIoU on SemanticKITTI [^32] and nuScenes [^47], respectively. Next, by integrating point-frustum fusion to hierarchically update point features, the semantic results --- ![Image](images/72fd39508c8a4ee62c4b6aad021c784e4e4225d1e3ad2071476c06227f192e1f.jpg) ![Image](images/da109cf982f362b3f2526c40f5d506fa63f24642731c2979bba72b1c916f4fae.jpg)
Fig. 8. Qualitative results among state-of-the-art LiDAR segmentation methods [^35], [^59], [^61] on the val set of SemanticKITTI [^32]. To highlight the differences compared with groundtruth, the correct and incorrect predictions are painted in gray and red, respectively. Best viewed in colors and zoomed-in for details.
are predicted with the updated point features in an end-to-end manner, resulting in improvements of 1.6% and 0.8% mIoU, respectively. To regularize the frustum features, the frustum-level supervision is introduced and it leads to significant gains of 1.9% and 2.8% mIoU, respectively. The fusion head module, which fuses multiple features at different levels for more accurate prediction, brings an additional 0.8% mIoU gain on both datasets. Furthermore, the proposed RangeInterpolation, aimed at alleviating empty pixels in the 2D representation by reconstructing semantic surfaces via surrounding range information, provides a performance boost of approximately 0.5% mIoU. Finally, adopting Test Time Augmentation during inference, as demonstrated in prior works, leads to improvements of 1.1% and 1.3% mIoU, respectively. 2) Frustum Representation Resolution: Finding a suitable frustum resolution is crucial for balancing accuracy and efficiency. In Tab. V, we conduct a series of experiments
TABLE V
ABLATION STUDY OF FRUSTUM RESOLUTION IN FRNET ON THE VAL SET OF SEMANTICKITTI [^32]. ALL MIOU AND MACC SCORES ARE GIVEN IN PERCENTAGE (%) WITHOUT TTA
Number of FrustumsHeight (H)Width (W)mIoUmAcc
131,07264204866.873.8
65,53664102467.173.4
65,53632204865.572.7
32,7686451267.674.2
32,76832102466.072.4
16,3846425664.370.5
16,3843251265.572.1
8,1923225664.370.8
with various frustum resolutions. While higher resolution can capture more detailed features, it may not always lead to optimal results. The limited area of the frustum region can result in a lack of meaningful structure, causing the model --- ![Image](images/bfaca3188cdd8ca6d3a31d3dea0d4d335c0713a26bac4d90e7017696571c6eca.jpg)
(a)
(c)
Fig. 9. Ablation study on different FRNet configurations. Plots show the comparisons of: a) Different range-view backbones on the val set of SemanticKITTI [^32]. b) Different mixing strategies on the val set of nuScenes [^47]. c) Different mixing strategies on the val set of SemanticKITTI [^32]. Color-filled boxes denote mAcc while empty boxes denote mIoU in (b) and (c), respectively.
to struggle with noisy or sparse data. Conversely, although lower resolution can improve inference efficiency and provide a more coherent representation, the larger area of the frustum region may introduce excessive noise information, particularly at object boundaries. Thus, finding an optimal frustum resolution is crucial for balancing the richness of features with computational demands and maintaining overall model performance. 3) Mixing Strategies: We further explore the effectiveness of the proposed FrustumMix. As shown in Tab. II, FrustumMix significantly improves performance on semi-supervised learning tasks and demonstrates generalizability to other modalities. To contextualize FrustumMix within common data augmentation techniques, we compare it with recent popular outdoor mixing strategies, including LaserMix [^48] and PolarMix [^49]. As illustrated in Fig. 9(b) and Fig 9(c), FrustumMix consistently outperforms these prior strategies. 4) RangeInterpolation Window Settings: RangeInterpolation reconstructs semantic surfaces based on the range image, facilitating a compact 2D representation for coherent semantic learning. Tab. VI showcases the performance of RangeInterpolation with different window settings. Notably, we observe consistent performance improvements when considering interpolation along the horizontal or vertical direction exclusively. Specifically, configurations employing horizontal windows, such as the $1 \times 3$ and $1 \times 5$ settings, demonstrate superior results compared to those focusing on the vertical direction ( $3 \times 1$ and $5 \times 1$ windows). We conjecture that points aligned along the same laser beams often share similar attributes, leading to more accurate and coherent semantic surfaces. Conversely, larger window sizes, such as the $3 \times 5$, $3 \times 3$, and $5 \times 5$ configurations, tend to exhibit non-uniform point distributions within the window, resulting in noisy points that fail to accurately represent the expected region and consequently leading to a drop in performance. 5) 2D Backbones: To validate the versatility of our method, we employ various range-view methods as our 2D backbone for extracting frustum features, including SalsaNext [^64],
TABLE VI
ABLATION STUDY ON WINDOW SETTINGS IN RANGE INTERPOLATION ON THE VAL SET OF SEMANTICKITTI [^32]. GREEN PIXEL DENOTES THE EMPTY PIXEL NEEDED TO BE INTERPOLATED. VIOLET PIXELS ARE THE SURROUNDING PIXELS USED TO GENERATE THE POINT. ALL MIOU SCORES ARE GIVEN IN PERCENTAGE (%)
Baseline$1 \times 3$$3 \times 1$$1 \times 5$$5 \times 1$
![Image](images/dc748fff0238ddd6b84a83ff5768b3a3cae29291f9e13e46d5be76baa26e4604.jpg)![Image](images/2a968859c9999048b6a9f9abf9a6189485e12e5292e30f98ef92b4317c8d71b0.jpg)![Image](images/c6fd2b765b1eb81cfeef9da48377ec933c0ad7cef023ee543ff0b7202db150b1.jpg)![Image](images/961d390a6ff38ef267eccd42b38866191eb3d56a6fd608dfbd484240f01a18ae.jpg)
67.067.6 $(+0.6)$67.4 $(+0.4)$67.5 $(+0.5)$67.4 $(+0.4)$
cross $3 \times 3$cross $5 \times 5$$3 \times 5$$3 \times 3$$5 \times 5$
![Image](images/ed27ccc3661aa5d13e7c4df38975382a454ca1bba8c06a72f90e8aba36c368c1.jpg)![Image](images/3987eae36c097e99c1eea8ab4bdaeb76081a85ea80f56292e7dbdfba206d25f8.jpg)![Image](images/4e386693b550c5b62fb34d8fbacb2a404a2d68f68806dc114768de4af512e83b.jpg)![Image](images/2ae6bc9ee4766d5d3e3a1c3d66a8208245afdd85bf7cdadd4734583e9a8873f6.jpg)![Image](images/ab12785f554c824061ece7ccbef391b2725f7f20ddfce539f8815cf58f3010e2.jpg)
67.5 $(+0.5)$67.4 $(+0.4)$67.1 $(+0.1)$66.8 $(-0.2)$66.6 $(-0.4)$
TABLE VII
INFERENCE EFFICIENCY COMPARISON OF FRNET AND MINKUNET [^17] IMPLEMENTED WITH VARIOUS CODEBASES AND SPARSE CONVOLUTION BACKENDS, AS WELL AS PTV3 [^12] ON THE VAL SET OF SEMANTICKITTI [^32]
CodebaseBackboneBackendFPS
MMDetection3DMinkUNetMinkowskiEngine9.1
MinkUNetSpConv8.6
MinkUNetTorchSparse10.2
MinkUNetTorchSparse++21.3
FRNetConvolution29.1
Fast-FRNetConvolution33.8
PCSegMinkUNetTorchSparse10.6
PointCeptMinkUNetMinkowskiEngine8.9
PTv3FlashAttention20.2
FIDNet [^59], and CENet [^61]. As depicted in Fig. 9(a), our proposed frustum-range representation consistently delivers performance gains ranging from 5.0% to 6.8% in mIoU across different range-view methods. 6) Inference Efficiency: Multiple sparse convolution backends, including MinkowskiEngine, SpConv, TorchSparse, and TorchSparse++, offer efficient processing for voxel-based methods. To compare FRNet with these backends, we implement MinkUNet [^17] using the MMDetection3D codebase across various backends. Additionally, we evaluate the FPS of MinkUNet implemented with the PCSeg and PointCept codebases, as well as the recent efficient method PTv3 [^12]. All evaluations are conducted on a single NVIDIA 2080Ti GPU. As shown in Tab. VII, while TorchSparse++ delivers satisfactory efficiency for LiDAR segmentation, FRNet achieves the fastest inference speed, making it suitable for real-time applications in autonomous driving systems. 7) Multi-Dataset Fusion: Recent studies [^51], [^83] highlight the importance of large-scale benchmarks achieved through multi-dataset fusion strategies across heterogeneous datasets. In this work, We extend Fast-FRNet to support multi-dataset joint training, as shown in Tab. VIII. Initially, we combine multi-source datasets using mean-variance --- ![Image](images/5ebff666c94aa23d702e4deb2f54933481f603f5895560de67fe25663c0cc331.jpg)
(a) car
![Image](images/d13af8750d3690eccfee6a63961b2045322032138426e07c5c27cfb37679ee8e.jpg) ![Image](images/c56f0dc29b9d4ebb37548736d9cd629867b72167c03d5113f028268dbedb80cb.jpg)
(b) person
![Image](images/055b54a518f00d87c33a82caa96e4c4b2e630b30c854a798ee127f4217302c4e.jpg)
(d) drivable surface
![Image](images/6f1e098abaea2a5a830edc50eee663c5a1d7d909e19bf078a078ecc6f90ced7a.jpg)
(c) vegetation
(e) sidewalk
![Image](images/f798aeb8b1019a8710d1c07601cfbcadca12f6dd26654a68f7c7ce44a158f1ea.jpg)
(f) trailer
Fig. 10. The open-vocabulary LiDAR semantic segmentation experiments of FRNet. Our model demonstrates the ability to perform open-vocabulary segmentation with various textual inputs. The segmented points corresponding to the provided text descriptions are highlighted in blue, showcasing its potential for open-world perception tasks.
TABLE VIII
THE MULTI-SOURCE TRAINING RESULTS OF THE PROPOSED FAST-FRNET ON THE VAL SET OF THE SEMANTICKITTI [^32], NUSCENES [^47], AND SEMANTICPOSS [^68] DATASETS. ALL MIOU AND MACC SCORES ARE GIVEN IN PERCENTAGE (%)
MethodSemKITTInuScenesSemPOSS
mIoUmAccmIoUmAccmIoUmAcc
Single-Source67.173.878.884.452.567.1
Multi-Source67.474.279.184.753.268.0
Fast-FRNet (w/ PPT [^83])68.274.779.384.953.968.4
TABLE IX
ZERO-SHOT SEGMENTATION PERFORMANCE ON THE VAL SET OF NUSCENES [^47]. ALL MIOU AND MACC SCORES ARE GIVEN IN PERCENTAGE (%)
MethodnuScenes
mIoUmAcc
OpenScene [^84]42.161.8
+ FRNet43.562.0
normalization to align intensity values, which provides slight performance gains compared to single-source training. Additionally, inspired by PPT [^83], we incorporate point-prompt techniques to facilitate multi-source joint training. This approach yields further improvements, demonstrating the effectiveness of prompt-guided strategies in multi-dataset fusion. 8) Open-Vocabulary Semantic Segmentation: To investigate the potential of FRNet in open-vocabulary segmentation, we adopt an approach inspired by OpenScene [^84]. Specifically, we integrate text and image features into the framework, aligning them with point features. During inference, text inputs are used to guide the segmentation of point clouds with similar features. As shown in Tab. IX, FRNet demonstrates promising performance in zero-shot segmentation. Additionally, in Tab. 10, we present qualitative results of segmentation using corresponding text inputs, highlighting the capability of FRNet to handle open-world perception tasks effectively. ### V. CONCLUSION In this work, we presented FRNet, a Frustum-Range network designed for efficient and effective LiDAR segmentation, which can be seamlessly integrated into range-view approaches. FRNet comprises three key components: the Frustum Feature Encoder Module for extracting per-point features at lower levels, the Frustum-Point Fusion Module for hierarchical updating of per-point features, and the Fusion Head Module for fusing features from different levels to predict more accurate labels. Additionally, we introduced two efficient augmentation methods, FurstumMix and RangeInterpolation, to enhance the generalizability of range-view LiDAR segmentation. Extensive experiments across diverse driving perception datasets verified that FRNet achieves competitive performance while maintaining high efficiency. We hope this work can lay a solid foundation for future works targeted at real-time, in-vehicle 3D perception. Moving forward, we will further enhance the representation learning capability of FRNet and incorporate such a method into other 3D perception tasks, such as 3D object detection and semantic occupancy prediction, and comprehensive vehicle-road coordination perception tasks. ## REFERENCES [^1]: Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3D point clouds: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 12, pp. 4338–4364, Dec. 2020. [^2]: D. Kong, X. Li, Q. Xu, Y. Hu, and P. Ni, “SC_LPR: Semantically consistent LiDAR place recognition based on chained cascade network in long-term dynamic environments,” IEEE Trans. Image Process., vol. 33, pp. 2145–2157, 2024. [^3]: Y. Li, L. Kong, H. Hu, X. Xu, and X. Huang, "Is your LiDAR placement optimized for 3D scene understanding?", in Proc. Adv. Neural Inf. Process. Syst., vol. 36, 2024, pp. 34980–35017. [^4]: L. Kong et al., “Calib3D: Calibrating model preferences for reliable 3D scene understanding,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis., Jun. 2025, pp. 1965–1978. [^5]: T. Sun, Z. Zhang, X. Tan, Y. Qu, and Y. Xie, “Image understands point cloud: Weakly supervised 3D semantic segmentation via association learning,” IEEE Trans. Image Process., vol. 33, pp. 1838–1852, 2024. [^6]: P. Hu, S. Sclaroff, and K. Saenko, “Leveraging geometric structure for label-efficient semi-supervised scene segmentation,” IEEE Trans. Image Process., vol. 31, pp. 6320–6330, 2022. --- [^7]: H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, "KPConv: Flexible and deformable convolution for point clouds," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2019, pp. 6411–6420. [^8]: Q. Hu et al., “RandLA-Net: Efficient semantic segmentation of large-scale point clouds,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 11108–11117. [^9]: H. Shuai, X. Xu, and Q. Liu, “Backward attentive fusing network with local aggregation classifier for 3D point cloud semantic segmentation,” IEEE Trans. Image Process., vol. 30, pp. 4973–4984, 2021. [^10]: T. Zhang, M. Ma, F. Yan, H. Li, and Y. Chen, "PIDS: Joint point interaction-dimension search for 3D point cloud," in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2023, pp. 1298–1307. [^11]: G. Puy, A. Boulch, and R. Marlet, “Using a waffle iron for automotive point cloud semantic segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., Oct. 2023, pp. 3379–3389. [^12]: X. Wu et al., “Point transformer v3: Simpler, faster, stronger,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2024, pp. 4840–4851. [^13]: J. Li, X. He, Y. Wen, Y. Gao, X. Cheng, and D. Zhang, “Panoptic-PHNet: Towards real-time and high-precision LiDAR panoptic segmentation via clustering pseudo heatmap,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 11799–11808. [^14]: F. Hong, L. Kong, H. Zhou, X. Zhu, H. Li, and Z. Liu, "Unified 3D and 4D panoptic segmentation via dynamic shifting networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 5, pp. 3480–3495, May 2024. [^15]: H. Tang et al., “Searching efficient 3D architectures with sparse point-voxel convolution,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 685–702. [^16]: X. Zhu et al., “Cylindrical and asymmetrical 3D convolution networks for LiDAR segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 9939–9948. [^17]: C. Choy, J. Gwak, and S. Savarese, “4D spatio-temporal ConvNets: Minkowski convolutional neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 3075–3084. [^18]: L. Zhao, S. Xu, L. Liu, D. Ming, and W. Tao, “SVASeg: Sparse voxel-based attention for 3D LiDAR point cloud semantic segmentation,” Remote Sens., vol. 14, no. 18, p. 4471, Sep. 2022. [^19]: V. Vanian, G. Zamanakos, and I. Pratikakis, “Improving performance of deep learning models for 3D point cloud semantic segmentation via attention mechanisms,” Comput. Graph., vol. 106, pp. 277–287, Aug. 2022. [^20]: A. Boulch, C. Sautier, B. Michele, G. Puy, and R. Marlet, “ALSO: Automotive LiDAR self-supervision by occupancy estimation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 13455–13465. [^21]: V. E. Liong, T. N. T. Nguyen, S. Widjaja, D. Sharma, and Z. J. Chong, “AMVNet: Assertion-based multi-view fusion network for LiDAR semantic segmentation,” 2020, arXiv:2012.04934. [^22]: J. Xu, R. Zhang, J. Dou, Y. Zhu, J. Sun, and S. Pu, "RPVNet: A deep and efficient range-point-voxel fusion network for LiDAR point cloud segmentation," in Proc. IEEE/CVF Int. Conf. Comput. Vis., Oct. 2021, pp. 16024–16033. [^23]: Y. Ali Alnaggar, M. Afifi, K. Amer, and M. ElHelw, “Multi projection fusion for real-time semantic segmentation of 3D LiDAR point clouds,” in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2021, pp. 1799–1808. [^24]: Y. Liu et al., “UniSeg: A unified multi-modal LiDAR segmentation network and the OpenPCSeg codebase,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2023, pp. 21662–21673. [^25]: R. Chen et al., "CLIP2Scene: Towards label-efficient 3D scene understanding by CLIP," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 7020–7030. [^26]: R. Chen et al., “Towards label-free scene understanding by vision foundation models,” in Proc. Adv. Neural Inf. Process. Syst., Jan. 2023, pp. 75896–75910. [^27]: M. Jaritz, T.-H. Vu, R. de Charette, É. Wirbel, and P. Pérez, “Crossmodal learning for domain adaptation in 3D semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 2, pp. 1533–1544, Feb. 2023. [^28]: J. Xu et al., “Visual foundation models boost cross-modal unsupervised domain adaptation for 3D semantic segmentation,” 2024, arXiv:2403.10001. [^29]: H. Yu et al., “DAIR-V2X: A large-scale dataset for vehicle-infrastructure cooperative 3D object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 21361–21370. [^30]: R. Xu et al., “V2V4Real: A real-world large-scale dataset for vehicle-to-vehicle cooperative perception,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 13712–13722. [^31]: H. Yu et al sequential dataset for vehicle-in-casting," in Proc. IF'n. 2023, [^32]: J. Behley et al., “SemanticKITTI: A dataset for semantic scene understanding of LiDAR sequences,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 9297–9307. [^33]: B. Wu, A. Wan, X. Yue, and K. Keutzer, “SqueezeSeg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2018, pp. 1887–1893. [^34]: A. Milioto, I. Vizzo, J. Behley, and C. Stachniss, “RangeNet++: Fast and accurate LiDAR semantic segmentation,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nov. 2019, pp. 4213–4220. [^35]: A. Ando, S. Gidaris, A. Bursuc, G. Puy, A. Boulch, and R. Marlet, "RangeViT: Towards vision transformers for 3D semantic segmentation in autonomous driving," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 5240–5250. [^36]: L. Kong et al., “Rethinking range view representation for LiDAR segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2023, pp. 228–240. [^37]: Y. Zhang et al., “PolarNet: An improved grid representation for online LiDAR point clouds semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 9601–9610. [^38]: Q. Chen, S. Vora, and O. Beijbom, “PolarStream: Streaming object detection and segmentation with polar pillars,” in Proc. Adv. Neural Inf. Process. Syst., vol. 34, May 2021, pp. 26871–26883. [^39]: E. E. Aksoy, S. Baci, and S. Cavdar, “SalsaNet: Fast road and vehicle segmentation in LiDAR point clouds for autonomous driving,” in Proc. IEEE Intell. Vehicles Symp. (IV), Oct. 2020, pp. 926–932. [^40]: Z. Zhou, Y. Zhang, and H. Foroosh, “Panoptic-PolarNet: Proposal-free LiDAR point cloud panoptic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 13189–13198. [^41]: J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 3431–3440. [^42]: H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, "Pyramid scene parsing network," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2881–2890. [^43]: E. Xie et al., “SegFormer: Simple and efficient design for semantic segmentation with transformers,” in Proc. Adv. Neural Inf. Process. Sys. (NIPS), vol. 34, Dec. 2021, pp. 12077–12090. [^44]: Z. Liang, M. Zhang, Z. Zhang, X. Zhao, and S. Pu, “RangeRCNN: Towards fast and accurate 3D object detection with range image representation,” 2020, arXiv:2009.00206. [^45]: Z. Tian, X. Chu, X. Wang, X. Wei, and C. Shen, "Fully convolutional one-stage 3D object detection on LiDAR range images," in Proc. Adv. Neural Inf. Process. Syst., Jan. 2022, pp. 34899–34911. [^46]: L. Kong, N. Quader, and V. E. Liong, "ConDA: Unsupervised domain adaptation for LiDAR segmentation via regularized domain concatenation," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2023, pp. 9338–9345. [^47]: W. K. Fong et al., “Panoptic nuscenes: A large-scale benchmark for LiDAR panoptic segmentation and tracking,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 3795–3802, Apr. 2022. [^48]: L. Kong, J. Ren, L. Pan, and Z. Liu, "LaserMix for semi-supervised LiDAR semantic segmentation," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 21706–21716. [^49]: A. Xiao, J. Huang, D. Guan, K. Cui, S. Lu, and L. Shao, “PolarMix: A general data augmentation technique for LiDAR point clouds,” in Proc. Adv. Neural Inf. Process. Syst., Jan. 2022, pp. 11035–11048. [^50]: A. Nekrasov, J. Schult, O. Litany, B. Leibe, and F. Engelmann, “Mix3D: Out-of-context data augmentation for 3D scenes,” in Proc. Int. Conf. 3D Vis. (3DV), 2021, pp. 116–125. [^51]: Y. Liu et al., “Multi-space alignments towards universal LiDAR segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2024, pp. 14648–14661. [^52]: Y. Liu et al., “Segment any point cloud sequences by distilling vision foundation models,” in Proc. Adv. Neural Inf. Process. Syst., vol. 36, Jan. 2023, pp. 37193–37229. [^53]: R. Li, A.-Q. Cao, and R. de Charette, "COARSE3D: Class-prototypes for contrastive learning in weakly-supervised 3D point cloud segmentation," in Proc. Brit. Mach. Vis. Conf., Jan. 2022, pp. 1–19. --- [^54]: B. Wu, X. Zhou, S. Zhao, X. Yue, and K. Keutzer, “SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud,” in Proc. Int. Conf. Robot. Autom. (ICRA), May 2019, pp. 4376–4382. [^55]: F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size,” 2016, arXiv:1602.07360. [^56]: C. Xu et al., “Squeezesegv3: Spatially-adaptive convolution for efficient point-cloud segmentation,” in Proc. 16th Eur. Conf. Comput. Vis. (ECCV), Aug. 2020, pp. 1–19. [^57]: Y. Chen et al., “PointMixup: Augmentation for point clouds,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2020, pp. 330–345. [^58]: J. Zhang et al., “PointCutMix: Regularization strategy for point cloud classification,” Neurocomputing, vol. 505, pp. 58–67, Sep. 2022. [^59]: Y. Zhao, L. Bai, and X. Huang, "FIDNet: LiDAR point cloud semantic segmentation with fully interpolation decoding," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Sep. 2021, pp. 4453–4458. [^60]: D. Kochanov, F. K. Nejadasl, and O. Booij, “KPRNet: Improving projection-based LiDAR semantic segmentation,” 2020, arXiv:2007.12668. [^61]: H. Cheng, X. Han, and G. Xiao, “CENet: Toward concise and efficient LiDAR semantic segmentation for autonomous driving,” in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jul. 2022, pp. 1–6. [^62]: Q. Hu, B. Yang, S. Khalid, W. Xiao, N. Trigoni, and A. Markham, "Towards semantic segmentation of urban-scale 3D point clouds: A dataset, benchmarks and challenges," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 4977–4987. [^63]: C. R. Qi, Y. Li, H. Su, and L. Guibas, “PointNet++: Deep hierarchical feature learning on point sets in a metric space,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, Jan. 2017, pp. 5105–5114. [^64]: T. Cortinhal, G. Tzelepis, and E. E. Aksoy, “SalsaNext: Fast, uncertainty-aware semantic segmentation of LiDAR point clouds,” in Proc. Int. Symp. Visual Comput., Oct. 2020, pp. 207–222. [^65]: M. Berman, A. R. Triki, and M. B. Blaschko, “The Lovász-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 4413–4421. [^66]: R. Razani, R. Cheng, E. Taghavi, and L. Bingbing, “Lite-HDSeg: LiDAR semantic segmentation using lite harmonic dense convolutions,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2021, pp. 9550–9556. [^67]: O. Unal, D. Dai, and L. Van Gool, “Scribble-supervised LiDAR semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 2687–2697. [^68]: Y. Pan, B. Gao, J. Mei, S. Geng, C. Li, and H. Zhao, “SemanticPOSS: A point cloud dataset with large quantity of dynamic instances,” in Proc. IEEE Intell. Vehicles Symp., Oct. 2020, pp. 687–693. [^69]: Y. Hou, X. Zhu, Y. Ma, C. C. Loy, and Y. Li, “Point-to-voxel knowledge distillation for LiDAR semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 8479–8488. [^70]: X. Wu, Y. Lao, L. Jiang, X. Liu, and H. Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,” in Proc. Adv. Neural Inf. Process. Syst., vol. 35, Jan. 2022, pp. 33330–33342. [^71]: X. Yan et al., “2DPASS: 2D priors assisted semantic segmentation on LiDAR point clouds,” in Proc. Eur. Conf. Comput. Vis., 2022, pp. 677–695. [^72]: X. Lai, Y. Chen, F. Lu, J. Liu, and J. Jia, “Spherical transformer for LiDAR-based 3D recognition,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 17545–17555. [^73]: X. Wu et al., “TASeg: Temporal aggregation network for LiDAR semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2024, pp. 15311–15320. [^74]: L. Li, H. P. H. Shum, and T. P. Breckon, “Less is more: Reducing task and model complexity for 3D point cloud semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 9361–9371. [^75]: L. Reichardt, N. Ebert, and O. Wasenmüller, “360° from a single camera: A few-shot approach for LiDAR segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. Workshops (ICCVW), Oct. 2023, pp. 1075–1083. [^76]: L. Kong et al., “Robo3D: Towards robust and reliable 3D perception against corruptions,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2023, pp. 19937–19949. [^77]: MMDetection3D Contributors. (2020). MMDetection3D: OpenMMLab Next-Generation Platform for General 3D Object Detection. [Online]. Available: [^78]: I. Loshchilov and F. Hutter, "Decoupled weight decay regularization," in Proc. Int. Conf. Learn. Represent., Jan. 2019, pp. 1–18. [^79]: L. N. Smith and N. Topin, “Super-convergence: Very fast training of neural networks using large learning rates,” in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, vol. 11006. Bellingham, WA, USA: SPIE, 2019, pp. 369–386. [^80]: K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770–778. [^81]: L. Kong, S. Xie, H. Hu, L. X. Ng, B. R. Cottereau, and W. T. Ooi, “RoboDepth: Robust out-of-distribution depth estimation under corruptions,” in Proc. Adv. Neural Inf. Process. Syst., Jan. 2023, pp. 21298–21342. [^82]: Q. Huang et al., “PointCAT: Contrastive adversarial training for robust point cloud recognition,” IEEE Trans. Image Process., vol. 33, pp. 2183–2196, 2024. [^83]: X. Wu et al., “Towards large-scale 3D representation learning with multi-dataset point prompt training,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2024, pp. 19551–19562. [^84]: S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, and T. Funkhouser, “OpenScene: 3D scene understanding with open vocabularies,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 815–824. ![Image](images/e5153c3a0cd5815bedaef316d3a059fcf83b480947dda5d525af53503bded2dc.jpg) Xiang Xu received the B.E. and M.S. degrees from the School of Automation, Nanjing University of Information Science and Technology, Nanjing, China, in 2018 and 2021, respectively. He is currently pursuing the Ph.D. degree with the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing. His research interests include 3D perception and label-efficient learning. ![Image](images/36aa0eb38da3df8c03ca72c64cb3de9a4d7c6a6e79de6e348d0f7098a265a474.jpg) Lingdong Kong (Student Member, IEEE) received the B.Eng. degree from South China University of Technology, Guangzhou, China, in 2019, and the M.Sc. and M.Eng. degrees from Nanyang Technological University, Singapore, in 2020 and 2022, respectively. He is currently pursuing the Ph.D. degree with the School of Computing, National University of Singapore. His research interests include robust and scalable 3D perception and generation. He was a recipient of the 2025 Apple Scholar in AI/ML Ph.D. Fellowship. ![Image](images/b694b4f82c5096431565a02445b283d75efb13613d0dabddeeb9645eefdd9f95.jpg) Hui Shuai received the M.S. and Ph.D. degrees from Nanjing University of Information Science and Technology, Nanjing, China, in 2018 and 2023, respectively. He is currently a Lecturer with Nanjing University of Posts and Telecommunications, Nanjing. His research interests include object detection and 3D point cloud analysis. ![Image](images/2a42e6b025606b50276f36a20276821d98bbe6a0961b1ad67d00b90f2485ba57.jpg) Qingshan Liu (Senior Member, IEEE) received the M.S. degree from Southeast University, Nanjing, China, in 2000, and the Ph.D. degree from Chinese Academy of Sciences, Beijing, China, in 2003. From June 2004 to April 2005, he was an Associate Researcher with the Multimedia Laboratory, The Chinese University of Hong Kong, Hong Kong. From 2010 to 2011, he was an Assistant Research Professor with the Department of Computer Science, Computational Biomedicine Imaging and Modeling Center, Rutgers, The State University of New Jersey, Piscataway, NJ, USA. Before he joined Rutgers University, he was an Associate Professor with the National Laboratory of Pattern Recognition, Chinese Academy of Sciences. He is currently a Professor with Nanjing University of Posts and Telecommunications, Nanjing. His research interests include image and vision analysis and machine learning.