# TransRVNet: LiDAR Semantic Segmentation With Transformer Hui-Xian Cheng, Xian-Feng Han $^{ID}$, Member, IEEE, and Guo-Qiang Xiao Abstract—Effective and efficient 3D semantic segmentation from large-scale LiDAR point cloud is a fundamental problem in the field of autonomous driving. In this paper, we present Transformer-Range-View Network (TransRVNet), a novel and powerful projection-based CNN-Transformer architecture to infer point-wise semantics. First, a Multi Residual Channel Interaction Attention Module (MRCIAM) is introduced to capture channel-level multi-scale feature and model intra-channel, interchannel correlations based on attention mechanism. Then, in the encoder stage, we use a well-designed Residual Context Aggregation Module (RCAM), including a residual dilated convolution structure and a context aggregation module, to fuse information from different receptive fields while reducing the impact of missing points. Finally, a Balanced Non-square-Transformer Module (BNTM) is employed as fundamental component of decoder to achieve locally feature dependencies for more discriminative feature learning by introducing the non-square shifted window strategy. Extensive qualitative and quantitative experiments conducted on challenging SemanticKITTI and SemanticPOSS benchmarks have verified the effectiveness of our proposed technique. Our TransRVNet presents superior performance over most existing state-of-the-art approaches. The source code and trained model are available at . ![Image](images/fad9c70bbbda7366a9f8a2958635cc62fc18719169fa94ebc2a406be4c84152e.jpg) Index Terms—CNN, transformer, point cloud, range image, semantic segmentation, autonomous driving. ### I. INTRODUCTION ![Image](images/3d600a88c43e3bacb9e4548a1c53d3c7cbea555511e9afa39326f4113870a0b9.jpg) A DEQUATELY accurate, robust, reliable and real-time 3D semantic perception and understanding of vehicles surrounding environment are critical requirements for autonomous driving. To achieve this end, multiple types of sensors with complementary characteristics, such as cameras, Light Detection And Ranging (LiDAR) [^1], Radio Detection And Ranging (RADAR) [^2], are usually used in self-driving systems, among which LiDARs play an important role because they can provide distance measurements and 3D geometry with high precision. Therefore, LiDAR-based semantic scene perception, especially 3D point cloud semantic

Fig. 1. Top: Velodyne HDL-64E [^6] laser scan from SemanticKITTI dataset [^7] with semantic predictions from our TransRVNet. Bottom: Pandora [^8] laser scan from SemanticPOSS dataset [^9] with semantic predictions from our TransRVNet.

segmentation [^3], [^4], [^5] as a primary task, has been attracting increasing research attention, which aims to perform prediction of per-point class labels. Recently, convolutional neural networks (CNNs) have brought significant performance improvement for semantic image segmentation [^10], whose success is attributed to their power of learning hierarchical feature representations. However, CNNs are not suitable for 3D point clouds, since 1) standard convolution is defined over regular grids [^11], while 2) 3D point cloud produced by LiDAR are inherently sparse, irregular, and unordered [^12]. 3) There is lack of sufficient large-scale training datasets. Fortunately, with the availability of recent large-scale datasets (e.g. SemanticKITTI), several state-of-the-art approaches using deep neural networks have been proposed to address these problems. And according to the representation of 3D LiDAR points, these methods are commonly classified into voxel-based methods [^13], [^14], image-based methods [^15], [^16], and point-based methods [^17], [^18], [^19]. --- For point-based approaches, although impressive progress has been achieved, high computational cost and memory consumption make this kind of methods too expensive for large outdoor scenarios in autonomous driving applications. RandLA-Net reduces the computational complexity by replacing the traditional FPS strategy with random sampling method and significantly improves the efficiency of point cloud processing. However, its segmentation accuracy is uncompetitive. Additionally, RandLA-Net uses K-nearest neighbor approach to maintain the local relationship among points in each frame, in which the usage of time-consuming KD tree construction severely affects the inference speed. While voxel-based and image-based methods may be a reasonable choice that converts the raw point cloud into structured format, allowing standard 3D CNNs or 2D CNNs to be used directly [^20]. However, sparsity and granular information loss challenges the application of voxel representation [^21]. And image-based methods also cannot produce satisfactory performance because the projected LiDAR and RGB image live in different domains [^22]. Therefore, it is necessary to design specific architecture for projection-based (i.e. LiDAR-based range image) semantic segmentation, which should take into account the following problems 1) How to project the full 360 LiDAR scan into 2D surface? 2) How to learn local and global contextual information? 3) How to model long-range feature dependencies from local and global regions? Recent advancements in Natural Language Processing (NLP) using Transformer [^23] have motivated the computer vision community to investigate the adaption of Transformer models to vision tasks, such as ViT [^24] for image recognition, and SETR [^25] on image segmentation task. However, from the other work that followed [^26], [^27], [^28], [^29], [^30], [^31], we find that 1) the pure Transformer segmentation networks like SETR cannot result in very good performance. 2) The SETR is difficult to train, since it is parametric intensive and computationally expensive. 3) Additionally, the SETR fails to perform multi-scale feature extraction which is important for dealing with large scale variations of objects [^22]. To tackle these issues, we contribute a novel CNN-Transformer architecture based upon an encoder-decoder skeleton, named Transformer-Range-View Network (TransRVNet). After representing the input 3D LiDAR point cloud as a 2D range image, a Multi Residual Channel Interaction Attention Module (MRCIAM) is developed to extract channel-wise multi-scale features and model context interactions within and across different information channels. Then, in our CNN-Transformer framework, a Residual Context Aggregation Module (RCAM) based on residual dilated convolution structure is used as encoding component to explore the information from different receptive fields. And, in the decoding stage, we construct a Balanced Non-square-Transformer Module (BNTM) to capture locally long-range feature dependencies using our well-designed non-square shifted window based Transformer. We perform extensive evaluation on two publicly challenging benchmarks SemanticKITTI [^7] and SemanticPOSS [^9]. The experimental results show that our TransRVNet is superior to several state-of-the-art models. In summary, the key contributions of this paper are listed as follows: - Based on attention mechanism, a Multi Residual Channel Interaction Attention Module is well designed to learn multi-scale context information from different modal channels respectively, and encodes the intra-channel as well as inter-channel interactions/feature dependencies, simultaneously. - We propose a new Balanced Non-square-Transformer Module using non-square shifted window based Transformer block to better deal with size imbalanced range image, and learn locally long-range feature dependencies. - We introduce a novel CNN-Transformer architecture, named Transformer-Range-View Network (TransRVNet) with a Residual Context Aggregation Module as encoder component to gain local-to-global context information from progressively enlarged receptive fields, and the proposed BNTM as decoder block. To the best of our knowledge, this is the first work focusing on the application of Transformer to semantic LiDAR point cloud segmentation. - Comprehensive experimental evaluation on publicly available datasets SemanticKITTI and SemanticPOSS validate the superior performance of TransRVNet. And we further analyze the effect of each designed module had on our neural network. The rest of our paper is structured as follows. Section II gives an overview of recent related works. Section III presents the detailed description of our TransRVNet. Comprehensive experimental evaluation and analysis are performed in Section IV. Section V provides the ablation study of our proposed method. Conclusions are drawn in Section VI. ### II. RELATED WORKS ### A. Semantic Segmentation of 3D Point Cloud With the remarkable advancement of deep learning technology and the availability of large-scale scene datasets, semantic 3D LiDAR point cloud segmentation for autonomous driving have achieved significant progress in recent years. The previous studies can be grouped into three categories, including point-based, voxel-based and image-based methods. Point-based methods directly deal with unstructured point cloud with shared point-wise MLPs [^17], or definition of point convolutional kernels [^32], or graph [^32]. PointNet can be considered as the pioneering work that learns point-wise feature using shared MLP. However, it fails to exploit the local spatial relationships in the point cloud [^33], [^34]. To address this problem, PointNet++ [^18] develops a hierarchical framework to aggregate information from local neighboring points. But it still treats each point independently [^35], [^36]. Motivated by the key idea of 2D convolution, some methods concentrate on defining point convolutional kernels [^35] or learn the kernel based on point positions [^37] to capture local information. However, these approaches may have high complexity during learning [^38]. In addition, other methods attempt to introduce --- graph neural network for local geometric topology aggregation. For example, DGCNN designs an Edge-Conv operator to update center point feature by aggregating features along the edges. RandLA-Net [^19] takes the random sampling strategy to remarkably improve the efficiency of point cloud processing and designs a local feature aggregation module to alleviate the information loss caused by random operations. SCF-Net [^21] presents a learnable SCF module for learning spatial context features from large-scale point clouds, containing three crucial components: a LPR block for learning rotation invariance, a DDAP model for automatic local feature learning, and a GCF module for learning the global context of each 3D point. FG-Net [^39] proposes the point correlation feature mining and geometry-aware modeling modules to understand large-scale point clouds. It integrates a residual learning strategy based on feature pyramids, a fast noise and outliers removal method and an inverse density point downsampling strategy to achieve a better balance between efficiency and performance. Theoretically and empirically, point-based approaches perform well on small point clouds. However, for LiDAR semantic segmentation, full $360^{\circ}$ large-scale scans limit the real-time processing capability of these methods due to inefficient computational complexity and memory requirement [^19]. Therefore, this kind of methods actually is less suitable for autonomous driving applications. Voxel-based methods first discretize the point cloud into 3D volumetric representation, where each point is associated with one corresponding voxel. Then, 3D CNNs framework are used to perform voxel-wise semantic label prediction [^14]. SEGCloud [^13] treat the coarse voxel label prediction as a intermediate stage in point cloud semantic segmentation task. Zhou et al. [^40] transformed the point cloud into a special cylindrical grid, and designed two novel CNN components. One is an asymmetric residual block to preserve the features related to cubic objects, while another is to context modeling to fuse multiple low-level contexts for high-level tensor model construction. VMNet [^41] combines the sparse voxel convolutions and graph convolutional networks to learn fused features from point clouds and meshes. (AF)2-S3Net [^42] introduces two well-designed attention blocks, namely multi-branch attention feature fusion module in the encoder and adaptive feature selection module in the decoder, to efficiently learn local and global contexts and to highlight detailed information. Although the voxel grid is one of ideal choice for irregular point cloud processing [^43], problem of sparsity, loss of granular information and introduction of discretization artifacts limit the performance of voxel-based methods [^44]. Image-based methods, here, also known as projection-based or range image-based methods, which project the 3D point cloud onto 2D image space either in multiple views [^45] or in Range-View representation [^46], follow by the application of 2D CNNs for semantic segmentation. The prior studies are usually built on transitional or improved encoder-decoder framework [^47], [^48], and Conditional Random Fields (CRF) [^49], [^50] is adopted as post-processing algorithm to render the final results. In subsequent works, RangeNet++ [^16], 3D-MiniNet [^51] utilizes a K-nearest neighbour post-processing instead of CRF to recover the consistent semantic information. SalsaNext [^52] performs an uncertainty-aware semantic segmentation, and introduces a new Lovász-Softmax loss [^53] to optimize the results. Triess et al. [^54] proposed a scan unfolding projection technique to solve the problems of systematic occlusions. KPRNet [^55] uses scan unfolding projection technology, together with a more robust ResNeXt101 [^56] backbone network, and KPConv to obtain more accurate 3D labels using learnable point-by-point components. MPF [^57] develops a multi-projection fusion framework which takes advantage of two independent 2D FCN models to process spherical and bird's-eye view projections. Then the segmentation results from both views are fused to reduce the information loss caused by single-projection methods. Experimental results demonstrate the competitive performance of proposed method. ### B. Vision Transformers More recently, immense progress have been made in the field of Natural Language Processing (NLP) with the advent of Transformer [^23]. Inspired by its great success, researchers from computer vision community attempt to explore the application of Transformer architectures to image based tasks [^58]. Detection Transformer (DETR) [^59] regards the detection problem as a set prediction task, which first learns initial feature maps using a CNN backbone. Then these features are fed into transformer-based encoder-decoder structure for object detection. Vision Transformer (ViT) [^24] is a pure Transformer model that achieves state-of-the-art performance on image classification. However, it is challenging for ViT to perform pixel-level dense prediction due to the following reasons. (1) ViT only outputs single-scale feature maps with low resolution. (2) ViT is computationally inefficient and parametric intensive when dealing with images, even those with smaller size. To handle these issues, many excellent works [^26], [^27], [^28], [^29], [^30], [^31] have been proposed. Swin Transformer [^31] constructs a hierarchical transformer model with shifted window based self-attention as fundamental component. The results on image classification, object detection and semantic segmentation are encouraging. In our paper, we adopt the core concept of Swin Transformer as the basic block of decoder part with skip connections for range images segmentation. ### III. METHODOLOGY Given a full-view 3D LiDAR scan, we aim to construct a function between input domain $\mathcal{P}$ of points and label space $\mathcal{Y}$ as $f: \mathcal{P} \longrightarrow \mathcal{Y}$ to perform efficient semantic segmentation. To this end, we first convert the point cloud segmentation into 2D range-view segmentation problem via spherical projection. Then, the Multi Residual Channel Interaction Attention Module (MRCIAM) processes each independent modal channel for channel-wise feature learning, and captures the intra-channel as well as inter-channel interactions. In the following encoder-decoder stage, we form a Residual Context Aggregation Module as encoder block to model local and global context, --- ![Image](images/7b2281ce9b962b10334d3981d350cca6d9c12a4d797c507a39bf7153fc4274a0.jpg)

3D Prediction

Fig. 2. The overall architecture of TransRVNet.

![Image](images/f034ba15b44ce330de9082b1053c34459d3d8cf1e3f0f9f90f81635b55bba091.jpg)

(a) Visualization of different channels of the input range image

![Image](images/e8b9d28bf6ccc7366390aa6aacb4433970d0d98af954c598230af30023628279.jpg)

(b) Quantitative analysis of the numerical distribution of the different channels.

Fig. 3. Visualization of a full LiDAR scan. From (a), we can see that different channels show some differential visualization results, which indicates that different modalities of the range image (xyz, depth, reflectivity) contain different information. From (b), the statistics also show that the different modalities show different distributions of values. (In each subplot of (b), the horizontal coordinate indicates the range of values, while the vertical coordinate indicates the proportion of pixel points within that region in the range image.)

while a Balanced Non-square-Transformer Module is designed as decoder component. Finally, we exploit the KNN post-processing operator to backproject the 2D range semantic to original points. Fig. 2 schematically shows the detailed architecture of the proposed TransRVNet. ### A. Range-View Representation for LiDAR Point Cloud Accurate and efficient semantic reasoning plays a key role in autonomous driving [^51]. Therefore, in order to obtain much higher segmentation performance and take full advantage of the benefits of 2D CNNs, we choose to deal with the spherical projection representation of the LIDAR point cloud. Actually, this representation, also known as range view or range image, is dense, compact, and computationally efficient [^60] 2D image which can be able to include the full 360° LiDAR scan. Specifically, following [^16], we find a mapping function $f: \mathbb{R}^3 \to \mathbb{R}^2$ to form an one-to-one correspondence between 3D point with coordinate $(x, y, z)$ and an pixel $(u, v)$ in the mapped range image of size $(H, W)$ (H and W represent the height and width of the generated range image). The spherical projection we used is formulated as, $$ u=\frac{1}{2}\left[1-arctan\left(y,x\right)\pi^{-1}\right]W $$ $$ v=\left[1-\left(a r c s i n\left(z,d^{-1}\right)+f_{u}\right)\frac{1}{f_{u}+f_{d}}\right]H $$ where $f = f_{up} + f_{down}$ refers to the sensor's vertical field-of-view. The depth d of each point is calculated as $d = \sqrt{x^{2} + y^{2} + z^{2}}$. The final projected range image can be denoted as a tensor with the shape of $(H, W, 5)$, where each pixel contains five channels $(x, y, z, d, r)$, r is the remission or reflection intensity information. Thought this spherical transformation, the point cloud segmentation problem is turned into image segmentation task. ### B. TransRVNet The pipeline of the proposed TransRVNet is illustrated in Figure 2. In the following subsections, we will describe the main components of the designed architecture in details. 1) Multi Residual Channel Interaction Attention Module: As shown in Figure 3, it can be reported that different channels contained in projection range image actually present different --- ![Image](images/296058de206735d5a53acc2bb0e5f1774cc66b9c29d8c9960a86e1f96a6d7c82.jpg)

(a) The Structure of MRCIAM on single channel

![Image](images/5290dc758badec317f63541dbbe1da57b171c13d75cbe6a28139327ac413d381.jpg)

(b) Overall Structure on three channels

Fig. 4. The Structure of Multi Residual Channel Interaction Attention Module. The individual features of the reflectivity, xyz, and depth channels are first extracted by the structure on the Top, and then fused by the structure on the Bottom Left. (ConvBNPReLU represents the basic feature extraction block consisting of 2D convolution with corresponding kernel size and dilation rate, batch normalization [^62] and PReLU activation function [^63]. The same description is used in the following figures.)

modal information, and their corresponding values follow different distributions, which means different channels make different contributions to the learned features. However, most of the prior approaches [^16], [^48], [^52] directly take the projected image as input, few works [^22], [^64] have noticed this phenomenon but lack more efficient feature extraction and channel interaction attention. Consequently, we decide to learn individual feature space for each independent channel with channel interaction, which is proved to be more effective and efficient. To achieve this end, we design a Multi Residual Channel Interaction Attention Module (MRCIAM) to capture multi-scale contextual information from each channel and aggregate these features, which consists of four essential blocks, as illustrated in Figure 4. Where the first and the third ones share the same structure, including three convolutions with different kernel sizes (e.g., $3 \times 3$, $5 \times 5$, $7 \times 7$) and a residual connections with a $1 \times 1$ convolution to capture and fuse local-to-global context. The second block is also a residual structure but with dilated convolution. Finally, we add a Spatial-Channel Attention (SCA) module adopted from the SA-NET [^61] to MRCIAM for modeling the pixel-level relationships in each channel and interactions/dependencies across different channels, i.e. intra-channel and inter-channel interactions. Specifically, the module groups channel dimensions into multiple sub-features and then integrates the complementary channels and spatial attention modules for each sub-feature using the Shuffle Unit. (Figure 4c shows the structure of SCA Block.) Finally, it achieves lightweight and efficient spatial and channel attention for the feature map. Overall, our MRCIAM operates on the depth/range channel, xyz channel and remission/reflectivity channel independently, and outputs local-to-global contextual features with spatial fine-grained information. 2) Network Architecture Design: The currently existing Transformer-based semantic segmentation works use either the Transformer as encoder and CNN as decoder [^25], [^31], [^65], [^66], or the pure Transformer structure [^67], [^68], [^69]. Although these models have achieved promising performance on several medical imaging datasets [^70], [^71], [^72] and the widely used ADE20k dataset [^73], from our experiments it can be found that these two kinds of structures are unsuitable for extracting features from range-image-style inputs, since it becomes more difficult to encode much larger patch tokens. In addition, it is too expensive to train these networks. Therefore, we decide to use our CNN-based Residual Context Aggregation Module (RCAM) as encoder and Transformer-based Balanced Non-square-Transformer Module (BNTM) as decoder. 3) Encoder: Specifically, the CNN-based Residual Context Aggregation Module (RCAM) includes the following components, a) Basic encoder block: In order to further capture more descriptive contextual features from channel-fused maps from MRCIAM, one simple to stack convolution operations with progressively enlarged kernel size to obtain much larger receptive field. However, this strategy usually brings remarkable increase of the number of parameters. Therefore, to reduce computational cost, we design a simple but effective encoder block based on dilated convolution and residual structure (shown in Figure 5). Here, we combine two $3 \times 3$ convolutions, one $3 \times 3$ dilation convolution with dilation rate 2, as well as a residual connection with $1 \times 1$ convolution for dimension reduction. Through this design, the encoder can make full use of multi-level multi-scale multi-channel information. b) Context aggregation module: Missing points, which we name missing noise (as illustrated in red-circled areas of GT image in Figure 3), have negative impact on performance of CNN models. This kind of noise commonly results from the limitation of sensors, mirror reflection, occlusion of the object itself, as well as spherical projection. To cope with this issue, we choose to use the Context Aggregation Module (CAM) (shown in Fig. 5) from SqueezeSegV2 [^47], and put it at the --- ![Image](images/60324205913e91d91c2f892da01b184e9557d3d2e851ef3cb2c1a2342e2ec553.jpg)

Fig. 5. The Basic Components of Encoder Module. Left: The Structure of Basic Encoder Block. Right: The Structure of Context Aggregation Module.

![Image](images/19aaf85f7cc6561ca17eb5427f5c04a9127950af973eb5e239fcb905be32188f.jpg)

Fig. 6. The Structure of Swin Transformer Block. Where LN denotes Layer Normalization [^74], (S)MSA denotes Multi-head Self Attention modules with Shifted or regular window configurations, respectively. More details refer to [^31].

end of each Basic Encoder Block to enhance the robustness of contextual features to specific noise. 4) Decoder: The decoder is based on our Balanced Non-square-Transformer Module, which is composed of the following sub-blocks. a) BN-transformer module: The standard Transformer architectures adopt the multi-head self attention (MSA) mechanism to obtain global relationships between a patch token and other tokens for image-based tasks. However, for large-resolution dense pixel-level prediction, Transformer model often has quadratic computational complexity with respect to image size [^31], which limits its performance. On the other hand, the fixed size of token makes Transformer unsuitable for computer vision tasks required to deal with visual objects with varying scales. To handle these issues, Liu et al. [^31] built the Swin Transformer with shifted window module MSA, MLP with GELU non-linearity, LayerNorm layer and residual connections to capture dependencies locally and reduce the computational complexity. Inspired by its tremendous success, we transfer the core idea of Swin architecture to our Transformer decoder component, as shown in Figure 6. The formulation can be summarized as, $$ Z_{l}^{\prime}=MSA(LN(Z_{l-1}))+Z_{l-1} $$ $$ Z_{l}=M L P(L N(Z_{l}^{\prime}))+Z_{l}^{\prime} $$ $$ Z_{l+1}^{\prime}=SMSA(LN(Z_{l}))+Z_{l} $$ $$ Z_{l+1}=M L P(L N(Z_{l+1}^{^{\prime}}))+Z_{l+1}^{^{\prime}} $$ where $Z'_{l}$ and $Z_{l}$ represent the outputs of the MSA module and the MLP module of the $l_{th}$ block, $Z'_{l+1}$ and $Z_{l+1}$ represent the outputs of the SMSA module and the MLP module of the next $l_{th}+1$ block, respectively. Here, it should be noted that the BN-Transformer Block we designed is different from the original Swin Transformer Block. Specifically, (1) we choose to design a non-square window based Transformer instead of a square version to ![Image](images/b4b1a22c24f83418008a2c08bdebfe0c2a7c70caa3294ce36829e9013adb4ece.jpg)

Fig. 7. The Structure of Balanced Non-square-Transformer Module.

![Image](images/97941a6a955c9a4ff6e29f9bc582a1051e501d7c08e7e5feac0d829041478268.jpg)

Fig. 8. The Structure of Conv Decoder Block.

accommodate the size imbalance of range image (since range images usually have much larger aspect ratio compared to common RGB images). (2) To enable much better feature learning, we stack four well-designed Transformer blocks after each upsampling layer instead of using different number of original Swin transformer blocks in each decoder module. (Section V-A and Section V-B present and discuss the details and implementations of these two components.) Based on the BN-Transformer Block above, our Balanced Non-square-Transformer Module is designed as follows (shown in Fig. 7). First, we upsample the input feature maps by a factor of 2, which then are concatenated with the feature maps from the corresponding encoder together to fuse multi-scale features. Finally, we use a linear layer to adjust the channel dimension. And our designed Transformer model takes the flatten features as input to perform decoding operation. The whole decoder stage consists of three BN-Transformer Modules with different scales. b) Conv decoder block: Although our BN-Transformer Modules have capability of achieving powerful performance, it is a little bit difficult for them to decode these feature maps with largest resolution. We, therefore, decide to put a convolution-based decoder block (shown in Figure 8) at the end of decoder pipeline, which includes three elements, an upsampling operation, Conv decoder, and a segmentation head. ### C. Loss Function In specific to our range-view based semantic segmentation task, actually there exists three challenging problems required to be alleviated. • The imbalanced category problem. - The problem of optimizing the intersection-over-union (IoU). - The ambiguous segmentation boundaries problem. --- Here, three different loss functions, i.e. weighted cross-entropy loss, Lovázs-Softmax loss and boundary loss, are used to supervise our model. First, the weighted cross-entropy loss can be formulated as, $$ L_{w c e}=-\sum_{c=1}^{C}(\frac{f_{t}}{f_{c}})^{i}\mathrm{y}_{c}l o g(\hat{\mathrm{y}}_{c}) $$ where $y_c$ and $\hat{y}_c$ denotes the ground truth and prediction, respectively. To address the class imbalance, we adopt median frequency class balance strategy from SegNet [^75]. And we attempt to incorporate the power operation to smooth the estimated class weights, $w_c = (\frac{f_t}{f_c})^i$, where $f_c$ is the frequency of class c, and $f_t$ is the median of all class frequencies. Second, to maximize the intersection-over-union (IoU) score, we chose the Lovázs-Softmax loss function [^53], represented as $L_{ls}$. Its formulation can be summarized as follows, $$ L_{l s}=\frac{1}{C}\sum_{c=1}^{C}\overline{{\triangle_{J_{n c}}}}\left(m\left(c\right)\right) $$ $$ m_{i}(c)=\begin{cases}{1-\hat{p}_{i}^{c}}&{{i f}c=p_{i}^{c}}\\ {\hat{p}_{i}^{c}}&{{o t h e r w i s e}}\\ \end{cases} $$ where $\overline{\triangle J_{nc}}$ indicates the Lovázs extension of the Jaccard index, $\hat{p}_i^c \in [0,1]$, $p_i^c \in [0,1]$ are the predicted probability and ground truth for the ith pixel of class c. The problem of boundary ambiguity between different objects/classes in the segmentation results often arises from downsampling and upsampling operations [^76], [^77], [^78]. And few studies focus on finding appropriate solution to this issue for range image based segmentation [^22]. Hence, motivated by [^1], we introduce a boundary loss function, which can be defined as, $$ L_{b d}(\hat{y},y)=1-\frac{2P^{c}R^{c}}{P^{c}+R^{c}} $$ where $P^{c}$ and $R^{c}$ denotes the precision and recall of the predicted boundary map $y_{pd}$ with respect to the ground truth $y_{gt}$ for class c. $\hat{y}$ and y indicate the prediction and ground truth for each class. We give the definition of boundaries as, $$ y_{g t}^{b}=p o o l(1-y_{g t},\theta_{0})-(1-y_{g t}), $$ $$ y_{p d}^{b}=p o o l(1-y_{p d},\theta_{0})-(1-y_{p d}) $$ Here pool(·) means max-pooling operation on a sliding window of size $\theta_0$. And in order to model vicious boundary, the $\theta_0$ should be set as small as possible. Figure 9 shows the boundary maps of different classes with $\theta_0 = 3$. Finally, our total loss is the weighted combination of these three loss functions, given by, $$ L=\lambda_{1}L_{w c e}+\lambda_{2}L_{l s}+\lambda_{3}L_{b d} $$ where $\lambda_1$, $\lambda_2$, and $\lambda_3$ are the corresponding weights to constrain the contributions of the weighted cross-entropy loss, Lovász-Softmax loss and boundary loss. In our experiments, we set i in $L_{wce}$ to be 0.5 and the weights $\lambda_1 = 1.0$, $\lambda_2 = 3.0$, $\lambda_3 = 1.0$. We will specifically study and analyze the effect of different loss items and their associated weights in Section V-D. ![Image](images/aeb37fde46f468699e03aab13cb9d7680c4e5319de5214baf91609d4e1238f9b.jpg)

Fig. 9. Visualization of the extracted boundaries. From the top to the bottom, the ground truth and the extracted boundaries of each different segmentation class are represented respectively.

### D. Post-Processing The prediction label re-projection from range image to 3D point cloud possibly comes with misclassification problem. The reason is that two or more LiDAR points might be associated with the same pixel. In order to cope with this issue, we employ a k-nearest neighbor (KNN) post-processing strategy to generate a new semantic label for each 3D point, where the closest neighbors are found in the subsampled point cloud instead of the complete scene using a sliding window technique [^16]. ### IV. EXPERIMENTS In this section, we present quantitative and qualitative evaluations of our TransRVNet on the large-scale SemanticKITTI [^7] and the newly proposed small-scale SemanticPOSS [^9] benchmarks with a single NVIDIA RTX 2080Ti GPU. In the following, we first give a brief description of the datasets and evaluation metric. Then, we provide the implementation details. Finally, by comparing with other state-of-the-art approaches, we further validate the effectiveness of our proposed network with quantitative analysis. ### A. Dataset SemanticKITTI dataset is constructed by providing dense point-wise semantic annotations for all $360^{\circ}$ scans in KITTI Odometry Benchmark [^90]. This datasets contains over 43,000 scans from 21 sequences, where over 21,000 scans of sequences 00 to 10 are used for training, sequence 08 is chosen as validation set, while the remaining scans from 11 to 21 for testing. SemanticPOSS dataset, collected by Peking University, is a small-scale much sparser benchmark with point-wise label provided. It consists of 2,988 diverse and complicated LiDAR scenes with large quantity of dynamic instances [^9]. For convenience, SemanticPOSS follows the same data format as SemanticKITTI, which is divided into 6 parts, with part 2 as testing set and the others as training set. ### B. Evaluation Metric To evaluate the performance of our TransRVNet and make a fair comparison with other state-of-the-art techniques, we use the standard mean Intersection over Union (mIoU) [^91] as our ---

TABLE I

THE MEAN IOU (%) COMPARISON ON SEMANTICKITTI TEST SET. NOTE THAT AN FPS ABOVE 10 IS CONSIDERED REAL-TIME. THIS IS BECAUSE THE ACQUISITION FREQUENCY OF THE VELODYNE HDL-64E [^6] LIDAR SENSOR IS 10 Hz

Category	Method	Input Size	Car	Bicycle	Motorcycle	Truck	Other-vehicle	Person	Bicyclist	Motorcyclist	Road	Parking	Sidewalk	Other-ground	Building	Fence	Vegetation	Trunk	Terrain	Pole	Traffic-sign	Mean IoU	FPS(Hz)
	Pointnet [^17]		46.3	1.3	0.3	0.1	0.8	0.2	0.2	0.0	61.6	15.8	35.7	1.4	41.4	12.9	21.0	4.6	17.6	2.4	3.7	14.6	7.0
	Pointnet++ [^18]		53.7	1.9	0.2	0.9	0.2	0.9	1.0	0.0	72.0	18.7	41.8	5.6	62.3	16.9	46.5	13.8	30.0	6.0	8.9	20.4	0.1
	TangentConv [^33]		86.8	1.3	12.7	11.6	10.2	17.1	20.2	0.5	82.9	15.2	61.7	9.0	82.8	44.2	75.5	42.5	55.5	30.2	22.2	35.9	0.3
	LatticeNet [^36]		92.9	16.6	22.2	26.6	21.4	35.6	43.0	46.0	90.0	59.4	74.1	22.0	88.2	58.8	81.7	63.6	63.1	51.9	48.4	52.9	7
Point-based	RandLa-Net [^19]	50K pts	94.2	26.0	25.8	40.1	38.9	49.2	48.2	7.2	90.7	60.3	73.7	20.4	86.9	56.3	81.4	61.3	66.8	49.2	47.7	53.9	22
	BAF-LAC [^79]		94.2	27.6	30.4	39.9	40.1	46.3	49.1	10.5	90.9	61.4	74.0	22.2	88.2	57.0	81.3	61.7	65.3	49.8	53.9	54.9	--
	Kpcov [^35]		96.0	30.2	42.5	33.4	44.3	61.5	61.6	11.8	88.8	61.3	72.7	31.6	90.5	64.2	84.8	69.2	69.1	56.4	47.4	58.8	0.8
	BAAF [^80]		95.4	31.8	35.5	48.7	46.7	49.5	55.7	53.0	90.9	62.2	74.4	23.6	89.8	60.8	82.7	63.4	67.9	53.7	52.0	59.9	5
	FG-NS [^81]																					53.9	--
Voxel-based	MinkNet [^44]	Voxel	94.3	23.1	26.2	26.1	36.7	43.1	36.4	7.9	91.1	63.8	69.7	29.3	92.7	57.1	83.7	68.4	64.7	57.3	60.1	54.3	--
	FSPCNN-lite [^83]		95.3	47.5	37.7	41.8	34.5	59.5	56.8	11.9	91.8	68.8	77.1	30.8	92.5	69.4	84.5	69.8	68.5	60.4	66.5	61.3	11
	SPVCNN [^83]		--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	58.5	8.1
	Cylinder3D V1 [^40]		96.1	54.2	47.6	38.6	45.0	65.1	63.5	13.6	91.2	62.2	75.2	18.7	89.6	61.6	85.4	69.7	69.3	62.6	64.7	61.8	3.2
	Cylinder3D V2 [^84]		97.1	67.6	64.0	59.0	58.6	73.9	67.9	36.0	91.4	65.1	75.5	32.3	91.0	66.5	85.4	71.8	68.5	62.6	65.6	67.8	7.6
	JSSC-Net [^85]		95.8	59.3	52.9	54.3	46.0	69.5	65.4	39.9	88.9	61.9	72.1	31.9	92.5	70.8	84.5	69.8	67.9	60.7	68.7	66.0	2.1
	APCS2 [^42]		94.5	64.4	86.8	39.2	41.1	80.7	80.4	74.3	91.3	68.8	72.5	53.5	87.8	63.2	70.6	68.5	51.5	64.5	44.5	69.4	--
	RPNet [^86]		97.6	68.4	68.7	44.2	61.1	75.9	74.4	73.4	93.4	70.3	80.7	33.3	93.5	72.1	86.5	75.1	71.7	64.8	61.4	70.3	6.0
Image-based	SqueezeSeg [^15]	64 \times 2048 px	68.8	16.0	4.1	3.3	3.6	12.9	13.1	0.9	85.4	26.9	54.3	4.5	57.4	29.0	60.0	24.3	53.7	17.5	24.5	29.5	66
	SqueezeSegV2 [^47]		81.8	18.5	17.9	13.4	14.0	20.1	25.1	3.9	88.6	45.8	67.6	17.7	73.7	41.1	71.8	35.8	60.2	20.2	36.3	39.7	50
	SqueezeSegV3 [^48]		92.5	38.7	36.5	29.6	33.0	45.6	46.2	20.1	91.7	63.4	74.8	26.4	89.0	59.4	82.0	58.7	65.4	49.6	58.9	55.9	6
	3D-MiniNet [^51]		64 \times 2048 px	90.5	42.3	42.1	28.5	29.4	47.8	44.1	14.5	91.6	64.2	74.5	25.4	89.4	60.8	82.8	60.8	48.0	56.6	55.8	28
	FPSNet [^64]		64 \times 2048 px	91.1	48.6	37.8	37.1	30	60.5	7.5	91.1	61.9	74.6	26.0	87.4	57.4	80.9	61.2	65.0	49.9	59.2	57.1	21
	PolarNet [^87]		480 \times 360 \times 32	93.8	30.1	22.9	28.5	43.2	40.8	5.6	90.8	61.7	74.4	21.7	90.0	61.3	84.0	65.5	67.8	51.8	54.3	57.5	16
	Hybrid CNN-LSTM [^88]		512 \times 512 \times 32	92.6	45.7	49.6	38.6	30.2	43.0	9.2	90.7	63.3	75.7	76.6	81.3	87.6	60.8	64.5	63.9	44.5	50.9	52.9	71
	TORNADONet [^89]		64 \times 2048 px	93.1	53.0	44.4	43.1	39.4	61.6	56.7	20.2	90.8	65.3	75.3	27.5	89.6	62.9	84.1	64.3	69.6	55.0	64.2	61.1
	TORNADONet-HiRes [^89]		128 \times 2048 px	94.2	55.7	48.1	40.0	38.2	63.6	60.1	34.9	89.7	66.3	74.5	28.7	91.3	65.6	85.0	71.5	58.0	65.9	63.1	7
	KPRNet [^55]		289 \times 4097 px	95.5	54.1	47.9	23.6	42.6	65.9	65.0	16.5	93.2	73.9	80.6	30.2	91.7	68.4	85.7	69.8	71.2	58.7	64.1	63.1
	MPF [^57]		64 \times 2048 px	93.4	30.2	38.3	26.1	28.5	48.1	46.1	18.1	90.6	62.3	74.5	30.6	88.5	59.7	83.5	59.7	69.2	49.7	58.1	55.5
	SalsaNext [^52]		64 \times 2048 px	92.7	28.2	30.5	26.9	25.2	42.5	45.5	9.5	90.5	64.7	74.3	32.0	88.3	59.0	83.4	56.6	69.8	46.0	54.9	29
	RangeNet++ [^16]		64 \times 1024 px	91.9	48.3	37.6	35.2	31.3	39.9	49.3	22.4	91.7	63.7	75.8	29.1	90.2	64.2	81.8	63.6	66.5	54.3	62.1	34
	TransRVNet(ours)		64 \times 2048 px	91.4	25.7	34.4	25.7	23.0	38.3	38.8	4.8	91.8	65.0	75.2	27.8	87.4	58.6	80.5	55.1	64.6	47.9	55.9	52.2
	TransRVNet(ours)		64 \times 1024 px	90.3	20.6	27.1	25.2	17.6	29.6	34.2	7.1	90.4	52.3	72.7	22.8	83.9	53.3	77.7	52.5	63.7	43.8	47.2	48.0

evaluation metric, which can be formalized as, $$ mIOU=\frac{1}{N}\sum_{c=1}^{N}\frac{TP_{c}}{TP_{c}+FP_{c}+FN_{c}} $$ where $TP_{c}$, $FP_{c}$ and $FN_{c}$ represent the number of true positive, false positive, and false negative point predictions for class c, respectively. ### C. Implementation Details We implement our TransRVNet architecture based on PyTorch framework. And we use average pooling of kernel size 3 for downsampling and the PixelShuffle [^92] method for upsampling. All models are trained using AdamW optimizer with weight decay of 0.0001. During training, random rotation, random dropping points, and random sign inverting for X and Y values with a probability of 0.5 are applied to augment the input point cloud to avoid overfitting. Specifically, As for SemanticKITTI dataset [^7], we train our model for 30 epochs with a batch size 6, and a one-cycle learning rate policy. The maximum learning rate is set to 0.002 and then is decayed to 0.0002 at 30 epochs. The dropout probability is 0.2. We set the height and width of input range images to H = 64 and W = 1,024, respectively. The window size for our Transformer block is set as [^4] [^64]. For the KNN post-processing stage, we use window size of 7 for neighbor search, with k=7, $\sigma=1$, and a cutoff of 2 m. For SemanticPOSS dataset [^9], we use the same learning rate strategy as SemanticKITTI, but with a difference that two cycles are used. We train our TransRVNet for 60 epochs using a batch size of 4. The dropout probability is set to be 0.2. The height and width of input range images are $H = 64$, W = 1,600. The corresponding window size of Transformer is [^4] [^50]. In the KNN post-processing stage, we exploit the same settings as SemanticKITTI. ### D. Results 1) Quantitative Results on SemanticKITTI: The quantitative comparisons with the state-of-the-art point-wise and image-based approaches on SemanticKITTI are reported in Table I. From these experimental results, we can obtain the following observations, - Our proposed TransRVNet achieves superior performance (62.8%) at $64 \times 1024$ px inputs, outperforming not only most image-based methods at low resolutions, but also all point-based methods. Moreover, compared to voxel-based methods, our method achieves a better balance between accuracy and computational complexity. - The TransRVNet noticeably surpasses the counterpart image-based baseline RangeNet++ (48.0%) by a significant margin, achieving an improvement of 14.8% mIoU score, and it also improves another version of RangeNet++ (52.2%) using range image of $64 \times 2048$ as input by +10.6% mIoU value. Compared to the more optimal methods TORNADONet [^89] and KPRNet [^55], we obtain fairly competitive results (62.8 mIoU vs. 63.1 mIoU) with much smaller input sizes ( $64 \times 1024$ px vs. $128 \times 2048$ px vs. $289 \times 4097$ px) while maintaining a high inference speed (26 fps vs. 4 fps vs. 0.3 fps). 2) Quantitative Results on SemanticPOSS: Table II present the mean IoU comparison with all the referred works on Sequence 02 SemanticPOSS [^9] dataset. It can be clearly reported that (1) TransRVNet obtains a leading performance on test sequences (a mIoU of 49.3%), which significantly ---

TABLE II

QUANTITATIVE COMPARISON WITH RECENT WORKS ON SEQUENCE 02 OF SEMANTICPOSS TEST SET USING MEAN IOU (%)

	Person	Rider	Car	Truck	Plants	Traffic-sign	Pole	Trashcan	Building	Cone/Stone	Fence	Bike	Ground	mIOU
SqueezeSeg [^15]	14.2	1.0	13.2	10.4	28.0	5.1	5.7	2.3	43.6	0.2	15.6	31.0	75.0	18.9
SqueezeSeg + CRF [^15]	3.8	0.6	6.7	4.0	2.5	9.1	1.3	0.4	37.1	0.2	8.4	18.5	72.1	12.9
SqueezeSegV2 [^47]	48.0	9.4	48.5	11.3	50.1	6.7	6.2	14.8	60.4	5.2	22.1	36.1	71.3	30.0
SqueezeSegV2 + CRF [^47]	43.9	7.1	47.9	18.4	40.9	4.8	2.8	7.4	57.5	0.6	12.0	35.3	71.3	26.9
RangeNet53 [^16]	55.7	4.5	34.4	13.7	57.5	3.7	6.6	23.3	64.9	6.1	22.2	28.3	72.9	30.3
RangeNet53 + KNN [^16]	57.3	4.6	35.0	14.1	58.3	3.9	6.9	24.1	66.1	6.6	23.4	28.6	73.5	30.9
MINet [^22]	61.8	12.0	63.3	22.2	68.1	16.3	29.3	28.5	74.6	25.9	31.7	44.5	76.4	42.7
MINet + KNN [^22]	62.4	12.1	63.8	22.3	68.6	16.7	30.1	28.9	75.1	28.6	32.2	44.9	76.3	43.2
TransRVNet	77.4	25.7	77.3	30.0	72.7	14.5	28.6	51.1	79.1	23.8	49.7	54.2	79.9	51.1
TransRVNet + KNN	77.9	26.1	77.8	30.1	73.1	14.6	29.1	53.5	79.8	24.7	49.5	53.8	78.8	51.5

promotes the previous state-of-the-art RangeNet++ [^16] and MINet [^22] by 20.6%, 8.3%, respectively. (2) All approaches have a much lower mIoU score due to the following reason that compared with SemanticKITTI, SemanticPOSS is a much smaller and more sparse dataset. To better understand the proposed TransRVNet, we carry out ablation studies to discuss the contributions of important designed techniques (including network architecture design, choice of window size, role of MRCIAM and CAM module) made to segmentation performance. For all the ablation experiments, we use Sequence 08 on SemanticKITTI dataset for testing, and the remaining sequences for training. Here, for efficient training and evaluation, we choose the similar setup as SqueezeSegV3 [^48]. The size of the input range image is set to $64 \times 512$, and the accuracy of the evaluation is chosen to be listed directly on the projected 2D images rather than on the original 3D points. ### V. ABLATION STUDY ### A. Network Architecture In order to investigate the effectiveness of our CNN-Transformer architecture on segmentation performance, we perform the following comparisons, (1) we design two Transformer-CNN frameworks based on TransRVNet, where we replace all the Transformer decoder blocks by our Conv Decoder Block, and use the high-performance Swin-T and ViT models as encoder with the following hyper-parameter settings. - We set the query dimension of each head in multi-head self-attention to 32, and integrate 4 expansion layers in MLP. • For Swin-T: C = 96, layer numbers = {2, 2, 6, 2} • For ViT: C = 64, layer numbers = {4, 4, 4, 4} Where C denotes the number of channels in the first stage. layer numbers refers to the number of Transformer blocks in each encoder module. (2) We remove MRCIAM from the ViT-CNN model. Table III lists the mIoU comparisons between three Transformer-CNN variants and CNN-Transformer structures. From this, it can be obviously observed that (1) Our CNN-BN-Transformer model achieves the best performance in terms of

TABLE III

IMPACT OF NETWORK ARCHITECTURE ON SEMANTICKITTI VALIDATION SET

Architecture	mIoU	Params
Swin-T-CNN (w/o MRCIAM)	46.3	34.40 M
ViT-CNN (w/o MRCIAM)	47.5	20.62 M
ViT-CNN (w MRCIAM)	53.0	20.86 M
CNN-BN-Transformer	58.1	26.52 M

TABLE IV

IMPACT OF WINDOW SIZE ON SEMANTICKITTI VALIDATION SET

Window Size	mIoU	FLOPs	GPU Memory (Batchsize=8)
[^4] [^4]	56.9	33.15G	7,701 MiB
[^4] [^8]	57.1	33.28G	7,881 MiB
[^4] [^16]	57.8	33.53G	8,153 MiB
[^4] [^32]	58.1	34.03G	8,757 MiB
[^4] [^64]	57.1	35.71G	10,513 MiB

accuracy, which is much more suitable for LiDAR semantic segmentation than Transformer-CNN framework. (2) The introduction of MRCIAM leads to a significant improvement of mIoU of 5.5% over the counterpart without MRCIAM. ### B. Impact of Window Size As discussed in Section III-B.4, the non-square shifted window scheme is a key design element for our TransRVNet. Therefore, the window size actually have a great impact on not only the segmentation performance, but also the parameter and computation complexity. To verify its effect on segmentation result, we provide quantitative comparison of TransRVNet using different window size in Table IV. As we expect, (1) the semantic segmentation performance increases with the increase of window size. This is mainly because larger window commonly corresponds to larger respective field, which means we can aggregate much richer context information. However, larger window usually leads to higher memory cost. (2) Compared with square window, non-square window yields much better performance. For example, windows of size [^4] [^8], [^4] [^16], [^4] [^32] improve the mIoU over 0.2%, 0.9%, and 1.2% against the counterpart with size [^4] [^4]. This is largely attributed to the fact that the width of range image is far larger than its height, which we define as size imbalance problem. (3) It is worth noting that when the window size is increased to [^4] [^64], although the receptive field of our model is enlarged, the ---

TABLE V

ANALYSIS OF IMPACT OF WINDOW SIZE ON SEMANTICKITTI VALIDATION SET

Window Size	Car	Bicycle	Motorcycle	Truck	Other-vehicle	Person	Bicyclist	Motorcyclist	Road	Parking	Sidewalk	Other-ground	Building	Fence	Vegetation	Trunk	Terrain	Pole	Traffic-sign
Square Window [^4] [^4]	94.3	36.1	43.9	81.7	33.5	43.2	69.4	0.0	94.9	48.3	81.5	0.3	84.9	57.9	84.6	57.2	71.0	62.3	36.5
Non-Square Window [^4] [^32]	94.5	44.2	48.1	81.7	34.8	49.8	73.7	0.0	94.8	48.2	81.3	0.7	83.2	51.6	85.4	59.0	73.2	62.7	37.2
	+0.2	+8.1	+4.2	0.0	+1.3	+5.6	+4.3	0.0	-0.1	-0.1	-0.2	+0.4	-0.3	-6.3	+0.8	+1.8	+2.2	+0.4	+0.7

![Image](images/dbe662f69ddee0531cdd3e85cdbc9c0dfb06ea0283fe1b45103657cdcec01af3.jpg)

Fig. 10. Qualitative analysis on SemanticKITTI validation set. In each example, visualizations of the input range channel, ground truth, predictions for the [^4] [^32] Non-Square Window Model, and predictions for the [^4] [^4] Square Window Model are shown from top to bottom. (The right side of each example shows a enlarged view of the red marked box on the left.)

TABLE VI

ANALYSIS OF MRCIAM MODULE ON SEMANTICKITTI VALIDATION SET

Architecture	Car	Bicycle	Motorcycle	Truck	Other-vehicle	Person	Bicyclist	Motorcyclist	Road	Parking	Sidewalk	Other-ground	Building	Fence	Vegetation	Trunk	Terrain	Pole	Traffic-sign
Full Model w/o MRCIAM	94.8	37.1	47.2	80.1	46.6	35.2	72.4	0.0	94.3	47.0	80.9	1.0	82.1	50.7	83.6	56.6	71.1	61.1	33.7
Full Model	94.5	44.2	48.1	81.7	34.8	49.8	73.7	0.0	94.8	48.2	81.3	0.7	83.2	51.6	85.4	59.0	73.2	62.7	37.2
	-0.3	+7.1	+0.9	+1.6	-11.8	+14.6	+1.3	0.0	+0.5	+0.8	+0.4	-0.3	+1.1	+0.9	+1.8	+1.4	+2.1	+1.6	+3.5

performance degrades. This is probably due to the fact that the limited parameters and model complexity make it rather more difficult to perform adequate feature decoding inside the oversized window. Therefore, for $64 \times 512$ inputs, a window size of [^4] [^32] contributes to gaining global self-attention on the smallest feature map (After $16 \times$ downsampling, the feature map size becomes $4 \times 32$), which can provide a good balance between receptive field and computational complexity. (4) From these results It can be concluded that non-square window is proved to be an ideal and reasonable choice for our Transformer model. Table V illustrates the specific effect of window size on each segmentation class. It can be clearly seen that larger window size significantly improves the segmentation performance on most classes, especially for smaller classes (e.g., bicycle), motorcycle, person, bicyclist). This may be because larger receptive field can help the model learn more high-dimensional semantic information, and global self-attention can work much better on the smallest feature maps. Fig 10 shows the qualitative comparison of the effect of different window sizes on the segmentation results. From left to right we can see that our model with [^4] [^32] non-square shifted window achieves much better results on all three classes, i.e. Bicyclist, Bicycle and Motorcycle. ### C. Effects of the MRCIAM and RCAM Module Table VII reports the extensive ablation studies of our proposed MRCIAM and RCAM modules. We add MRCIAM and RCAM to the backbone one by one to validate the effectiveness of each module. From these experimental results, we have the following observation, (1) with only MRCIAM and only RCAM block, the segmentation performance of our TransRVNet is boosted to 57.2%, 56.6%, respectively,

TABLE VII

IMPACT OF MRCIAM MODULE AND CAM MODULE ON SEMANTICKITTI VALIDATION SET

	Backbone	MRCIAM	RCAM	mIOU
Architecture	✓			55.0
	✓		✓	56.6
	✓	✓		57.2
	✓	✓	✓	58.1

outperforming the baseline by 2.2% and 1.6%. (2) Combining MRCIAM and RCAM together, our model achieves a significant improvement of 3.1%. To further explore the superiority of MRCIAM, we present the quantitative results on all classes of SemanticKITTI dataset in Table VI. With MRCIAM, our TransRVNet makes evident improvements on almost all classes in terms of class mIoU. We attribute this to the powerful ability of MRCIAM to capture multi-scale channel-wise context and model inter-channel, intra-channel interactions. Fig. 11 shows the qualitative comparison of the segmentation results on class of person, which reports that our network with MRCIAM can yield better segmentation results on different input frames. ### D. Effect of Weight Parameters for Loss Functions To investigate the impact of different loss items on model optimization, we first fix these three weight coefficients to 1.0 and conduct the experiments. From Table VIII, it can be clearly reported that all there loss items play important roles in performance improvement. Then, we begin to study the effect of different choices of these weighting factors. Since the cross-entropy loss is the most fundamental item, we set its weight $\lambda_{1}$ to be 1.0 in all the experiments. --- ![Image](images/f2e5d58312d778240edaa881db83a6b999c9403fe41cbbc075b6e25c264025f7.jpg)

Fig. 11. Qualitative analysis on SemanticKITTI validation set. In each example, visualizations of the input range channel, ground truth, predictions of Full Model, and prediction of Full Model w/o MRCIAM are shown from top to bottom. (The right side of each example shows a enlarged view of the red marked box on the left.)

TABLE VIII

EFFECT OF DIFFERENT LOSSES ON SEMANTICKITTI VALIDATION SET

Cross-Entropy Loss	Lovázz-Softmax Loss	Boundary Loss	mIoU
✓			50.1
✓	✓		57.3
✓		✓	55.1
	✓	✓	52.0
✓	✓	✓	59.1

TABLE IX

EFFECT OF $\lambda_{2}$

	0.5	1.0	2.0	2.5	3.0	3.5	4.0	5.0
mIoU	57.8	59.1	59.3	60.1	60.2	59.6	59.2	58.8

TABLE X

EFFECT OF $\lambda_{3}$, AND i IN $L_{wc}\epsilon$

	0.5	1.0	1.5	2.0
mIoU	59.7	60.2	59.7	59.5

Effect of $\lambda_{3}$

	0.25	0.5	0.75	1.0
mIoU	60.2	60.5	59.8	60.2

Effect of i in $L_{wce}$ First, we investigated the effect of the weighting factor $\lambda_2$. We set $\lambda_1 = 1.0$, $\lambda_3 = 1.0$ and $i = 1.0$ to train our model with $\lambda_2 \in \{0.5, 1.0, 2.0, 2.5, 3.0, 3.5, 4.0, 5.0\}$. From Table IX, we see that our model achieves the best performance at $\lambda_2 = 3.0$. Second, we investigated the effect of the weighting factor $\lambda_3$. We set $\lambda_1 = 1.0$, $\lambda_2 = 3.0$ and $i = 1.0$ to train our network with $\lambda_3 \in \{0.5, 1.0, 1.5, 2.0\}$. The results from Table X indicate that the model $\lambda_3 = 1.0$ is the optimal choice for achievement of the best performance. Finally, we investigated the effect of weighting factor i. We use $\lambda_1 = 1.0$, $\lambda_2 = 3.0$ and $\lambda_3 = 1.0$ to train our framework with $i \in \{0.25, 0.5, 0.75, 1.0\}$. It can be stated that the best result is obtained when $i = 0.5$. Therefore, we set $\lambda_1$, $\lambda_2$, $\lambda_3$, i to 1.0, 3.0, 1.0, and 0.5 in our experiments. ### VI. CONCLUSION In this paper, we transfer the encouraging performance of Transformer in NLP to image-based LiDAR semantic segmentation task by constructing a CNN-Transformer architecture, named TransRVNet. We first present a Multi Residual Channel Interaction Attention Module to aggregate contextual information and model inter-channel, intra-channel interactions. Then, a Residual Context Aggregation Module is used as encoder block to model multi-level multi-scale features. Finally, we design a Balanced Non-square-Transformer Module as decoder element to capture local-global dependencies. During training, we weight the weighted cross-entropy loss, Lovász-Softmax loss and boundary loss to supervise our network. Extensive experimental results on SemanticKITTI and SemanticPOSS demonstrate that our TransRVNet outperforms several state-of-the-art point-based, voxel-based and image-based models in terms of mIoU, achieving promising performance. And ablation studies further verify the effectiveness of our proposed learning modules. Limitations: In our TransRVNet, the window size is a hyperparameter that needs to be chosen. Although based on the prior knowledge, we can define it as the size of the minimum feature map to obtain global self-attention. However, the optimal setting for window size may require more searching and testing for different model sizes, input sizes, and different datasets. In addition, the traditional CNN models can handle different scales of input very well. For Transformer-based ViT model, however, it is limited by the setting of position embedding. For different input sizes, the learned relative position bias may have to be fine-tuned by bi-cubic interpolation to accommodate different window sizes. ## REFERENCES [^1]: R. Razani, R. Cheng, E. Taghavi, and L. Bingbing, "Lite-HDSeg: LiDAR semantic segmentation using lite harmonic dense convolutions," 2021, arXiv:2103.08852. [^2]: D. Feng et al., “Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 3, pp. 1341–1360, Mar. 2021. [^3]: J. Mei, B. Gao, D. Xu, W. Yao, X. Zhao, and H. Zhao, “Semantic segmentation of 3D LiDAR data in dynamic scene using semi-supervised learning,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 6, pp. 2496–2509, Jun. 2020. [^4]: L. Ma, Y. Li, J. Li, W. Tan, Y. Yu, and M. A. Chapman, “Multi-scale point-wise convolutional neural networks for 3D object segmentation from LDiAR point clouds in large-scale environments,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 2, pp. 821–836, Feb. 2021. [^5]: B. Gao, Y. Pan, C. Li, S. Geng, and H. Zhao, “Are we hungry for 3D LiDAR data for semantic segmentation? A survey of datasets and methods,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 6063–6081, Jul. 2022. [^6]: Velodyne HDL-64 LiDAR Sensor. Accessed: Jul. 30, 2021. [Online]. Available: [^7]: J. Behley et al., “SemanticKITTI: A dataset for semantic scene understanding of LiDAR sequences,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., Oct. 2019, pp. 9297–9307. [^8]: Hesai Pandar40 40-Channel Mechanical LiDAR. Accessed: Jul. 30, 2021. [Online]. Available: [^9]: Y. Pan, B. Gao, J. Mei, S. Geng, C. Li, and H. Zhao, “SemanticPOSS: A point cloud dataset with large quantity of dynamic instances,” in Proc. IEEE Intell. Vehicles Symp. (IV), Oct. 2020, pp. 687–693. [^10]: X. Zhang et al., “DCNAS: Densely connected neural architecture search for semantic image segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jul. 2021, pp. 13956–13967. [^11]: H. Lei, N. Akhtar, and A. Mian, “Spherical kernel for efficient graph convolution on 3D point clouds,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3664–3680, Oct. 2021. [^12]: X.-F. Han, H. Laga, and M. Bennamoun, "Image-based 3D object reconstruction: State-of-the-art and trends in the deep learning era," IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 5, pp. 1578–1604, May 2021. --- [^13]: L. Tchapmi, C. Choy, I. Armeni, J. Gwak, and S. Savarese, “SEGCloud: Semantic segmentation of 3D point clouds,” in Proc. Int. Conf. 3D Vis. (3DV), Oct. 2017, pp. 537–547. [^14]: B. Graham, M. Engelcke, and L. V. D. Maaten, “3D semantic segmentation with submanifold sparse convolutional networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 9224–9232. [^15]: B. Wu, A. Wan, X. Yue, and K. Keutzer, “SqueezeSeg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2018, pp. 1887–1893. [^16]: A. Milioto, I. Vizzo, J. Behley, and C. Stachniss, “RangeNet++: Fast and accurate LiDAR semantic segmentation,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nov. 2019, pp. 4213–4220. [^17]: R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 652–660. [^18]: C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “PointNet++: Deep hierarchical feature learning on point sets in a metric space,” 2017, arXiv:1706.02413. [^19]: Q. Hu et al., “RandLA-Net: Efficient semantic segmentation of large-scale point clouds,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2020, pp. 11108–11117. [^20]: T. He, C. Shen, and A. van den Hengel, "DyCo3D: Robust instance segmentation of 3D point clouds through dynamic convolution," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 354–363. [^21]: S. Fan, Q. Dong, F. Zhu, Y. Lv, P. Ye, and F.-Y. Wang, "SCF-Net: Learning spatial contextual features for large-scale point cloud segmentation," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 14504–14513. [^22]: S. Li, X. Chen, Y. Liu, D. Dai, C. Stachniss, and J. Gall, “Multi-scale interaction for real-time LiDAR data segmentation on an embedded platform,” 2020, arXiv:2008.09162. [^23]: A. Vaswani et al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 5998–6008. [^24]: A. Dosovitskiy et al., “An image is worth $16 \times 16$ words: Transformers for image recognition at scale,” 2020, arXiv:2010.11929. [^25]: S. Zheng et al., “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jul. 2021, pp. 6881–6890. [^26]: L. Yuan et al., “Tokens-to-token ViT: Training vision transformers from scratch on ImageNet,” 2021, arXiv:2101.11986. [^27]: H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” 2020, arXiv:2012.12877. [^28]: W. Wang et al., “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” 2021, arXiv:2102.12122. [^29]: K. Han, A. Xiao, E. Wu, J. Guo, C. Xu, and Y. Wang, "Transformer in transformer," 2021, arXiv:2103.00112. [^30]: H. Wu et al., “CVT: Introducing convolutions to vision transformers,” 2021, arXiv:2103.15808. [^31]: Z. Liu et al., “Swin transformer: Hierarchical vision transformer using shifted windows,” 2021, arXiv:2103.14030. [^32]: Z. Liu, H. Hu, Y. Cao, Z. Zhang, and X. Tong, “A closer look at local aggregation operators in point cloud analysis,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2020, pp. 326–342. [^33]: M. Tatarchenko, J. Park, V. Koltun, and Q.-Y. Zhou, "Tangent convolutions for dense prediction in 3D," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 3887–3896. [^34]: H. Su et al., “SPLATNet: Sparse lattice networks for point cloud processing,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 2530–2539. [^35]: H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. Guibas, "KPConv: Flexible and deformable convolution for point clouds," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2019, pp. 6411–6420. [^36]: R. A. Rosu, P. Schütt, J. Quenzel, and S. Behnke, "LatticeNet: Fast point cloud segmentation using permutohedral lattices," 2019, arXiv:1912.05905. [^37]: Y. Liu, B. Fan, S. Xiang, and C. Pan, "Relation-shape convolutional neural network for point cloud analysis," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 8895–8904. [^38]: M. Xu, R. Ding, H. Zhao, and X. Qi, "PAConv: Position adaptive convolution with dynamic kernel assembling on point clouds," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 3173–3182. [^39]: K. Liu, Z. Gao, F. Lin, and B. M. Chen, “FG-Net: Fast large-scale LiDAR point clouds understanding network leveraging correlated feature mining and geometric-aware modelling,” 2020, arXiv:2012.09439. [^40]: H. Zhou et al., “Cylinder3D: An effective 3D framework for driving-scene LiDAR semantic segmentation,” 2020, arXiv:2008.01550. [^41]: Z. Hu et al., “VMNet: Voxel-mesh network for geodesic-aware 3D semantic segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., Oct. 2021, pp. 15488–15498. [^42]: R. Cheng, R. Razani, E. Taghavi, E. Li, and B. Liu, "(AF)2-S3Net: Attentive feature fusion with adaptive feature selection for sparse semantic segmentation network," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 12547–12556. [^43]: B. Zoph and Q. V. Le, “Neural architecture search with reinforcement learning,” 2016, arXiv:1611.01578. [^44]: C. Choy, J. Gwak, and S. Savarese, "4D spatio-temporal ConvNets: Minkowski convolutional neural networks," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 3075–3084. [^45]: S. Xie, S. Liu, Z. Chen, and Z. Tu, “Attentional ShapeContextNet for point cloud recognition,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 4606–4615. [^46]: F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size,” 2016, arXiv:1602.07360. [^47]: B. Wu, X. Zhou, S. Zhao, X. Yue, and K. Keutzer, “SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud,” in Proc. Int. Conf. Robot. Autom. (ICRA), May 2019, pp. 4376–4382. [^48]: C. Xu et al., “SqueezeSegV3: Spatially-adaptive convolution for efficient point-cloud segmentation,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2020, pp. 1–19. [^49]: J. Lafferty, A. McCallum, and F. C. N. Pereira, “Conditional random fields: Probabilistic models for segmenting and labeling sequence data,” in Proc. Int. Conf. Mach. Learn., 2001, pp. 282–289. [^50]: S. Zheng et al., “Conditional random fields as recurrent neural networks,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 1529–1537. [^51]: I. Alonso, L. Riazuelo, L. Montesano, and A. C. Murillo, “3D-MiniNet: Learning a 2D representation from point clouds for fast and efficient 3D LiDAR semantic segmentation,” IEEE Robot. Automat. Lett., vol. 5, no. 4, pp. 5432–5439, Jul. 2020. [^52]: T. Cortinhal, G. Tzelepis, and E. E. Aksoy, “SalsaNext: Fast, uncertainty-aware semantic segmentation of LiDAR point clouds,” in Proc. Int. Symp. Vis. Comput. Cham, Switzerland: Springer, 2020, pp. 207–222. [^53]: M. Berman, A. R. Triki, and M. B. Blaschko, “The Lovász-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 4413–4421. [^54]: L. T. Triess, D. Peter, C. B. Rist, and J. M. Zollner, “Scan-based semantic segmentation of LiDAR point clouds: An experimental study,” in Proc. IEEE Intell. Vehicles Symp. (IV), Oct. 2020, pp. 1116–1121. [^55]: D. Kochanov, F. K. Nejadasl, and O. Booij, “KPRNet: Improving projection-based LiDAR semantic segmentation,” 2020, arXiv:2007.12668. [^56]: S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 1492–1500. [^57]: Y. Ali Alnaggar, M. Afifi, K. Amer, and M. ElHelw, “Multi projection fusion for real-time semantic segmentation of 3D LiDAR point clouds,” in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2021, pp. 1800–1809. [^58]: X. Wang, R. Girshick, A. Gupta, and K. He, "Non-local neural networks," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 7794–7803. [^59]: N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2020, pp. 213–229. [^60]: E. E. Aksoy, S. Baci, and S. Cavdar, “SalsaNet: Fast road and vehicle segmentation in LiDAR point clouds for autonomous driving,” in Proc. IEEE Intell. Vehicles Symp., Oct. 2020, pp. 926–932. --- [^61]: Q.-L. Zhang and Y.-B. Yang, "SA-Net: Shuffle attention for deep convolutional neural networks," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Jun. 2021, pp. 2235–2239. [^62]: S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. Int. Conf. Mach. Learn., 2015, pp. 448–456. [^63]: K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 1026–1034. [^64]: A. Xiao, X. Yang, S. Lu, D. Guan, and J. Huang, "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation," ISPRS J. Photogramm. Remote Sens., vol. 176, pp. 237–249, Jun. 2021. [^65]: J. Chen et al., “TransUNet: Transformers make strong encoders for medical image segmentation,” 2021, arXiv:2102.04306. [^66]: A. Hatamizadeh, D. Yang, H. Roth, and D. Xu, “UNETR: Transformers for 3D medical image segmentation,” 2021, arXiv:2103.10504. [^67]: E. Xie et al., “Segmenting transparent object in the wild with transformer,” 2021, arXiv:2101.08461. [^68]: S. Wu, T. Wu, F. Lin, S. Tian, and G. Guo, “Fully transformer networks for semantic image segmentation,” 2021, arXiv:2106.04108. [^69]: H. Cao et al., “Swin-Unet: Unet-like pure transformer for medical image segmentation,” 2021, arXiv:2105.05537. [^70]: A. L. Simpson et al., “A large annotated medical image dataset for the development and evaluation of segmentation algorithms,” 2019, arXiv:1902.09063. [^71]: N. Kumar et al., “A multi-organ nucleus segmentation challenge,” IEEE Trans. Med. Imag., vol. 39, no. 5, pp. 1380–1391, May 2020. [^72]: K. Sirinukunwattana et al., “Gland segmentation in colon histology images: The glas challenge contest,” Med. Image Anal., vol. 35, pp. 489–502, Jan. 2017. [^73]: B. Zhou et al., “Semantic understanding of scenes through the ADE20K dataset,” Int. J. Comput. Vis., vol. 127, no. 3, pp. 302–321, Mar. 2019. [^74]: J. L. Ba, J. R. Kiros, and G. E. Hinton, "Layer normalization," 2016, arXiv:1607.06450. [^75]: V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A deep convolutional encoder–decoder architecture for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 12, pp. 2481–2495, Dec. 2017. [^76]: Y. Zhu et al., “Improving semantic segmentation via video propagation and label relaxation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2019, pp. 8856–8865. [^77]: X. Li et al., “Improving semantic segmentation via decoupled body and edge supervision,” in Proc. 16th Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2020, pp. 435–452. [^78]: A. Bokhovkin and E. Burnaev, “Boundary loss for remote sensing imagery semantic segmentation,” in Proc. Int. Symp. Neural Netw. Cham, Switzerland: Springer, 2019, pp. 388–401. [^79]: H. Shuai, X. Xu, and Q. Liu, “Backward attentive fusing network with local aggregation classifier for 3D point cloud semantic segmentation,” IEEE Trans. Image Process., vol. 30, pp. 4973–4984, 2021. [^80]: F. Zhang, J. Fang, B. Wah, and P. Torr, “Deep FusionNet for point cloud semantic segmentation,” in Proc. 16th Eur. Conf. Comput. Vis. Glasgow, U.K.: Springer, Aug. 2020, pp. 644–663. [^81]: K. Liu, Z. Gao, F. Lin, and B. M. Chen, “FG-Net: A fast and accurate framework for large-scale LiDAR point cloud understanding,” IEEE Trans. Cybern., vol. 53, no. 1, pp. 553–564, Jan. 2023. [^82]: S. Qiu, S. Anwar, and N. Barnes, “Semantic segmentation for real point cloud scenes via bilateral augmentation and adaptive fusion,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 1757–1767. [^83]: H. Tang et al., “Searching efficient 3D architectures with sparse point-voxel convolution,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2020, pp. 685–702. [^84]: X. Zhu et al., “Cylindrical and asymmetrical 3D convolution networks for LiDAR segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jul. 2021, pp. 9939–9948. [^85]: X. Yan et al., “Sparse single sweep LiDAR point cloud segmentation via learning contextual shape priors from scene completion,” 2020, arXiv:2012.03762. [^86]: J. Xu, R. Zhang, J. Dou, Y. Zhu, J. Sun, and S. Pu, "RPVNet: A deep and efficient range-point-voxel fusion network for LiDAR point cloud segmentation," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 16024–16033. [^87]: Y. Zhang et al., “PolarNet: An improved grid representation for online LiDAR point clouds semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2020, pp. 9601–9610. [^88]: S. Wen, T. Wang, and S. Tao, “Hybrid CNN-LSTM architecture for LiDAR point clouds semantic segmentation,” IEEE Robot. Autom. Lett., vol. 7, no. 3, pp. 5811–5818, Jul. 2022. [^89]: M. Gerdzhev, R. Razani, E. Taghavi, and L. Bingbing, "TORNADO-Net: Multiview total variation semantic segmentation with diamond inception module," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2021, pp. 9543–9549. [^90]: A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 3354–3361. [^91]: M. Everingham, S. M. A. Eslami, L. J. V. Gool, C. K. I. Williams, J. M. Winn, and A. Zisserman, “The PASCAL visual object classes challenge: A retrospective,” Int. J. Comput. Vis., vol. 111, no. 1, pp. 98–136, 2015. [^92]: W. Shi et al., “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 1874–1883. ![Image](images/3a90fc7dc9bc7c9a86c3e850c5877495a4499cc06d4c7aee900c883026e7ebd5.jpg) Hui-Xian Cheng received the B.S. degree from the Wuhan University of Technology in 2019. He is currently pursuing the M.S. degree with the College of Computer and Information Science, Southwest University, Chongqing, China. His research interests include 3D scene understanding, point cloud semantic segmentation, and multi-modal fusion. ![Image](images/5372a1b626f9f1cc72657f2d5d80d29be8ec9c730172a7c86a041216095deb6f.jpg) Xian-Feng Han (Member, IEEE) received the Ph.D. degree in software engineering from Tianjin University in 2019. He is currently a Lecturer with the College of Computer and Information Science, Southwest University, Chongqing, China. His research interests include 3D point cloud processing, 3D reconstruction, and machine learning. ![Image](images/a1df19367b3aef0c5fc946edb169dacfdbcc0408823e4fe4e387c07eeea03c65.jpg) Guo-Qiang Xiao received the B.E. degree in radio technology from Chongqing University, Chongqing, China, in 1986, and the Ph.D. degree in signal and information processing from the University of Electronic Science and Technology of China, Chengdu, China, in 1999. Since 1986, he has been with the College of Computer and Information Science, Southwest University, Chongqing, where he is currently a Professor. His research interests include computer vision, pattern recognition, machine learning, and big data mining.