# TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning

Peixiang Huang $^{1\dagger}$, Li Liu $^{2\dagger\S}$, Renrui Zhang $^{3\dagger}$, Song Zhang $^{4}$, Xinli Xu $^{2}$

Baichao Wang $^{2}$ Guoyi Liu $^{2}$

 $^{1}$Peking University $^{2}$NIO $^{3}$The Chinese University of Hong Kong

 $^{4}$University of Chinese Academy of Sciences

<mailto:liuli.119412@gmail.com>, <mailto:huangpx@stu.pku.edu.cn>, <mailto:1155186671@link.cuhk.edu.hk>, <mailto:zhangsong20@mails.ucas.ac.cn>, <mailto:xx1bigbrother@gmail.com>, {baichao.wang, gary.liu}@nio.com

## Abstract

To achieve accurate multi-view 3D object detection, existing methods propose to benefit camera-based detectors with spatial cues provided by the LiDAR modality, e.g., dense depth supervision and bird-eye-view (BEV) feature distillation. However, they directly conduct point-to-point mimicking from LiDAR to camera, which neglects the inner-geometry of foreground targets and suffers from the modal gap between 2D-3D features. In this paper, we propose the learning scheme of Target Inner-Geometry from the LiDAR modality into camera-based BEV detectors for both dense depth and BEV features, termed as TiG-BEV. First, we introduce an inner-depth supervision module to learn the low-level relative depth relations between different foreground pixels. This enables the camera-based detector to better understand the object-wise spatial structures. Second, we design an inner-feature BEV distillation module to imitate the high-level semantics of different keypoints within foreground targets. To further alleviate the BEV feature gap between two modalities, we adopt both interchannel and inter-keypoint distillation for feature-similarity modeling. With our target inner-geometry learning, TiG-BEV can effectively boost student models by different margins on nuScenes val set, e.g., +2.3% NDS and +2.4% mAP for BEVDepth, along with +9.1% NDS and +10.3% mAP for BEVDet without CBGS. Code will be available at <https://github.com/ADLab3Ds/TiG-BEV>.

### 1. Introduction

3D object detection aims to recognize and localize objects in 3D space, which is more challenging than its 2D counterpart and has achieved outstanding progress in vari-

![Image](images/c689087fb774cbf739296b35793f874137de23833d0f38fddfc0411fb1f5da12.jpg)

<div style="text-align: center;">(a) Dense Depth Supervision</div>

![Image](images/0bd4706d6098e936af1c1eb69cb325369f8c9a759c50eece1d6d886677db501e.jpg)

<div style="text-align: center;">(b) BEV Feature Distillation</div>

![Image](images/94b02f0024f64609941724317153f1cd546cbc959ea8ee81230e3b50ca69df73.jpg)

<div style="text-align: center;">(c) TiG-BEV (Ours)</div>

<div style="text-align: center;">Figure 1. Different LiDAR-to-Camera Learning Schemes: (a) Dense Depth Supervision [^23] [^37], (b) BEV Feature Distillation [^6] [^16], (c) Our Target Inner-Geometry Learning (TiG-BEV).</div>

ous applications, such as robotics [^1], virtual reality [^39], and autonomous driving [^3] [^7] [^41] [^47]. Mainstream methods for 3D object detection can be categorized into LiDAR-based detectors [^11] [^41] [^42] and camera-based detectors [^23] [^24] [^45] [^46] [^53] [^54]. Therein, LiDAR-based methods have attained excellent performance by taking 3D point clouds as input, which inherently contains sufficient spatial structures for accurate object localization. In contrast, camera-based methods are relatively low-cost with colored context information, but are constrained by the lack of geometric depth cues.

Considering the performance gap between camera-based and LiDAR-based detectors, existing methods leverage the spatial cues provided by the LiDAR modality to improve the accuracy of camera-based detectors, which are mainly in two schemes shown in Figure 1. (1) Dense Depth Supervision (Figure 1 (a)), e.g., CaDDN [^37] and BEVDepth [^23]. These methods first project the input LiDAR points onto image planes using intrinsic and extrinsic camera param-

---

![Image](images/c9f5f6f48b91b3d3f226ad7f961bcccdb15199820f80e14622257480e6805922.jpg)

<div style="text-align: center;">Figure 2. Inner-depth Supervision. We guide the camera-based detector to learn the relative spatial structures within the target foreground areas. A depth reference point (dotted in yellow) is adaptively selected to calculate relative depth values.</div>

eters. Then, these derived depth maps are applied to explicitly supervise the categorical depth prediction within both foreground and background areas. (2) BEV Feature Distillation (Figure 1 (b)), e.g., CMKD [^16] and BEVDistill [^6]. There methods adopt the teacher-student paradigm for BEV feature distillation. They force the BEV representation generated by the camera-based detector (student) to imitate that produced by a pre-trained LiDAR-based detector (teacher). By directly mimicking the BEV features, the student is expected to inherit the encoded high-level BEV semantics from the teacher.

However, existing methods fail to capture the inner-geometric characteristics of different foreground targets, i.e., the spatial contours and part-wise semantic relations. As examples, BEVDepth simply adopts pixel-by-pixel depth supervision without specializing the relative depth within objects, and BEVDistill applies foreground-guided distillation but neglects the inner relations of BEV features. In addition, methods [^6] [^16] for BEV feature distillation directly enforce the channel-by-channel alignment between cross-modal BEV representations. Such strict mimicking might adversely affect the performance due to the modal gap between camera and LiDAR BEV features, i.e., visual appearances vs. spatial geometries.

To alleviate this issue, we propose a novel LiDAR-to-camera learning scheme, TiG-BEV, which involves the inner-geometry of foreground targets into the camera-based detectors for multi-view BEV 3D object detection. As shown in Figure 1 (c), we conduct simultaneous target inner-geometry learning for dense depth prediction and BEV feature generation. First, besides the previous absolute depth map prediction [^23] [^37], we introduce an inner-depth supervision module within pixels of different foreground targets. A reference depth point is adaptively selected for each target to obtain the relative depth relationships shown in Figure 2, which contributes to high-quality depth map prediction with better target structural understanding. Second, we propose an inner-feature BEV distillation module, which imitates the high-level foreground BEV semantics produced by a pre-trained LiDAR-based detector. Different

![Image](images/90a2140468ce94a1cd3ec6ff81c2fde91d886bb228a0645e21824441174fa1f9.jpg)

<div style="text-align: center;">Figure 3. Inner-feature BEV Distillation. We respectively conduct inter-channel and inter-keypoint feature distillation in BEV space for the camera-based detector, which alleviates the cross-modal semantic gap and boosts inner-geometry learning.</div>

from previous dense and strict feature distillation [^6] [^16], we sample several keypoints within each BEV foreground area and guide the camera-based detector to learn their inner feature-similarities shown in Figure 3, which are in both inter-channel and inter-keypoint manners. In this way, the camera-based detector can not only inherit the high-level part-wise LiDAR semantics, but also relieve the modal gap by avoiding the strict feature mimicking. Through extensive experiments, we observe consistent performance improvement brought by TiG-BEV upon the baseline models. On nuScenes [^3] val set with equivalent settings, the powerful BEVDepth [^23] can be boosted by +2.3% NDS and +2.4% mAP, and BEVDet [^19] without CBGS can be further enhanced by +9.1% NDS and +10.3% mAP for BEVDet without CBGS, which well demonstrates the significance of our TiG-BEV.

The contributions of TiG-BEV are summarized below:

- We introduce an inner-depth supervision module to model the relative depth relations of foreground targets, which leads to better depth map prediction.

- We propose an inner-feature BEV distillation module to transfer the well-learned knowledge from LiDAR modality to camera-based BEV representations.

- Extensive experiments have verified the effectiveness of our approach to enhance the capacity for multi-view BEV 3D object detection.

### 2. Related Works

Camera-based 3D Object Detection. Camera-based 3D object detection has been widely used for applications like autonomous driving since its low cost compared with LiDAR-based detectors. FCOS3D [^45] first predicts the 3D attributes of objects through the features around 2D centers and PGD [^44] utilizes the relational graphs to improve the depth estimation for 3D monocular object detection. Further, MonoDETR [^53] introduces DETR-like [^4] architectures without complex post-processing. Recently, Bird's-Eye-View (BEV), as a unified representation of surrounding views same as LiDAR-based detector, has at-

---

tracted much attention. DETR3D [^46] follows the DETR [^4] to adopt the 3D reference points in BEV space by using object queries. BEVDet [^19] utilizes the Lift-Splat-Shoot (LSS) operation [^34] to transform 2D image features into 3D Ego-car coordinate to generate 3D BEV feature. PETR [^27] obtain the 3D position-aware ability by 3D positional embedding. Inspired by the recently developed attention mechanism, BEVFormer [^24] and PolarFormer [^21] automates the camera-to-BEV process with learnable attention modules and queries a BEV feature according to its position in 3D space. To further improve the detection performance, the temporal information has been introduced in BEVDet4D [^18] and PETRv2 [^28], which achieve significant performance enhancement. Moreover, BEVDepth [^23] observes that accurate depth estimation is essential for BEV 3D object detection supervised by projected LiDAR points. MonoDETR-MV [^54] proposes a depth-guided transformer for multi-view geometric cues, but predicts only foreground depth map without dense depth supervision. As a LiDAR-to-camera learning schematic, our TiG-BEV leverages the pretrained LiDAR-based detector to improve the performance of camera-based detectors for multi-view BEV 3D object detection.

Depth Estimation. Depth estimation is a classical problem in computer vision. These method can be divided into single-view depth estimation and multi-view depth estimation. Single-view depth estimation is either regarded as a regression problem of a dense depth map or a classification problem of the depth distribution. [^2] [^12] [^13] [^35] [^36] generally build an encoder-decoder architecture to regress the depth map from contextual features. Multi-view depth estimation methods usually construct a cost volume to regress disparities based on photometric consistency [^15] [^33] [^40] [^49] [^52] [^56]. For 3D object detection, previous methods [^16] [^32] [^37] also introduce additional networks for depth estimation to improve the localization accuracy in 3D space. Notably, MonoDETR [^53] [^54] proposes to only predict the foreground depth maps instead of the dense depth values, but cannot leverage the advanced geometries provided by LiDAR modality. Different from them, our TiG-BEV conducts inner-depth supervision that captures local spatial structures of different foreground targets.

Knowledge Distillation. Knowledge Distillation has shown very promising ability in transferring learned representation from the larger model (teacher) to the smaller one (student). Prior works [^20] [^26] [^43] [^51] have been proposed to help the student network learn the structural representation for better generalization ability. These methods generally utilize the correlation of the instances to describe the geometry, similarity, or dissimilarity in the feature space. The following methods extend the teacher-student paradigm to many vision task, demonstrating its effectiveness including action recognition [^9], video caption [^31], 3D representation learning [^14] [^29] [^38] [^55], object detection [^5] [^10] and semantic segmentation [^17] [^48]. However, only a few of works consider the multi-modal setting between different sensor sources. For 3D representation learning, I2P-MAE [^55] leverages masked autoencoders to distill 2D pre-trained knowledge into 3D transformers. UVTR [^22] presents a simple approach by directly regularizing the voxel representations between the student and teacher models. BEVDistill [^6] transfer knowledge from LiDAR feature to the cam feature by dense feature distillation and sparse instance distillation. Our TiG-BEV also follows such teacher-student paradigm and effectively distills knowledge from the LiDAR modality into the camera modality,

### 3. Method

The overall architecture of TiG-BEV is shown in Figure 4, which consists of three components: the student camera-based detector, the teacher LiDAR-based detector, and our proposed target inner-geometry learning scheme. In Section 3.1, we first introduce the adopted baseline models. Then, we specifically illustrate the designs of TiG-BEV for inner-depth supervision in Section 3.2 and inner-BEV feature distillation in Section 3.3. Finally in Section 3.4, we present the overall loss of our TiG-BEV for LiDAR-to-camera learning.

#### 3.1. Baseline Models

Student Camera-based Detector. By default, we adopt BEVDepth [^23] as our student camera-based detector for multi-view 3D object detection. Given the input multi-view images (normally 6 views for a scene), the student model first utilizes a shared 2D backbone and FPN module [^25] to extract the C-channel visual features $\{F_i\}_{i=1}^6$, where $F_i \in \mathbb{R}^{C \times H_v \times W_v}$, and $H_v, W_v$ denote the size of feature maps. These features are fed into a shared depth network to generate the categorical depth map [^37], $\{D_i\}_{i=1}^6$, where $D_i \in \mathbb{R}^{D \times H_v \times W_v}$, where D denotes the pre-defined number of depth bins. During training, BEVDepth adopts dense depth supervision for the predicted depth maps, which projects the paired LiDAR input onto multi-view image planes to construct pixel-by-pixel absolute depth ground truth, $\{D_i\}_{i=1}^{6}$, where $D_i^{gt} \in \mathbb{R}^{1 \times H_v \times W_v}$. Then, following [^34], the multi-view visual features are projected into a unified BEV representation via the predicted depth maps, which is further encoded by a BEV encoder, denoted as $F_{blev}^{2d} \in \mathbb{R}^{C \times H_{blev} \times W_{blev}}$. Finally, the detection heads are applied on top to predict objects in 3D space. We represent the two basic losses of the student model as $\mathcal{L}_{depth}^A$ and $\mathcal{L}_{det}$, respectively denoting the Binary Cross Entropy loss for dense absolute depth values and the 3D detection loss.

---

![Image](images/54df155c59be0665c15382b82db29e03bfcbfe7be534f8ca1d4acce0e07590c5.jpg)

<div style="text-align: center;">Figure 4. Overall Framework of TiG-BEV, which contains a pre-trained LiDAR-based detector as teacher, a camera-based detector as student, and a target inner-geometry scheme for cross-model learning. Our proposed learning paradigm effectively transfers the inner-geometry semantics of the LiDAR modality via two components, an inner-depth supervision (Section 3.2) for foreground relative depth, and an inner-feature BEV distillation (Section 3.3) from both channel-wise and keypoint-wise.</div>

Teacher LiDAR-based Detector. We select the popular LiDAR detector CenterPoint [^50] as the teacher for target inner-geometry learning. Given the input point cloud data, CenterPoint voxelizes into grid-based data and utilizes a 3D backbone to obtain the C-channel LiDAR BEV feature $F_{bev}^{3d} \in \mathbb{R}^{C \times H_{bev} \times W_{bev}}$, which has the same feature size as $F_{bev}^{2d}$ from the student detector. As the CenterPoint has been well pre-trained, $F_{bev}^{3d}$ can provide the student BEV feature with sufficient geometric and semantic knowledge, especially in the target foreground areas. Note that the LiDAR-based teacher is merely required during training for crossmodal learning, and for inference, only multi-view images are taken as input for the camera-based detector.

#### 3.2. Inner-depth Supervision

In addition to the dense absolute depth supervision, we propose to guide the student model to learn the inner-depth geometries in different target foreground areas. As shown in Figure 5 (a), for the instance level, the existing absolute depth supervision with categorical representation ignores the relative structural information inside each object and provide no explicit fine-grained depth signals. Therefore, we propose to additionally conduct inner-depth supervision with continuous values from the LiDAR projected depth maps shown in Figure 5 (b), which effectively boosts the network to capture the inner-geometry of object targets.

Foreground Target Localization. To accurately obtain the inner-depth values, we first localize the foreground pixels for each object targets in the depth maps. Given the ground-truth 3D bounding boxes, we extract the corresponding 3D LiDAR points inside the box for each object target, and project them onto different image planes. In this way, we can attain the pixels within foreground object areas on both the predicted and ground-truth depth maps, $\{D_i, D_i^{gt}\}_{i=1}^6$. The foreground pixels can roughly depict the geometric contour of different target objects and well improve the subsequent inner-depth learning. We taking the i-th view as an example and omit the index i in the following texts for simplicity. Suppose there exist M target objects on the image, we denote the foreground depth-value set for the M objects as $\{S_j, S_j^{gt}\}_{j=1}^M$, where each $\{S_j, S_j^{gt}\}$ includes the foreground categorical depth prediction and ground-truth depth values for the j-th target.

---

![Image](images/f98ff9ce37819a93acc70eb5689c51387039b1d617dd8bc16388cbe21da3cb82.jpg)

<div style="text-align: center;">Figure 5. Comparison of Categorical Absolute Depth and Continuous Inner Depth. We adopt the inner-depth supervision with continuous depth values to guide the camera-based student to learn local spatial structures of foreground object targets.</div>

Continuous Depth Representation. Different from the categorical representation of absolute depth values, we represent the predicted inner depth of foreground targets by continuous values, which reflects more fine-grained geometric variations. For pixel $(x, y)$ of the j-th target object $S_j$, the predicted possibility of k-th depth bin is denoted as $S_j(x, y)[k]$, where $1 \leq k \leq D$. Referring to MonoDETR [^53] [^54], we calculate the continuous depth value $d_j(x, y)$ for the pixel $(x, y)$ as

 $$ d_{j}(x,y)=\sum_{k=1}^{D}(d[k]\cdot S_{j}(x,y)[k]), $$

where $d[k]$ denotes the depth value of the k-th bin center. By this, we convert the categorical depth prediction of different target objects, $\{S_j\}_{j=1}^M$, into continuous representations, denoted as $\{\hat{S}_j\}_{j=1}^M$.

Adaptive Depth Reference. To calculate the relative depth values, we propose to utilize an adaptive depth reference for different foreground targets. Specifically, according to the predicted continuous depth values in $\{\hat{S}_j\}_{j=1}^M$, we select the pixel with the smallest depth prediction error as the reference point for each target, and correspondingly set its depth value as the depth reference, as shown in Figure 5. For the j-th target with the ground-truth inner-depth $\{\hat{S}_j, \hat{S}_j^{gt}\}_{j=1}^M$, we calculate the depth reference point $(x_r, y_r)$ by

 $$ \left(x_{r},y_{r}\right)=\underset{(x,y)\in\hat{S}_{j}}{\mathrm{A r g m i n}}\left(S_{j}^{g t}(x,y)-\hat{S}_{j}(x,y)\right). $$

Then, the predicted and ground-truth reference depth values are denoted as $d_{j}(x_{r}, y_{r})$ and $d_{j}^{gt}(x_{r}, y_{r})$, respectively. By adaptively selecting the reference point with the smallest error, the inner-depth distribution can dynamically adapt to objects with different shapes and appearances, which stabilizes the network learning for some truncated and occluded objects.

Inner-depth Calculation. On top of the reference depth value, we calculate the relative depth values within the foreground area of each target object. For pixel $(x, y)$ of the j-th target $\{\hat{S}_j, S_j^{gt}\}$, the predicted and ground-truth inner-depth values are formulated as

 $$ \begin{align*}rd_{j}(x,y)&=d_{j}(x,y)-d_{j}(x_{r},y_{r}),\\rd_{j}^{gt}(x,y)&=d_{j}^{gt}(x,y)-d_{j}^{gt}(x_{r},y_{r}).\end{align*} $$

We denote the obtained relative depth-value sets for M target objects as $\{\hat{R}_j, R_j^gt\}_{j=1}^M$. Finally, we supervise the inner-depth prediction of the student detector by an L2 loss, formulated as

 $$ \mathcal{L}_{\mathrm{d e p t h}}^{R}=\sum_{j=1}^{M}||\hat{R}_{j}-R_{j}^{g t}||_{2}. $$

#### 3.3. Inner-feature BEV Distillation

Besides the depth supervision for low-level spatial information, our TiG-BEV also adopts the inner-geometry learning for high-level BEV semantics from pre-trained LiDAR-based detectors. Previous works [^6] [^16] for BEV distillation directly force the student to imitate the teacher's features point-to-point in the BEV space. In spite of the performance improvement, such strategies are constrained by the following two aspects. On the one hand, due to the sparsity of scanned point clouds, the LiDAR-based BEV features might contain redundant and noisy information in the background areas. Although BEVDistill [^6] utilizes foreground masks to alleviate this issue, such dense feature distillation still cannot provide focused and effective guidance to the student network. On the other hand, the camera-based and LiDAR-based BEV features depict different characteristics of the scene, respectively, visual appearances and spatial structures. Therefore, forcing the BEV features to be completely consistent between two modalities is sub-optimal considering the semantic gap. In our TiG-BEV, we propose an inner-feature BEV distillation (Figure 6) consisting of inter-channel and inter-keypoint learning schemes, which conducts attentive target features distillation and relieves the cross-modal semantic gap.

Target Keypoint Extraction. To distill the knowledge of LiDAR-based detectors only within sparse foreground regions, we extract the BEV area of each object target and represent it by a series of keypoint features. Given the ground-truth 3D bounding box for each target, we first enlarge the box size for a little bit in the BEV space to cover the entire foreground area, e.g., object contours and edges.

---

![Image](images/f5062533494bf4cb344e1b05d55bef3dd9bcb4c7cc8198d00f7209f933e9afa6.jpg)

<div style="text-align: center;">Figure 6. Details of Inner-feature BEV Distillation. For each foreground area in BEV space, we represent each target feature by a set of keypoints and conduct feature distillation in both interchannel and inter-keypoint manners.</div>

Then, we uniformly sample its BEV bounding box by N keypoints, and adopt bilinear interpolation to obtain the keypoint features from the encoded BEV representations. From both camera-based $F_{bev}^{2d}$ and LiDAR-based $F_{bev}^{3d}$, we respectively extract the keypoint features for all M object targets as $\{f_j^{2d}, f_j^{3d}\}_{j=1}^M$, where $f_j^{2d}, f_j^{3d} \in \mathbb{R}^{N \times C}$. By the uniform sampling, such BEV keypoints can well represent the part-wise features and the inner-geometry semantics of foreground targets.

Inter-channel BEV Distillation. Taking the j-th object target as an example, we first apply an inter-channel BEV distillation, which guides the student keypoint features to mimic the channel-wise relationships of the teacher's. Such inter-channel signals imply the overall geometric semantics of each object target. Compared with the previous channel-by-channel supervision, our inter-channel distillation can preserve the distinctive aspects of the two modalities, while effectively transfer the well pre-trained knowledge of LiDAR-based detectors. Specifically, we calculate the inter-channel similarities of both camera-based and LiDAR-based keypoint features, formulated as

 $$ A_{j}^{2d}=f_{j}^{2d}f_{j}^{2d^{\top}};\quad A_{j}^{3d}=f_{j}^{3d}f_{j}^{3d^{\top}}, $$

where $A_j^{2d}$, $A_j^{3d} \in \mathbb{R}^{C \times C}$ denote the feature relationships between different C channels for the two modalities. For all M objects in a scene, we adopt L2 loss between the two inter-channel similarities for feature distillation, formulated as

 $$ \mathcal{L}_{\mathrm{b e v}}^{I C}=\sum_{j=1}^{M}||A_{j}^{3d}-A_{j}^{2d}||_{2}. $$

Inter-keypoint BEV Distillation. The inter-channel distillation guides the camera-based detector to learn the channel-wise diversity from the LiDAR-based teacher. However, it is conducted without considering the inner correlation of different keypoints within each object target, which is not capable of capturing the local geometries among different foreground parts, e.g., the front and rear of cars. To this end, we propose to utilize the inter-keypoint correlations of LiDAR-based BEV features and transfer such inner-geometry semantics into camera-based detectors. Analogous to the aforementioned inter-channel module, for the j-th target object, we calculate the inter-keypoint similarities in a transposed manner for the two modalities as

 $$ B_{j}^{2d}=f_{j}^{2d}{}^{\top}f_{j}^{2d};\quad B_{j}^{3d}=f_{j}^{3d}{}^{\top}f_{j}^{3d}, $$

where $B_j^{2d}, B_j^{3d} \in \mathbb{R}^{N \times N}$ denote the feature relationships between different N keypoints respectively for camera and LiDAR. We also adopt L2 loss for all M targets as

 $$ \mathcal{L}_{\mathrm{b e v}}^{I K}=\sum_{j=1}^{M}||B_{j}^{3d}-B_{j}^{2d}||_{2}. $$

Then, the distillation loss for inter-channel and inter-keypoint features in BEV space is formulated as

 $$ \mathcal{L}_{\mathrm{b e v}}=\mathcal{L}_{\mathrm{b e v}}^{I C}+\mathcal{L}_{\mathrm{b e v}}^{I K}, $$

where the two terms are orthogonal respectively for the channel-wise feature diversity and keypoint-wise semantic correlations.

#### 3.4. Overall Loss

To sum up, we benefit the student camera-based detector by target inner-geometry from two complementary aspects, i.e., an inner-depth supervision for low-level signals and an inner-feature BEV distillation for high-level semantics. They produce two losses as $\mathcal{L}_{\mathrm{depth}}^{R}$ and $\mathcal{L}_{\mathrm{bev}}$. Together with the original two losses, i.e., dense absolute depth supervision $\mathcal{L}_{\mathrm{depth}}^{A}$, and 3D detection $\mathcal{L}_{\mathrm{det}}$, the overall loss of our TiG-BEV is formulated as

 $$ \mathcal{L}_{\mathrm{T i G}}=\mathcal{L}_{\mathrm{d e t}}+\mathcal{L}_{\mathrm{d e p t h}}^{A}+\mathcal{L}_{\mathrm{d e p t h}}^{R}+\mathcal{L}_{\mathrm{b e v}}^{I C}+\mathcal{L}_{\mathrm{b e v}}^{I K}. $$

### 4. Experiment

In this section, we first introduce our adopted dataset and implementation settings. Then, we conduct a series of experiments with detailed ablation studies to show the effectiveness of our approach.

#### 4.1. Experimental Settings

Dataset and Metrics. We evaluate our TiG-BEV on nuScenes dataset [^3], which is one of the most popular large-scale outdoor public datasets for autonomous driving. It consists of 700, 150, 150 scenes for training, validation

---

<div style="text-align: center;">Table 1. Performance Comparison on nuScenes [^3] Val Set. 'C' and 'L' denote the camera-based and LiDAR-based methods, which refer to the input data during inference. * denotes our implementation using BEVDet [^19] codebase.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Method</td><td style='text-align: center; word-wrap: break-word;'>Modality</td><td style='text-align: center; word-wrap: break-word;'>Backbone</td><td style='text-align: center; word-wrap: break-word;'>Resolution</td><td style='text-align: center; word-wrap: break-word;'>mAP $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>NDS $\uparrow$</td><td style='text-align: center; word-wrap: break-word;'>mATE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>mASE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>mAOE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>mAVE $\downarrow$</td><td style='text-align: center; word-wrap: break-word;'>mAAE $\downarrow$</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>FCOS3D [^45]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$900 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.343</td><td style='text-align: center; word-wrap: break-word;'>0.415</td><td style='text-align: center; word-wrap: break-word;'>0.725</td><td style='text-align: center; word-wrap: break-word;'>0.263</td><td style='text-align: center; word-wrap: break-word;'>0.422</td><td style='text-align: center; word-wrap: break-word;'>1.292</td><td style='text-align: center; word-wrap: break-word;'>0.153</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>PGD [^44]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$900 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.369</td><td style='text-align: center; word-wrap: break-word;'>0.428</td><td style='text-align: center; word-wrap: break-word;'>0.683</td><td style='text-align: center; word-wrap: break-word;'>0.260</td><td style='text-align: center; word-wrap: break-word;'>0.439</td><td style='text-align: center; word-wrap: break-word;'>1.268</td><td style='text-align: center; word-wrap: break-word;'>0.185</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>MonoDETR [^53]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$900 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.372</td><td style='text-align: center; word-wrap: break-word;'>0.434</td><td style='text-align: center; word-wrap: break-word;'>0.676</td><td style='text-align: center; word-wrap: break-word;'>0.258</td><td style='text-align: center; word-wrap: break-word;'>0.429</td><td style='text-align: center; word-wrap: break-word;'>1.253</td><td style='text-align: center; word-wrap: break-word;'>0.176</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>DETR3D [^46]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$900 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.303</td><td style='text-align: center; word-wrap: break-word;'>0.374</td><td style='text-align: center; word-wrap: break-word;'>0.860</td><td style='text-align: center; word-wrap: break-word;'>0.278</td><td style='text-align: center; word-wrap: break-word;'>0.437</td><td style='text-align: center; word-wrap: break-word;'>0.967</td><td style='text-align: center; word-wrap: break-word;'>0.235</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>PETR [^27]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$512 \times 1408$</td><td style='text-align: center; word-wrap: break-word;'>0.357</td><td style='text-align: center; word-wrap: break-word;'>0.421</td><td style='text-align: center; word-wrap: break-word;'>0.710</td><td style='text-align: center; word-wrap: break-word;'>0.270</td><td style='text-align: center; word-wrap: break-word;'>0.490</td><td style='text-align: center; word-wrap: break-word;'>0.885</td><td style='text-align: center; word-wrap: break-word;'>0.224</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVFormer [^24]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$900 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.416</td><td style='text-align: center; word-wrap: break-word;'>0.517</td><td style='text-align: center; word-wrap: break-word;'>0.673</td><td style='text-align: center; word-wrap: break-word;'>0.274</td><td style='text-align: center; word-wrap: break-word;'>0.372</td><td style='text-align: center; word-wrap: break-word;'>0.394</td><td style='text-align: center; word-wrap: break-word;'>0.198</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>PETRv2 [^28]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$640 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.421</td><td style='text-align: center; word-wrap: break-word;'>0.524</td><td style='text-align: center; word-wrap: break-word;'>0.681</td><td style='text-align: center; word-wrap: break-word;'>0.267</td><td style='text-align: center; word-wrap: break-word;'>0.357</td><td style='text-align: center; word-wrap: break-word;'>0.377</td><td style='text-align: center; word-wrap: break-word;'>0.186</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>MonoDETR-MV [^54]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$640 \times 1600$</td><td style='text-align: center; word-wrap: break-word;'>0.428</td><td style='text-align: center; word-wrap: break-word;'>0.531</td><td style='text-align: center; word-wrap: break-word;'>0.676</td><td style='text-align: center; word-wrap: break-word;'>0.268</td><td style='text-align: center; word-wrap: break-word;'>0.352</td><td style='text-align: center; word-wrap: break-word;'>0.380</td><td style='text-align: center; word-wrap: break-word;'>0.169</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>CenterPoint [^50] (Teacher)</td><td style='text-align: center; word-wrap: break-word;'>L</td><td style='text-align: center; word-wrap: break-word;'>VoxelNet</td><td style='text-align: center; word-wrap: break-word;'>-</td><td style='text-align: center; word-wrap: break-word;'>0.564</td><td style='text-align: center; word-wrap: break-word;'>0.646</td><td style='text-align: center; word-wrap: break-word;'>0.299</td><td style='text-align: center; word-wrap: break-word;'>0.254</td><td style='text-align: center; word-wrap: break-word;'>0.330</td><td style='text-align: center; word-wrap: break-word;'>0.286</td><td style='text-align: center; word-wrap: break-word;'>0.191</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDet $^{*}$ [^19]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-50</td><td style='text-align: center; word-wrap: break-word;'>$256 \times 704$</td><td style='text-align: center; word-wrap: break-word;'>0.298</td><td style='text-align: center; word-wrap: break-word;'>0.379</td><td style='text-align: center; word-wrap: break-word;'>0.725</td><td style='text-align: center; word-wrap: break-word;'>0.279</td><td style='text-align: center; word-wrap: break-word;'>0.589</td><td style='text-align: center; word-wrap: break-word;'>0.860</td><td style='text-align: center; word-wrap: break-word;'>0.245</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TIG-BEV</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-50</td><td style='text-align: center; word-wrap: break-word;'>$256 \times 704$</td><td style='text-align: center; word-wrap: break-word;'>0.331</td><td style='text-align: center; word-wrap: break-word;'>0.411</td><td style='text-align: center; word-wrap: break-word;'>0.678</td><td style='text-align: center; word-wrap: break-word;'>0.271</td><td style='text-align: center; word-wrap: break-word;'>0.589</td><td style='text-align: center; word-wrap: break-word;'>0.784</td><td style='text-align: center; word-wrap: break-word;'>0.218</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>+3.3%</td><td style='text-align: center; word-wrap: break-word;'>+3.2%</td><td style='text-align: center; word-wrap: break-word;'>-4.7%</td><td style='text-align: center; word-wrap: break-word;'>-0.8%</td><td style='text-align: center; word-wrap: break-word;'>-0.0%</td><td style='text-align: center; word-wrap: break-word;'>-7.6%</td><td style='text-align: center; word-wrap: break-word;'>-2.7%</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDet4D $^{*}$ [^18]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-50</td><td style='text-align: center; word-wrap: break-word;'>$256 \times 704$</td><td style='text-align: center; word-wrap: break-word;'>0.322</td><td style='text-align: center; word-wrap: break-word;'>0.451</td><td style='text-align: center; word-wrap: break-word;'>0.724</td><td style='text-align: center; word-wrap: break-word;'>0.277</td><td style='text-align: center; word-wrap: break-word;'>0.520</td><td style='text-align: center; word-wrap: break-word;'>0.366</td><td style='text-align: center; word-wrap: break-word;'>0.212</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TIG-BEV</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-50</td><td style='text-align: center; word-wrap: break-word;'>$256 \times 704$</td><td style='text-align: center; word-wrap: break-word;'>0.356</td><td style='text-align: center; word-wrap: break-word;'>0.477</td><td style='text-align: center; word-wrap: break-word;'>0.648</td><td style='text-align: center; word-wrap: break-word;'>0.273</td><td style='text-align: center; word-wrap: break-word;'>0.517</td><td style='text-align: center; word-wrap: break-word;'>0.364</td><td style='text-align: center; word-wrap: break-word;'>0.210</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>+3.4%</td><td style='text-align: center; word-wrap: break-word;'>+2.6%</td><td style='text-align: center; word-wrap: break-word;'>-7.6%</td><td style='text-align: center; word-wrap: break-word;'>-0.4%</td><td style='text-align: center; word-wrap: break-word;'>-0.3%</td><td style='text-align: center; word-wrap: break-word;'>-0.2%</td><td style='text-align: center; word-wrap: break-word;'>-0.2%</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDepth $^{*}$ [^23]</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$512 \times 1408$</td><td style='text-align: center; word-wrap: break-word;'>0.416</td><td style='text-align: center; word-wrap: break-word;'>0.521</td><td style='text-align: center; word-wrap: break-word;'>0.605</td><td style='text-align: center; word-wrap: break-word;'>0.268</td><td style='text-align: center; word-wrap: break-word;'>0.455</td><td style='text-align: center; word-wrap: break-word;'>0.333</td><td style='text-align: center; word-wrap: break-word;'>0.203</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TIG-BEV</td><td style='text-align: center; word-wrap: break-word;'>C</td><td style='text-align: center; word-wrap: break-word;'>ResNet-101</td><td style='text-align: center; word-wrap: break-word;'>$512 \times 1408$</td><td style='text-align: center; word-wrap: break-word;'>0.440</td><td style='text-align: center; word-wrap: break-word;'>0.544</td><td style='text-align: center; word-wrap: break-word;'>0.570</td><td style='text-align: center; word-wrap: break-word;'>0.267</td><td style='text-align: center; word-wrap: break-word;'>0.392</td><td style='text-align: center; word-wrap: break-word;'>0.331</td><td style='text-align: center; word-wrap: break-word;'>0.201</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>+2.4%</td><td style='text-align: center; word-wrap: break-word;'>+2.3%</td><td style='text-align: center; word-wrap: break-word;'>-3.5%</td><td style='text-align: center; word-wrap: break-word;'>-0.1%</td><td style='text-align: center; word-wrap: break-word;'>-6.3%</td><td style='text-align: center; word-wrap: break-word;'>-0.2%</td><td style='text-align: center; word-wrap: break-word;'>-0.2%</td></tr></table>

<div style="text-align: center;">Table 2. Performance Comparison without CBGS Strategy [^57]. For all methods, we adopt ResNet-101 as the 2D backbone and  $512 \times 1408$ as the image resolution. * denotes our implementation.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Method</td><td style='text-align: center; word-wrap: break-word;'>mAP</td><td style='text-align: center; word-wrap: break-word;'>NDS</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDet $^{{*}}$</td><td style='text-align: center; word-wrap: break-word;'>0.272</td><td style='text-align: center; word-wrap: break-word;'>0.297</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.375 (+10.3%)</td><td style='text-align: center; word-wrap: break-word;'>0.388 (+9.1%)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDet4D $^{{*}}$</td><td style='text-align: center; word-wrap: break-word;'>0.336</td><td style='text-align: center; word-wrap: break-word;'>0.435</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.409 (+7.3%)</td><td style='text-align: center; word-wrap: break-word;'>0.489 (+5.4%)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDepth $^{{*}}$</td><td style='text-align: center; word-wrap: break-word;'>0.393</td><td style='text-align: center; word-wrap: break-word;'>0.487</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.430 (+3.7%)</td><td style='text-align: center; word-wrap: break-word;'>0.514 (+2.7%)</td></tr></table>

and testing, respectively. It provides synced data captured from a 32-beam LiDAR at 20Hz and six cameras covering 360-degree horizontally at 12Hz. We adopt the official evaluation toolbox provided by nuScenes, which reports the nuScenes Detection Score (NDS) and mean Average Precision (mAP), along with mean Average Translation Error (mATE), mean Average Scale Error (mASE), mean Average Orientation Error (mAOE), mean Average Velocity Error (mAVE), and mean Average Attribute Error (mAAE).

Implementation Details. We implement our TiG-BEV using the BEVDet [^18] [^19] code base on 8 NVIDIA A100 GPUs, which is built on MMDetection3D toolkit [^8]. A pretrained CenterPoint [^50] with voxel size of [0.1, 0.1, 0.2]

<div style="text-align: center;">Table 3. Comparison with BEVDistill [^6]. † and * denote the implementation of BEVDistill and ours, respectively. We present the performance improvement of the learning methods correspondingly to their implemented baselines.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Method</td><td style='text-align: center; word-wrap: break-word;'>mAP</td><td style='text-align: center; word-wrap: break-word;'>NDS</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDepth $^{\dagger}$</td><td style='text-align: center; word-wrap: break-word;'>0.311</td><td style='text-align: center; word-wrap: break-word;'>0.432</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDepth $^{*}$</td><td style='text-align: center; word-wrap: break-word;'>0.329</td><td style='text-align: center; word-wrap: break-word;'>0.431</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ BEVDistill</td><td style='text-align: center; word-wrap: break-word;'>0.332 (+2.1%)</td><td style='text-align: center; word-wrap: break-word;'>0.454 (+2.2%)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.366 (+3.7%)</td><td style='text-align: center; word-wrap: break-word;'>0.461 (+3.0%)</td></tr></table>

is adopted as the LiDAR-based teacher, and the camera-based students include BEVDepth [^23], BEVDet [^19] and BEVDet4D [^18]. During inference, the camare-based detectors only take multi-view images as input without the LiDAR data or teachers. Referring to BEVDepth, we additionally add the dense depth supervision on top of BEVDet and BEVDet4D besides our TiG-BEV. We follow their official training configurations and hyperparameters as default, including data augmentation (random flip, scale and rotation), training schedule (2x), and others (AdamW optimizer [^30], 2e-4 learning rate and batch size 8). For the main results on nuScenes val set in Table 1, we compare our TiG-BEV with all other methods under the CBGS strategy [^57]. For all other results, we do not utilize CBGS to better reveal the significance of proposed methods.

---

<div style="text-align: center;">w TiG-BEV</div>

![Image](images/89c9bb4e54e00e291194da7970847373dd5adcb9395b4dd84d974e8bf920783c.jpg)

<div style="text-align: center;">Figure 7. Visualization of Detection Results. From left to right, we show the 3D object detection before and after the TiG-BEV learning schemes, ground-truth annotations, along with the overall BEV-space results.</div>

<div style="text-align: center;">Table 4. Ablation Study of Target Inner-geometry Learning.  $\mathcal{L}_{\text{depth}}^{R}$ and  $\mathcal{L}_{\text{bev}}$ denote the losses of inner-depth supervision and inner-feature BEV distillation, respectively.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>$\mathcal{L}^{R}_{depth}$</td><td style='text-align: center; word-wrap: break-word;'>$\mathcal{L}_{bev}$</td><td style='text-align: center; word-wrap: break-word;'>mAP</td><td style='text-align: center; word-wrap: break-word;'>NDS</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>0.329</td><td style='text-align: center; word-wrap: break-word;'>0.431</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>0.339</td><td style='text-align: center; word-wrap: break-word;'>0.440</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>0.359</td><td style='text-align: center; word-wrap: break-word;'>0.454</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>0.366</td><td style='text-align: center; word-wrap: break-word;'>0.461</td></tr></table>

#### 4.2. Main Results

On nuScenes Val Set. In Table 1, we compare our TiGBEV with other 3D object detectors on nuScenes val set. As shown, our LiDAR-to-camera learning schemes respectively boost the three baseline models, BEVDet, BEVDet4D, and BEVDepth, by +3.2%, +2.6%, and +2.3% NDS. This clearly demonstrates the significance of our TiGBEV to improve the detection performance of multi-view BEV 3D object detection.

Without CBGS [^57] Strategy. In Table 2, we present the results of TiG-BEV without the CBGS training strategy. Without the resampling of training data, the performance improvement of learning target inner-geometry becomes more notable, +10.3%, +7.3%, and +3.7% mAP for the three baselines, which indicates the superior LiDAR-to-camera knowledge transfer of our TiG-BEV.

Comparison with BEVDistill [^6]. In Table 3, we compare our TiG-BEV with another LiDAR-to-camera learning method BEVDistill in the same setting. As shown, on

<div style="text-align: center;">Table 5. Ablation Study of Inner-depth Supervision. We compare different settings for relative depth value calculation and depth reference selection. * denotes our implementation.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Setting</td><td style='text-align: center; word-wrap: break-word;'>Depth Reference</td><td style='text-align: center; word-wrap: break-word;'>mAP</td><td style='text-align: center; word-wrap: break-word;'>NDS</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>BEVDepth $^{{*}}$</td><td style='text-align: center; word-wrap: break-word;'>-</td><td style='text-align: center; word-wrap: break-word;'>0.329</td><td style='text-align: center; word-wrap: break-word;'>0.431</td></tr><tr><td rowspan="2">All-to-Certain</td><td style='text-align: center; word-wrap: break-word;'>3D Center</td><td style='text-align: center; word-wrap: break-word;'>0.358</td><td style='text-align: center; word-wrap: break-word;'>0.452</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>2D Center</td><td style='text-align: center; word-wrap: break-word;'>0.358</td><td style='text-align: center; word-wrap: break-word;'>0.452</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>One-to-One</td><td style='text-align: center; word-wrap: break-word;'>Each Pair</td><td style='text-align: center; word-wrap: break-word;'>0.360</td><td style='text-align: center; word-wrap: break-word;'>0.458</td></tr><tr><td rowspan="2">All-to-Adaptive</td><td style='text-align: center; word-wrap: break-word;'>Highest Conf</td><td style='text-align: center; word-wrap: break-word;'>0.357</td><td style='text-align: center; word-wrap: break-word;'>0.455</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Smallest Error</td><td style='text-align: center; word-wrap: break-word;'>0.366</td><td style='text-align: center; word-wrap: break-word;'>0.461</td></tr></table>

top of a better baseline model, our approach can achieve higher performance boost for both mAP and NDS. This well demonstrates the superiority of target inner-geometry learning to BEVDistill's foreground-guided dense distillation.

Visualization. As visualized in Figure 7, we show the detection results of BEVDepth before and after TiG-BEV, and the ground-truth annotations. We can clearly observe that more accurate results are obtained by our inner-geometry learning. Specifically, within the orange circles, the detection of false positives and ghosting objects can be reduced, and some 3D locations and orientations of the bounding boxes are also refined.

#### 4.3. Ablation Study

Here, we provide detailed experiments to validate the effectiveness of our approach from each of its components. We adopt BEVDepth as the student model by default.

---

<div style="text-align: center;">Table 6. Ablation Study of 2D Backbones and Temporal Information. CenterPoint [^50] and BEVDepth [^23] are adopted as the teacher and student models, respectively.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Backbone</td><td style='text-align: center; word-wrap: break-word;'>Resolution</td><td style='text-align: center; word-wrap: break-word;'>Multi-frame</td><td style='text-align: center; word-wrap: break-word;'>Method</td><td style='text-align: center; word-wrap: break-word;'>mAP</td><td style='text-align: center; word-wrap: break-word;'>NDS</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>VoxelNet</td><td style='text-align: center; word-wrap: break-word;'>-</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>Teacher</td><td style='text-align: center; word-wrap: break-word;'>0.564</td><td style='text-align: center; word-wrap: break-word;'>0.646</td></tr><tr><td rowspan="2">ResNet-18</td><td rowspan="2">$256 \times 704$</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>Student\n+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.285\n0.323 (+3.8%)</td><td style='text-align: center; word-wrap: break-word;'>0.405\n0.430 (+2.5%)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>Student\n+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.260\n0.294 (+3.4%)</td><td style='text-align: center; word-wrap: break-word;'>0.295\n0.335 (+4.0%)</td></tr><tr><td rowspan="2">ResNet-50</td><td rowspan="2">$256 \times 704$</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>Student\n+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.329\n0.366 (+3.7%)</td><td style='text-align: center; word-wrap: break-word;'>0.431\n0.461 (+3.0%)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>Student\n+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.298\n0.338 (+4.0%)</td><td style='text-align: center; word-wrap: break-word;'>0.328\n0.375 (+4.7%)</td></tr><tr><td rowspan="2">ResNet-101</td><td rowspan="2">$512 \times 1408$</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>Student\n+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.393\n0.430 (+3.7%)</td><td style='text-align: center; word-wrap: break-word;'>0.487\n0.514 (+2.7%)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>Student\n+ TiG-BEV</td><td style='text-align: center; word-wrap: break-word;'>0.345\n0.403 (+5.8%)</td><td style='text-align: center; word-wrap: break-word;'>0.366\n0.416 (+5.0%)</td></tr></table>

<div style="text-align: center;">Before Inner-Depth</div>

<div style="text-align: center;">After Inner-Depth</div>

![Image](images/615203883746bff415f867c475e3e52113b4decbef183a34c7fbca34a5d0c415.jpg)

<div style="text-align: center;">Figure 8. Visualization of Predicted Depth Maps, which are before and after the inner-depth supervision, respectively.</div>

Inner-geometry Learning. The individual effectiveness of the two main components can be examined by only equipping one of them. We first study the impact of the inner-depth supervision. As shown in Table 4, we only introduce the inner-depth supervision to the vanilla baseline, i.e., BEVDepth, by which its mAP improves from 32.9% to 33.9% with a +1.0% gain and its NDS reaches 44.0% from 43.1% with a +0.9% gain. Instead, when we only use the inner-feature BEV distillation, the mAP boosts from 32.9% to 35.9% with a +3.0% improvement and the NDS boosts from 43.1% to 45.4% with a +2.3% improvement. In addition, the combination of both components achieves better +3.7% mAP and +2.7% NDS, demonstrating that the two proposed objectives might be complementary.

Inner-depth Supervision. To calculate the relative depth values within foreground targets, we compare several methods concerning the relationships among different inner points, which can be divided into three paradigms, 1) All-

<div style="text-align: center;">Table 7. Ablation Study of Inner-feature BEV Distillation.  $\mathcal{L}_{\mathrm{bev}}^{IC}$ and  $\mathcal{L}_{\mathrm{bev}}^{IK}$ denote the losses of inter-channel and inter-keypoint distillation, respectively.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>$\mathcal{L}^{IC}_{bev}$</td><td style='text-align: center; word-wrap: break-word;'>$\mathcal{L}^{IK}_{bev}$</td><td style='text-align: center; word-wrap: break-word;'>mAP</td><td style='text-align: center; word-wrap: break-word;'>NDS</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>0.329</td><td style='text-align: center; word-wrap: break-word;'>0.431</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>0.342</td><td style='text-align: center; word-wrap: break-word;'>0.444</td></tr><tr><td style='text-align: center; word-wrap: break-word;'></td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>0.358</td><td style='text-align: center; word-wrap: break-word;'>0.452</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>✓</td><td style='text-align: center; word-wrap: break-word;'>0.366</td><td style='text-align: center; word-wrap: break-word;'>0.461</td></tr></table>

to-Certain calculates the relative depth from all sampled points to a certain reference point, such as the projected center of 3D bounding box or the center of 2D bounding box. 2) All-to-Adaptive sets the reference depth in a dynamic manner, which selects the reference pixel with the highest confidence across all depth bins or the smallest depth error to the ground truth (Ours). 3) One-to-One calculates the relative depth from each two sampled point pair. As shown in Table 5, compared with other patterns, our All-to-Adaptive with smallest depth errors obtains the best performance improvement, which indicates the dynamic depth reference point can flexibly adapt to different targets for inner-geometry learning. What's more, we visualize our depth prediction with and without the inner-depth supervision in Figure 8, which effectively refines the contours and edges of foreground objects.

Inner-feature BEV Distillation. Our TiG-BEV explores the BEV feature distillation from two perspectives, interchannel and inter-keypoint. To validate their effectiveness,

---

we also equip the model with one of them at a time and report the results in Table 7. As shown, both inter-channel and inter-keypoint distillation contribute to the final performance, respectively boosting the NDS by +1.3% and +2.1%. This well illustrates the importance of learning inner-geometry semantics within different foreground targets in BEV space. Further combining them two can benefit the performance by +3.7% and +3.0% for mAP and NDS.

2D Backbones and Temporal Information. We further explore the influence of 2D backbones and temporal information to our TiG-BEV in Table 6. We observe that our TiG-BEV brings significant performance improvement consistent over different 2D backbones. Also, our target inner-geometry learning schemes can provide positive effect for both single-frame and multi-frame settings. The improvement of mAP ranges from +3.4% to +5.8% and the improvement of NDS ranges from +2.5% to +5.0%.

### 5. Conclusion

In this paper, we propose a novel target inner-geometry learning framework that enables the camera-based detector to inherit the effective foreground geometric semantics from the LiDAR modality. We first introduce an inner-depth supervision with target-adaptive depth reference to help the student learn better local geometric structures. Then, we conduct inner-feature distillation in BEV space for both channel-wise and keypoint-wise, which contributes to high-level inner-geometry semantics learning from the LiDAR modality. Extensive experiments are implemented to illustrate the significance of TiG-BEV for multi-view BEV 3D object detection. For future works, we will focus on exploring multi-modal learning strategy that can boost both camera and LiDAR modalities for unified real-world perception.

## References

[^1]: Morris Antonello, Marco Carraro, Marco Pierobon, and Emanuele Menegatti. Fast and robust detection of fallen people from a mobile robot. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 4159–4166. IEEE, 2017. 1

[^2]: Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021. 3

[^3]: Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. Nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 1, 2, 6, 7

[^4]: Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213–229. Springer, 2020. 2, 3

[^5]: Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker. Learning efficient object detection models with knowledge distillation. Advances in neural information processing systems, 30, 2017. 3

[^6]: Zehui Chen, Zhenyu Li, Shiquan Zhang, Liangji Fang, Qinhong Jiang, and Feng Zhao. Bevdistill: Cross-modal bev distillation for multi-view 3d object detection. arXiv preprint arXiv:2211.09386, 2022. 1, 2, 3, 5, 7, 8

[^7]: Zehui Chen, Zhenyu Li, Shiquan Zhang, Liangji Fang, Qinghong Jiang, Feng Zhao, Bolei Zhou, and Hang Zhao. Autoalign: Pixel-instance feature aggregation for multimodal 3d object detection. arXiv preprint arXiv:2201.06493, 2022. 1

[^8]: MMDetection3D Contributors. MMDetection3D: Open-MMLab next-generation platform for general 3D object detection. <https://github.com/open-mmlab/mmdetection3d>, 2020.7

[^9]: Zijun Cui, Tengfei Song, Yuru Wang, and Qiang Ji. Knowledge augmented deep neural networks for joint facial expression and action unit recognition. Advances in Neural Information Processing Systems, 33:14338–14349, 2020. 3

[^10]: Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang, Si Liu, and Erjin Zhou. General instance distillation for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7842–7851, 2021. 3

[^11]: Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, and Houqiang Li. Voxel r-cnn: Towards high performance voxel-based 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1201–1209, 2021. 1

[^12]: David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. Advances in neural information processing systems, 27, 2014. 3

[^13]: Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao. Deep ordinal regression network for monocular depth estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2002–2011, 2018. 3

[^14]: Kexue Fu, Peng Gao, Renrui Zhang, Hongsheng Li, Yu Qiao, and Manning Wang. Distillation with contrast is all you need for self-supervised point cloud representation learning. arXiv preprint arXiv:2202.04241, 2022. 3

[^15]: Vitor Guizilini, Igor Vasiljevic, Rares Ambrus, Greg Shakhnarovich, and Adrien Gaidon. Full surround monodepth from multiple cameras. IEEE Robotics and Automation Letters, 7(2):5397–5404, 2022. 3

Ding. Cross-modality knowl-

3d object detection.

res 87–104.

---

[^17]: Yuenan Hou, Xinge Zhu, Yuexin Ma, Chen Change Loy, and Yikang Li. Point-to-voxel knowledge distillation for lidar semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8479–8488, 2022. 3

[^18]: Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues in multi-camera 3d object detection. arXiv preprint arXiv:2203.17054, 2022. 3, 7

[^19]: Junjie Huang, Guan Huang, Zheng Zhu, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view. arXiv preprint arXiv:2112.11790, 2021. 2, 3, 7

[^20]: Zehao Huang and Naiyan Wang. Like what you like: Knowledge distill via neuron selectivity transfer. arXiv preprint arXiv:1707.01219, 2017. 3

[^21]: Yanqin Jiang, Li Zhang, Zhenwei Miao, Xiatian Zhu, Jin Gao, Weiming Hu, and Yu-Gang Jiang. Polarformer: Multicamera 3d object detection with polar transformers. arXiv preprint arXiv:2206.15398, 2022. 3

[^22]: Yanwei Li, Yilun Chen, Xiaojuan Qi, Zeming Li, Jian Sun, and Jiaya Jia. Unifying voxel-based representation with transformer for 3d object detection. arXiv preprint arXiv:2206.00630, 2022. 3

[^23]: Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detection. arXiv preprint arXiv:2206.10092, 2022. 1, 2, 3, 7, 9

[^24]: Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong-hao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. Bevformer: Learning bird's-eye-view representation from multi-camera images via spatiotemporal transformers. arXiv preprint arXiv:2203.17270, 2022. 1, 3, 7

[^25]: Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017. 3

[^26]: Li Liu, Qingle Huang, Sihao Lin, Hongwei Xie, Bing Wang, Xiaojun Chang, and Xiaodan Liang. Exploring inter-channel correlation for diversity-preserved knowledge distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8271–8280, 2021. 3

[^27]: Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. arXiv preprint arXiv:2203.05625, 2022. 3, 7

[^28]: Yingfei Liu, Junjie Yan, Fan Jia, Shuailin Li, Qi Gao, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petrv2: A unified framework for 3d perception from multi-camera images. arXiv preprint arXiv:2206.01256, 2022. 3, 7

[^29]: Yueh-Cheng Liu, Yu-Kai Huang, Hung-Yueh Chiang, Hung-Ting Su, Zhe-Yu Liu, Chin-Tang Chen, Ching-Yu Tseng, and Winston H Hsu. Learning from 2d: Contrastive pixel-to-point knowledge transfer for 3d pretraining. arXiv preprint arXiv:2104.04687, 2021. 3

[^30]: Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. ArXiv, abs/1711.05101, 2017.7

[^31]: Boxiao Pan, Haoye Cai, De-An Huang, Kuan-Hui Lee, Adrien Gaidon, Ehsan Adeli, and Juan Carlos Niebles. Spatio-temporal graph for video captioning with knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10870–10879, 2020. 3

[^32]: Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon. Is pseudo-lidar needed for monocular 3d object detection? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3142–3152, 2021. 3

[^33]: Rui Peng, Rongjie Wang, Zhenyu Wang, Yawen Lai, and Ronggang Wang. Rethinking depth estimation for multiview stereo: A unified representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8645–8654, 2022. 3

[^34]: Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In European Conference on Computer Vision, pages 194–210. Springer, 2020. 3

[^35]: Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mattoccia. On the uncertainty of self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3227–3237, 2020. 3

[^36]: René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12179–12188, 2021. 3

[^37]: Cody Reading, Ali Harakeh, Julia Chae, and Steven L Waslander. Categorical depth distribution network for monocular 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8555–8564, 2021. 1, 2, 3

[^38]: Corentin Sautier, Gilles Puy, Spyros Gidaris, Alexandre Boulch, Andrei Bursuc, and Renaud Marlet. Image-to-lidar self-supervised distillation for autonomous driving data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9891–9901, 2022. 3

[^39]: Martijn J Schuemie, Peter Van Der Straaten, Merel Krijn, and Charles APG Van Der Mast. Research on presence in virtual reality: A survey. CyberPsychology & Behavior, 4(2):183–201, 2001. 1

[^40]: Zhelun Shen, Yuchao Dai, and Zhibo Rao. Cfnet: Cascade and fused cost volume for robust stereo matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13906–13915, 2021. 3

[^41]: Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10529–10538, 2020. 1

[^42]: Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Point-cnn: 3d object proposal generation and detection from point cloud. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 770–779, 2019. 1

---

[^43]: Frederick Tung and Greg Mori. Similarity-preserving knowledge distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1365–1374, 2019. 3

[^44]: Tai Wang, ZHU Xinge, Jiangmiao Pang, and Dahua Lin. Probabilistic and geometric depth: Detecting objects in perspective. In Conference on Robot Learning, pages 1475–1485. PMLR, 2022. 2, 7

[^45]: Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. Fcos3d: Fully convolutional one-stage monocular 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 913–922, October 2021. 1, 2, 7

[^46]: Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Det3d: 3d object detection from multi-view images via 3d-to-2d queries. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 180–191, 08–11 Nov 2022. 1, 3, 7

[^47]: Yingjie Wang, Qiuyu Mao, Hanqi Zhu, Yu Zhang, Jianmin Ji, and Yanyong Zhang. Multi-modal 3d object detection in autonomous driving: a survey. arXiv preprint arXiv:2106.12735, 2021. 1

[^48]: Yukang Wang, Wei Zhou, Tao Jiang, Xiang Bai, and Yongchao Xu. Intra-class feature variation distillation for semantic segmentation. In European Conference on Computer Vision, pages 346–362. Springer, 2020. 3

[^49]: Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Yongming Rao, Guan Huang, Jiwen Lu, and Jie Zhou. Surrounddepth: Entangling surrounding views for self-supervised multi-camera depth estimation. arXiv preprint arXiv:2204.03636, 2022. 3

[^50]: Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11784–11793, 2021. 4, 7, 9

[^51]: Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928, 2016. 3

[^52]: Feihu Zhang, Victor Prisacariu, Ruigang Yang, and Philip HS Torr. Ga-net: Guided aggregation net for end-to-end stereo matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 185–194, 2019. 3

[^53]: Renrui Zhang, Han Qiu, Tai Wang, Xuanzhuo Xu, Ziyu Guo, Yu Qiao, Peng Gao, and Hongsheng Li. Monodetr: Depth-aware transformer for monocular 3d object detection. arXiv preprint arXiv:2203.13310, 2022. 1, 2, 3, 5, 7

[^54]: Renrui Zhang, Han Qiu, Tai Wang, Xuanzhuo Xu, Ziyu Guo, Yu Qiao, Peng Gao, and Hongsheng Li. Monodetr: Depth-guided transformer for monocular 3d object detection. arXiv preprint arXiv:2203.13310, 2022. 1, 3, 5, 7

[^55]: Renrui Zhang, Liuhui Wang, Yu Qiao, Peng Gao, and Hongsheng Li. Learning 3d representations from 2d pre-trained models via image-to-point masked autoencoders. arXiv preprint arXiv:2212.06785, 2022. 3

[^57]: Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced grouping and sampling for point cloud 3d object detection. arXiv preprint arXiv:1908.09492, 2019. 7, 8

[^56]: Renrui Zhang, Ziyao Zeng, Ziyu Guo, and Yafeng Li. Can language understand depth? In Proceedings of the 30th ACM International Conference on Multimedia, pages 6868–6874, 2022. 3