{
  "paper_id": "76bc8db23512b16796032a86e4a58d4e",
  "paper_title": "Voxel Transformer for 3D Object Detection",
  "paper_authors": [
    "Jiageng Mao",
    "Yujing Xue",
    "Xiaodan Liang",
    "Minzhe Niu",
    "Hang Xu",
    "Haoyue Bai",
    "Jiashi Feng",
    "Chunjing Xu"
  ],
  "publication_date": "",
  "publication_venue": "2021 IEEE/CVF International Conference on Computer Vision (ICCV)",
  "abstract": "We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds. Conventional 3D convolutional backbones in voxel-based 3D detectors cannot efficiently capture large context information, which is crucial for object recognition and localization, owing to the limited receptive fields. In this paper, we resolve the problem by introducing a Transformer-based architecture that enables long-range relationships between voxels by self-attention. Given the fact that non-empty voxels are naturally sparse but numerous, directly applying standard Transformer on voxels is non-trivial. To this end, we propose the sparse voxel module and the submanifold voxel module, which can operate on the empty and non-empty voxel positions effectively. To further enlarge the attention range while maintaining comparable computational overhead to the convolutional counterparts, we propose two attention mechanisms for multi-head attention in those two modules: Local Attention and Dilated Attention, and we further propose Fast Voxel Query to accelerate the querying process in multi-head attention. VoTr contains a series of sparse and submanifold voxel modules, and can be applied in most voxel-based detectors. Our proposed VoTr shows consistent improvement over the convolutional baselines while maintaining computational efficiency on the KITTI dataset and the Waymo Open dataset.",
  "keywords": [
    "3D object detection",
    "Transformer",
    "Voxel",
    "Point clouds",
    "Self-attention",
    "Sparse convolution",
    "Autonomous driving"
  ],
  "paper_institutions": [],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:02.889119Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本论文提出了一种基于体素（voxel）的Transformer主干网络Voxel Transformer（VoTr），用于从点云中进行3D物体检测。现有的基于体素的3D检测器通常采用3D卷积骨干网络，但受限于有限的感受野，难以高效捕获大范围上下文信息，这对识别和定位稀疏或不完整的3D物体至关重要。VoTr通过自注意力机制建立体素之间的长距离依赖，并针对非空体素稀疏且数量庞大的特点，设计了稀疏体素模块和子流形体素模块，分别处理空体素和非空体素位置。为了在保持计算开销与卷积相当的同时扩大注意力范围，论文提出了局部注意力（Local Attention）和扩张注意力（Dilated Attention）两种注意力机制，并设计快速体素查询（Fast Voxel Query）加速多头注意力中的查询过程。VoTr由一系列稀疏和子流形体素模块组成，可嵌入大多数基于体素的检测框架中。在KITTI和Waymo开放数据集上的实验表明，与卷积基线相比，VoTr在保持计算效率的同时取得了持续的性能提升，例如在Waymo上车辆LEVEL-1 mAP达到74.95%，在KITTI上中等难度汽车类mAP达到82.09%。\n\n### 关键要点\n- 传统3D卷积骨干网络因感受野有限，难以捕获足够大的上下文信息，限制了3D物体检测性能，尤其是对只有少数点的小物体。\n- VoTr是首个基于Transformer的3D体素骨干网络，通过自注意力机制在体素间建立长距离依赖，替换传统3D卷积骨干。\n- 非空体素在场景中稀疏（通常占总体素不到0.1%），但数量仍然很大（如Waymo单帧约9万个），直接使用标准Transformer计算代价过高。\n- 稀疏体素模块能够在空位置输出特征，扩大非空体素空间；子流形体素模块仅对非空体素操作，保持原始3D几何结构。\n- 局部注意力关注邻近区域以保留细节信息，扩张注意力通过逐渐增大的搜索步长在不增加太多注意力体素的前提下覆盖更远范围（可达15米以上）。\n- 快速体素查询使用GPU哈希表存储非空体素索引，将每个查询的查找时间复杂度从O(N_sparse)降低到O(N_Ω)，N_Ω远小于N_sparse。\n- 在Waymo数据集上，VoTr-TSD将PV-RCNN的LEVEL-1车辆mAP从70.3%提升至74.95%，远距离（50m-Inf）mAP提升4.83%。\n- 在KITTI测试集上，VoTr-TSD的中等难度汽车类3D AP达到82.09%，硬难度达到79.14%，均优于之前的方法。\n- 消融实验表明，加入扩张注意力比仅用局部注意力提升2.79%的中等mAP；增加注意力体素数从24到48提升1.19%。\n- 在模块中移除dropout可避免性能下降，dropout概率为0.3时mAP降低8.52%。\n- VoTr-SSD相比SECOND，模型参数从5.3M降至4.8M，推理速度为14.65Hz，每帧仅增加约20ms延迟。\n- VoTr可应用于单阶段和两阶段检测器，分别基于SECOND和PV-RCNN实现VoTr-SSD和VoTr-TSD，且仅需替换骨干网络。"
}