{
  "paper_id": "96ef6f6719c1b85e437d0e0fa947506c",
  "paper_title": "Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds",
  "paper_authors": [
    "Martin Simon",
    "Karl Amende",
    "Andrea Kraus",
    "Jens Honer",
    "Timo Sämann",
    "Hauke Kaulbersch",
    "Stefan Milz",
    "Horst Michael Gross"
  ],
  "publication_date": "",
  "publication_venue": "2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)",
  "abstract": "Accurate detection of 3D objects is a fundamental problem in computer vision and has an enormous impact on autonomous cars, augmented/virtual reality and many applications in robotics. In this work we present a novel fusion of neural network based state-of-the-art 3D detector and visual semantic segmentation in the context of autonomous driving. Additionally, we introduce Scale-Rotation-Translation score (SRTs), a fast and highly parameterizable evaluation metric for comparison of object detections, which speeds up our inference time up to 20% and halves training time. On top, we apply state-of-the-art online multi-target feature tracking on the object measurements to further increase accuracy and robustness utilizing temporal information. Our experiments on KITTI show that we achieve same results as state-of-the-art in all related categories, while maintaining the performance and accuracy trade-off and still run in real-time. Furthermore, our model is the first one that fuses visual semantic with 3D object detection.",
  "keywords": [
    "3D object detection",
    "semantic segmentation",
    "point clouds",
    "real-time",
    "multi-target tracking",
    "SRTs",
    "KITTI"
  ],
  "paper_institutions": [
    "Valeo Schalter und Sensoren GmbH",
    "Ilmenau University of Technology"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "ark-code-latest",
  "prompt_template": "simple",
  "extracted_at": "2026-08-16T19:47:23.027919Z",
  "template_tag": "simple",
  "summary": "### 摘要\n本文提出Complexer-YOLO，一种在语义点云上进行实时3D物体检测与跟踪的融合方法，旨在解决自动驾驶等场景中的环境感知问题。相关研究背景包括基于CNN的2D/3D检测、语义分割及多目标跟踪。该方案将Complex-YOLO扩展为基于体素输入，融合来自相机语义分割的视觉类别特征，并增加高度和垂直偏移的回归，同时引入Scale-Rotation-Translation score（SRTs）作为快速可参数化的评估指标，替代IoU以更好处理旋转框。数据集使用KITTI基准，采用训练/验证划分和官方测试集，实现设置包括体素分辨率768×1024×21、语义特征范围[1,2]等。实验表明，在KITTI的Orientation、BEV和3D检测任务上达到与最先进方法相当的性能，3D检测Car中度难度AP为49.44%，跟踪MOTA为75.70%，且运行速度15.6 FPS，满足实时性。SRTs的使用使训练时间减半、推理速度提升10-20%。此外，模型首次融合视觉语义与3D检测。未来可探索进一步优化旋转处理和小目标检测，并应用于城市环境中的自动驾驶。核心机制包括体素化语义特征、SRTs评分公式和基于LMB RFS的扩展目标跟踪。\n### 关键要点\n- 提出Complexer-YOLO，融合相机语义分割与雷达点云，实现实时3D检测和跟踪。\n- 引入SRTs评估指标，由缩放、旋转、平移三个子分数加权组合，可参数化逼近IoU并考虑方向。\n- 使用基于体素的输入表示，每个体素填充语义类别值，结合点云空间信息和视觉上下文。\n- 网络结构基于Complex-YOLO，增加高度和z偏移回归，采用49个卷积层和残差连接。\n- 采用LMB RFS扩展目标模型，使用协调转弯运动模型和无迹卡尔曼滤波进行预测。\n- 在KITTI测试集上，Orientation Car中度AP为79.08%，BEV为66.07%，3D为49.44%。\n- 跟踪性能MOTA为75.70%，MOTP为78.46%，运行速度达到100 FPS（跟踪部分）。\n- 消融实验表明，语义特征优于RGB、占用和强度特征，mAP达35.43%（SRTs阈值0.7）。\n- 使用SRTs代替IoU训练可提升1.3% mAP，并将训练时间减半、推理速度提升10-20%。\n- 后处理中根据点数过滤检测结果，平均提升1.3%但会降低某些类别结果，表明需要权衡。\n- 使用21个高度通道与51个通道效果相当，但21通道在速度和精度上更平衡。"
}