{
  "paper_id": "ff6450c8386183873ff25c20212236ca",
  "paper_title": "People Identification in Private Car Using 3D LiDAR With Generative Image Inpainting and YOLOv5",
  "paper_authors": [
    "Weirong Shao",
    "Mondher Bouazizi",
    "Xiang Meng",
    "Tomoaki Ohtsuki"
  ],
  "publication_date": "2024-02-27",
  "publication_venue": "IEEE Access",
  "abstract": "People Identification is a critical aspect in developing modern vehicles, aimed at enhancing safety and comfort levels. Most traditional methods of people identification in vehicles use RGB images or videos. In this study, we introduce a novel methodology for identifying individuals in private car scenarios, utilizing 3D Light Detection and Ranging (LiDAR) technology, generative image inpainting based on Contextual Attention, and the YOLOv5 model. Initially, we gather data utilizing a 3D-LiDAR instrument and subsequently convert the acquired depth data into depth images. Following this, the depth images are annotated manually to indicate the positions and identifiers of various individuals occupying distinct seats. This annotated data serves as the training material for the YOLOv5 model, facilitating the recognition and categorization of subjects. However, given that individuals seated in the back often have parts of their bodies occluded by the front seats and the passengers in them, we employ generative image inpainting techniques to reveal the occluded portions. This step significantly enhances the precision in detecting and identifying individuals situated in the back seats. We implemented our strategy on a restricted group of four participants, conducting training and testing phases within identical environments. Prior to the inpainting process, the classification's F1 score stood at 66.5%. After inpainting, we observed a notable surge in the F1 score for the rear-seat passengers increased by 17.1%.\n",
  "keywords": [
    "YOLOv5",
    "GAN",
    "people identification",
    "3D LiDAR",
    "deep learning"
  ],
  "paper_institutions": [
    "Graduate School of Science and Technology, Keio University",
    "Faculty of Science and Technology, Keio University"
  ],
  "output_language": "zh",
  "provider": "doubao",
  "model": "deepseek-v4-flash",
  "prompt_template": "simple",
  "extracted_at": "2026-08-18T10:30:46.693394Z",
  "template_tag": "simple",
  "summary": "### 摘要\n\n本文针对私家车场景中的人员识别问题，提出了一种结合 3D LiDAR、基于上下文注意力的生成式图像修复和 YOLOv5 的识别方法。传统 RGB 图像或视频在暗光、恶劣天气下容易失效，且存在隐私泄露风险；3D LiDAR 通过红外激光获取深度信息，不受光照影响且不记录面部等敏感细节。方法首先用 3D LiDAR 采集深度数据，并通过动态非线性映射（DNLM）转换为深度图像，再对后排乘客被前排座椅和乘客遮挡的部分进行生成式图像修复，最后用 YOLOv5 对人员进行检测和识别。实验使用 3D+RGB IP67 Kit 设备，在模拟私家车环境中对四名参与者采集数据，设备距前排乘客小于 1.0 m、距后排约 1.5 m；图像修复模型使用 7000 张训练图像和 1000 张验证图像，YOLOv5 采用 yolov5s 权重、batch size 4、30 个 epoch、学习率 0.01 和 SGA 优化器。评估使用 F1、mAP50 和 mAP50-95 指标；修复前整体 F1 为 66.5%，修复后后排乘客 F1 提升 17.1%，后排左侧整体 F1 从 0.737 提升到 0.885，后排右侧从 0.655 提升到 0.850，且与已有方法相比达到 0.807 的准确率。消融实验表明 RGB 在明亮环境下 F1 为 0.937，但在黑暗环境下仅 0.347，而深度图像在暗光下仍保持可用。未来需要扩大数据集规模、增加参与者数量，并在真实私家车环境中验证。核心机制之一是生成修复网络采用 Wasserstein 距离 W(Pr,Pg)=inf_{γ∈Π(Pr,Pg)} E_{(x,y)~γ}[||x-y||] 作为对抗损失，以稳定训练并提升修复质量。\n\n### 关键要点\n\n- 本文提出使用 3D LiDAR 获取深度数据，并通过动态非线性映射（DNLM）转换为深度图像，以避免 RGB 图像在暗光下失效和隐私泄露问题。\n- 针对后排乘客被前排座椅和乘客遮挡的问题，采用基于上下文注意力的生成式图像修复方法，先以 Canny 边缘检测生成遮挡掩膜，再重建被遮挡的身体区域。\n- 使用 YOLOv5 作为人员检测与识别模型，其单阶段架构包含 CSPDarknet 骨干、PANet 颈部和检测头，在保证速度的同时提供较高精度。\n- 实验使用 3D+RGB IP67 Kit（Helios2 & Triton 3.2MP）采集数据，设备距前排乘客小于 1.0 m、距后排约 1.5 m，模拟真实私家车场景。\n- 数据采集包含三种场景：四名已知被试轮换座位、加入未知个体、以及为图像修复采集后排左右座位的真值数据，共形成 2000 帧/场景的规模。\n- 图像修复模型使用 7000 张训练图像和 1000 张验证图像，采用两阶段粗网络与精修网络，精修网络使用 WGAN-GP 全局和局部对抗损失及重建损失训练。\n- 在未修复深度图像上，YOLOv5 的整体 F1 为 0.665，mAP50 为 0.753，mAP50-95 为 0.626；不同个体的 F1 随遮挡率升高而下降。\n- 修复后，后排左侧乘客整体 F1 从 0.737 提升到 0.885，后排右侧从 0.655 提升到 0.850，整体后排 F1 提升 17.1%，mAP50 和 mAP50-95 也普遍提升。\n- 与已有方法相比，本文方法在自建深度图像数据集上达到 0.807 的准确率，高于 Yamada 等人的 0.718 和 Jin 等人的 0.780/0.802。\n- 消融实验显示，RGB 图像在明亮环境下 F1 为 0.937，但在黑暗环境下降至 0.347；深度图像在暗光下仍保持可用，验证了 3D LiDAR 方案在低光场景的优势。\n- 当前研究存在数据集规模较小、参与者仅四人、场景为模拟私家车且数据分布不均等限制，未来需扩大参与者数量和真实车辆环境实验。"
}