# 俯视视角下的行人检测：综述
Misbah Ahmad$^{1}$, Imran Ahmed$^{2}$, Kaleem Ullah$^{3}$, Iqbal khan$^{4}$, Ayesha Khattak$^{5}$, Awais Adnan$^{6}$ 信息技术卓越中心
巴基斯坦白沙瓦市海亚塔巴德管理科学研究所（IMSciences）
摘要——近年来，相较于正面视角，基于俯视视角的行人检测因能够处理遮挡问题、提供更好的场景覆盖范围而愈发重要。在计算机视觉领域，俯视视角行人检测在行人检测、行人计数、行人跟踪、行为分析、无遮挡监控系统等诸多应用中都具有重要意义。本文针对俯视视角下行人检测的最新进展与相关挑战开展全面综述。据我们所知，这是首次对不同的俯视行人检测技术进行综述。本文概述了当前最先进的基于俯视视角的行人检测方法，并为实际应用中选择合适的方法提供指导。本文将相关技术分为两大类：基于块的技术和基于特征的技术。同时，本文还结合多种俯视视角数据集，分析了视场、感兴趣区域、色彩空间、图像分辨率等多种检测影响因素。
关键词——行人检测；背景减除；分割；基于块的技术；基于特征的技术
## 一、引言
在视频监控领域，核心任务之一就是在机场、火车站、超市这类拥挤的公共场景中检测、识别并监控人员。近几十年来，从俯视角度在监控图像和视频序列中定位人员的方向一直受到广泛研究。与正面视角相比，基于俯视视角的人员检测更受重视，因为它能更好地应对遮挡问题，还可以覆盖更广的场景范围。基于俯视角度的人员检测在多个领域都有应用，其中最重要的应用就是监控系统。其他应用还包括室内外监控系统中的人员检测、人员计数[^1]（包括行人计数[^2]、[^3]、[^4]，以及火车站[^5]、购物中心、机场、公交车[^6]的乘客计数）、人员跟踪[^7]、[^8]、[^9]、[^6] [^10]、[^11]、行为理解[^12]、动作识别、人员姿态表征[^13]、人群分析[^14]、工业工作流[^15]、[^10]、覆盖更大更广的区域。此外，它在搜救场景中也很有帮助。使用俯视摄像头还可以减少隐私问题，因为这种方式捕获的是人体的俯视图像，而非面部图像[^16]。
在俯拍视频和图像中检测行人是一项极具挑战性的任务，影响因素包括：行人外观多样、姿态范围广、背景复杂、光照条件不受约束以及存在自遮挡。要在顶视角视频和图像中完成行人检测，必须先掌握俯视视角下人体的形状、结构和特征。检测到行人后，应用系统的性能可以得到进一步提升。
目前已有多种顶视角行人检测算法，但这些算法仅凭单一线索检测图像和视频中行人的能力仍远不及人类。
行人检测任务的主要难点之一在于人体本身具有多变性，比如人体的姿态、尺寸、朝向和方向都会发生变化。发色、发质以及服装颜色、纹理的差异也会进一步增加检测复杂度。同理，复杂的环境因素，例如背景杂乱、人群密集、光照条件多变，也会给研究人员带来阻碍。
近年来，研究者已经开发出多种俯拍行人检测技术。本文研究了俯视视角行人检测技术的研究意义。据我们所知，本文是首篇旨在涵盖迄今为止文献中报道的该领域最重要进展的综述研究。
我们从海量的俯拍行人检测相关文献中选取了具有代表性的研究成果。本文将现有已开发的技术分为两类：基于blob的技术和基于特征的技术。
如前文所述，行人检测任务可从两种不同视角开展：正面视角与俯视视角。本文内容及俯视视角行人检测的整体框架已总结于图1中。可见，已开发的技术按类别可分为两大类：基于 Blob 的技术和基于特征的技术。
图1中还展示了各类影响检测的因素，包括相机视场（窄或宽）、感兴趣区域（行人头部、头肩区域、俯视下的整个身体）、色彩空间（RGB、Depth、HSV 及 YCbCr）、视频和图像采集所用设备、采集环境（室内与室外）。此外，图1还展示了基于俯视视角的行人检测面临的挑战与应用场景。
本文剩余部分结构安排如下：第二节首先总体介绍基于俯视视角的行人检测技术。该部分分为两个小节：第一小节介绍大多数学者使用的各类基于 Blob 的技术，第二小节讨论基于特征的技术。第二节还梳理了现有文献中使用的各类不同数据集。第三节探讨现有技术、数据集以及现有基于俯视视角的行人检测相关问题。第四节对本文进行总结归纳，并给出未来研究方向。
---
Fig. 1. Overview of the paper and general frame work of Person detection.
### II. 基于俯视视角的行人检测技术
在俯视视角的图像和视频中定位行人仅为两步流程：定义ROI，并在输入图像和视频序列中对行人进行定位。在俯视场景下，图像中的行人可通过头部、头肩区域定位，部分情况下会将整个俯视视角下的人体作为检测对象。下图2中可以清晰看到人体的俯视视角，正如图2所示，我们可以清楚观察到俯视视角下行人的形状、尺寸和身体朝向的变化规律。在计算机视觉领域，研究如何在俯视图像和视频中检测行人具有重要意义。本章主要聚焦于适用于俯视视频或图像的行人检测算法，将相关研究大致分为两类方法。第二节对两类不同方法展开讨论：第一类是基于简单Blob的算法，第二个小节则探讨用于行人检测的基于特征的技术。
Fig. 2. Some of the images captured from Overhead surveillance cameras [^3], [^17], [^18], [^2].
### A. 基于Blob的技术
本节将讨论适用于俯视视角行人检测的基于Blob的技术。基于Blob技术的整体框架如图3所示，这类技术包含背景减法、前景提取、分割和预处理步骤。通常，在基于Blob的技术中，通过背景减法得到前景图像，同时会使用不同的预处理技术去除噪声、阴影和光照干扰，再通过设置阈值得到符合要求的前景图像。
Fig. 3. Blob based Person Detection.
---
从前景图像中提取出blob，进一步将其分类为单人、多人或其他物体等类别，分类基于blob的形状、颜色、运动或其他特征。下文对俯视视角下基于blob和背景差分的人体检测技术综述如下：
Cohan等人[^19]在研究中采用基础背景差分与分割方法，检测俯视图像中的人体。表I表明，大多数研究人员都采用基于背景差分与分割的方法，检测俯视图像和视频序列中的个体。在预处理阶段，主流研究人员通常使用包含腐蚀和膨胀在内的形态学算子提取目标blob。如表I所示，为去除噪声，研究中使用了高斯滤波和中值滤波方法。连通分量标记法用于提取blob，该方法被[^2]、[^20]、[^18]和[^13]采用，如图4所示。通过分析表I可以明确得出，背景差分处理后，绝大多数研究人员利用提取出的blob信息检测人体，会考虑blob的形状、颜色、边缘和尺寸等基础特征。
Fig. 4. Connected Component Labeling Method used by [^2], [^18] and [^13].
如表I所示，大多数研究人员都使用blob的形状特征。Zhang等人[^7]利用圆柱形blob检测图像中的单个人体。[^8]的作者借助图像拼接，利用blob形状信息重建了人体的半椭球头部模型，如图5(a)所示。Ozturk等人[^21]利用椭圆形blob检测输入图像中的人体（图5(b)）。
Fig. 5. Images of some shape based features [^8], [^21] used elliptical based shape, [^16] two spherical bounds to detect the person.
类似地，[^22]使用霍夫圆检测图5(c)图像中的球形斑点。这些斑点被进一步划分为两个具有相同中心点的子球形边界。内球形区域用于检测人头，外球形区域用于检测人的头肩区域，如图5(c)所示。Nakatani等人[^16]利用发旋的形状信息在俯视图像中检测人物，如图5(d)所示。
此外，研究人员还会考虑斑点的颜色信息，例如Cohan等人[^19]利用颜色信息在俯视图像中检测人体。类似地，[^17]、[^23]、[^16]、[^21]、[^24]也利用颜色信息在俯视图像中检测人物。
Fig. 6. Color information used by some of the discussed used by some of the authors a [^23], b[^16], c[^21].
如图7所示，部分研究人员[^25]、[^7]、[^16]、[^21]、[^9]和[^26]利用斑点的边缘信息检测人物。Ozturk等人[^21]和Garcia等人[^9]使用Sobel边缘法，Mukherjee等人[^26]则使用Canny边缘法，分别在图7(a)、图7(b)和图7(c)所示的俯视图像中检测人物。
Fig. 7. Sobel edge detector used by [^21] and [^9], Canny Edge detector [^26].
纹理信息也是检测过程中的一项重要因素，本文讨论的部分相关工作也利用纹理特征检测人物，例如Cohan等人[^19]、Chia等人与Nakatani [^16]利用头发纹理信息在俯视图像中检测人头，如下图8所示。
Fig. 8. Hair texture information used by [^16] [^24].
---
从表I可以看出，在俯视图像中检测行人时，感兴趣区域ROI的选择存在差异。部分研究人员以人头作为感兴趣区域检测行人，另有部分研究人员以头肩作为ROI，还有少部分研究人员将俯视视角下的整个人体作为检测用的ROI。本文讨论的部分研究人员所采用的ROI示例如图9所示。第一行中，研究人员[^16]、[^9]将头部作为ROI。第二行中，部分研究人员[^27]、[^25]、[^23]和[^18]将头肩作为ROI。本文讨论的部分研究人员[^1]、[^27]、[^21]和[^28]还将俯视下的整个人体作为ROI。
Fig. 9. The First Row shows Head as ROI [^16], [^9] the second row whose Head-shoulder [^27], [^25], [^23] & [^18] as ROI while the third in third row [^1], [^27], [^21] & [^28] whole overhead body has been considered as ROI, respectively.
表I表明，除[^12]、[^16]、[^22]、[^22]、[^8]和[^19]之外，大多数研究人员都采用俯视背景减法进行人数统计。[^25]、[^29]、[^28]和[^18]的作者采用虚拟线法对顶视图像中的人数进行统计，如下图10所示。
a
b
C
d
Fig. 10. Virtual lines used for person counting e.g (a) [^25] (b) [^28] (c) [^29] & [^18].
在监控系统中，结合追踪功能的俯视行人检测方法也已经得到发展。行人追踪工作基于俯视图像和视频序列开展，如表I所示，目前已经针对追踪需求采用了多种不同的追踪算法。
[^17]、[^1]、[^9]、[^4]与[^23]的作者使用卡尔曼滤波进行人体跟踪，而[^8]、[^17]、[^24]、[^21]与[^20]的作者使用粒子滤波对人体进行跟踪。Vera等人[^2]使用匈牙利算法进行人体跟踪。此外，[^24]使用了中值滤波，Velipasalar[^27]在跟踪中采用了均值漂移算法。Burbano等人[^6]使用基于图结构的方法进行人体跟踪。不同跟踪算法的部分示例如图11所示。
Fig. 11. Results of Different tracking algorithms used by authors discussed in this paper (a) Kalman filter [^17], (b,c,d) particle filter [^21], [^17] [^20] and (e) [^27] uses mean shift tracking algorithm, (f) [^6].
正如表I和图12所示，本文调研的多数研究都使用深度信息来检测图像中的人体，也有少部分研究采用RGB图像。[^17]和[^21]的作者在俯拍图像的人体检测中采用了HSV模型，如图12所示，仅有[^12]使用YCbCr模型进行检测。在数据录制方面，众多研究人员采用了多种多样的录制设备与传感器。
Fig. 12. First Row shows the example of depth images, (d)[^30], third row contains images of RGB color space while last row includes YCbCr [^12] and HSV [^21] color model, respectively.
---
TABLE I. BLOB BASED OVERHEAD PERSON DETECTION TECHNIQUES
| S.No | Ref | Year | Purpose | Methodology | Field of View | Region of Interest | Color Space | Data Set | Number of Images/Videos |
| 1 | [^19] | 2000 | Detection, Identification | BS, Segmentation, Gaussian Probability, Blob Color/Texture Features, Hair(Color, Texture), DTC, Bayesian Network | Narrow | Head | Gray | Indoor images, Low Resolution, | 720 |
| 2 | [^17] | 2005 | Detection, Counting, Tracking | BS, Segmentation, Blob shape and color features, RGB-Gray conversion, Blob statistical features, Kalman Filter | Narrow | Overhead Body | RGB and HSV | Indoor images, $384 \times 288$ | 13138 |
| 3 | [^1] | 2008 | Detection, Counting, Tracking | BS, Segmentation, Binary Image Conversion, Morphological (Opening), Kalman Filter | Narrow | Head-Shoulder | Depth | Indoor | 150 Video Sequences |
| 4 | [^27] | 2006 | Detection, Counting | BS, Blob Statistical Features (Mean & Standard Deviation), Mean Shift Tracking Algorithm | Narrow | Overhead Body | RGB | Indoor, $320 \times 240$ | 6 Video Sequences (3-5 min) |
| 5 | [^25] | 2008 | Detection, Counting | BS, Forground Background Edge Model, RGB-Gray conversion, Canny edge detector, Bounding Box, Virtual lines for counting, Tracking (Mass Center) | Narrow | Overhead Body | RGB | Indoor, $320 \times 240$, $640 \times 480$ | Multiple Video clips |
| 6 | [^7] | 2008 | Detection, Counting | BS, Frame Differencing, blob based 3D human cylindrical model(Head, torso, leg), Camera Calibration, Matching Score(size, shape, visibility) | Narrow | Overhead Body | RGB | Indoor, Multiple Sensor Based Cameras, $320 \times 240$ | 38 hrs video clip 50fps |
| 7 | [^12] | 2012 | Detection, Behaviour Understanding | BS, GMM, Geometric transformation, Hu moments, Behaviour feature (squat, stand, standing) | Narrow | Overhead body | YCbCr | Indoor images | 300(100 sample of each posture) |
| 8 | [^23] | 2013 | Detection, Counting | BS, segmentation, GMM, Blob color features, U-Disparity, Kalman & particle filter | Narrow | Head | RGB, Depth | Indoor RGB and Depth images | Video sequences |
| 9 | [^8] | 2014 | Detection, Tracking | BS, Graph based segmentation, Image stitching, Head hemiillisoid model, geodesic model distance map, particle tracking filter | Narrow | Head-shoulder | RGB, Depth | Indoor images | 888 |
| 10 | [^16] | 2012 | Detection, Identification | BS, Shadow & Noise removal, Blob features (size, color), Hair (color, whorl) | Narrow | Head | Gray | Indoor images, $400 \times 500$ | 240 |
---
| S.No | Ref | Year | Purpose | Methodology | Field of View | Region of Interest | Color Space | Data Set | Number of Images/Videos |
| 11 | [^21] | 2009 | Detection, Tracking | BS, Blob shape features (Elliptical), Blob(color, histogram orientation), Sobel edge detection, Morphological(opening), Median & particle filter | Narrow | Overhead | HSV | Indoor, $1440 \times 10^{80}$\nHD, Human Model( $70 \times 90$) | Video clips |
| 12 | [^9] | 2011 | Detection, Counting, Tracking | BS, Segmentation, Blob Shape, Gaussian Distribution, Morphological, Blob separation, Connected component labeling | Narrow | Head-Shoulder | RGB, Depth | Indoor RGB, Depth, Color images | Video clip ( 292 frames) |
| 13 | [^2] | 2016 | Detection, Counting | BS, Segmentation, Blob Shape, Morphological(Filling), Camera Calibration, hungerian Tracking Algorithm | Narrow | Head | Depth, RGB | Indoor RGB & Depth images, $240 \times 320$ | 2384 |
| 14 | [^31] | 2016 | Detection, Counting | BS, Gaussian Smoothing, Virtual line | Narrow | Head | RGB, Depth | Indoor and Outdoor, $640 \times 480$, $320 \times 240$, $160 \times 120$ | Video Sequences |
| 15 | [^32] | 2017 | Detection | BS, Connected component labeling, morphological(opening), Blob(split merge) | Narrow | Overhead body | RGB, Depth | Indoor and outdoor images $640 \times 480$ | Indoor = 33599, Outdoor = 37799 |
| 16 | [^33] | 2018 | Detection, Counting | BS, Gaussian filter, MOG, ROC | Narrow | Overhead | RGB, Depth | Indoor images | 4500 Video Sequences |
| 17 | [^20] | 2018 | Detection, Counting, Tracking | BS, Depth images subtraction, Gaussian Filter, particle Filter | Narrow | Head-shoulder | Depth | Indoor images | |
| 18 | [^26] | 2011 | Detection, Counting | Segmentation, Consecutive frame differencing, Aspatio-temporal background subtraction AMBS, Blob (shape.edge) features, Hough Circle, Canny edge detector, Optical flow(tracking) | Narrow | Head | RGB | Indoor, $480 \times 640$ | Video Sequences (2000 & 5000) samples |
| 19 | [^34] | 2010 | Detection, Counting | BS, Segmentation, Blob features(shape, size, velocity, orientation) extraction, spatial temporal map, linear regression model, optical flow | Narrow | Overhead body | RGB | Indoor&Outdoor $640 \times 240$& $240 \times 320$ | Video Sequences (30 min 25fps & 2hr 30fps) |
| 20 | [^9] | 2013 | Detection, Counting, Tracking | BS, Segmentation, Equalized difference image, Dilation, Erosion, Circular patterns, Sobel Edge Detector, Kalman Filter | Narrow | Head | Gray | Indoor, $320 \times 240$ | Video Sequence |
| 21 | [^9] | 2013 | Detection, Counting | BS, For ground Extraction, Segmentation, Kalman Filter | Narrow | Head | RGB | Indoor | - |
---
| S.NO | Ref | Year | Purpose | Methodology | Field of View | Region of Interest | Color Space | Data Set | Number of Images/Videos |
| 22 | [^35] | 2014 | Detection, Tracking | BS, MOG, Forground Mask, Erode, Di-late, contour threshold | Narrow | Overhead body | RGB, YUV | Indoor 320 $\times$ 240 | Video sequences |
| 23 | [^5] | 2014 | Detection, Counting | BS, Segmentation, Head candidate de-tection, Local maxima detection | Narrow | Head | Depth | Indoor 640 $\times$ 480 | 500 images |
| 24 | [^6] | 2015 | Detection, Counting, Tracking | BS, Light weight segmentation, Estima-tion of relative position & orientation, Blob size features, graph structure rep-resentation, camera calibration | Narrow | Head-shoulder | Depth | Indoor 640 $\times$ 480 | |
| 25 | [^14] | 2015 | Detection, Counting | Color Based BS, Segmentation, Mor-phological Operations (Dilation, Ero-sion), Local Minima estimation, Blob grouping | Narrow | Head-shoulder, torso, leg | Color Depth | Indoor | 268 images |
| 26 | [^28] | 2016 | Detection, Counting | BS, Forground extraction, Blob merg-split, Gaussian model for noise removal, counting zone | Narrow | Overhead Body | RGB | Indoor 1080 $\times$ 720 | HD Video Sequence |
| 27 | [^4] | 2016 | Detection, Counting | Adaptive background model, Segmen-tation, Reconstruction of 3D points, Kalman Filter | Narrow | Body | Depth | Outdoor Thermal Images 640 $\times$ 480 | Two Five min videos |
| 28 | [^24] | 2016 | Detection, Counting | Color based BS, Water Filling , GMM, Detected Regions Shape., Color gradient model, Median filter | Narrow | Overhead body | Depth | Indoor | 11890 images |
| 29 | [^18] | 2016 | Detection, Counting | BS, Frame differencing, Morphologi-cal operations(Dilation, Erosion), Con-nected component labeling, Counting zone, Virtual line, Euclidean Distance | Narrow | | Overhead Body RGB | Indoor 320 $\times$ 240 | Video 30fps |
| 30 | [^13] | 2016 | Detection, Action recognition | BS, Segmentation, Forground extrac-tion, Connected component labeling, Action recognition (sitting down, stand-ing up, lying down, waking up, bending and falling on ) | Narrow | Overhead Body | Depth | Indoor | Video sequence |
---
### B. 基于特征的技术
近几十年来，随着计算机视觉与机器学习的发展，各类基于特征的技术凭借其鲁棒性和出色的检测精度得到了广泛重视。本文调研的适用于俯视角人体检测的部分基于特征的技术已在表II中汇总。基于特征的人体检测的通用框架如下图13所示。
这些技术作用于从俯视视角视频和图像中提取得到的特征，提取出的特征会进一步用于对人物图像和非人物图像进行分类。提取得到的特征包含形状（以轮廓、边缘、角点或其他描述符的形式呈现）、颜色和纹理（头发或衣物的颜色纹理）、方向、朝向或运动信息，有时还会结合上述多种信息。如表II所示，已有研究使用基于特征的算法（包括SIFT和HOG算法）检测俯视视角图像中的人物。图像通常会被划分为训练样本和测试样本，这些样本随后会被输入机器学习分类器，即SVM、adaboost、KNN。
Fig. 13. General Frame Work of Feature based Person Detection.
方向梯度直方图是目前受关注、被优先选用且应用广泛的基于特征的人物检测算法之一 [^36]。这种基于特征的方法会统计图像局部区域内梯度方向的出现次数。针对俯视视角人物检测，Pang等人在 [^37] 中提出了一种高效的基于HoG特征的人体检测方法，如图14所示。Pang等人 [^38] 提出了另一种同样使用俯视图的人体特征提取技术，该方法主要研究基于HOG的人体检测方法中的方向梯度共现直方图。Rauter等人 [^11] 提出了一种基于局部三值模式特征的人体检测与跟踪方法，用于训练SVM的主要特征是人体的头部和肩部。
Ahmed等人[^15]在工业环境的俯视图像中应用了Rotated-HoG算法，如图14所示。Ahmed等人[^39]提出了另一种鲁棒算法，该算法使用广角镜头相机采集人物的俯视视角图像。所提算法采用了具有不同朝向的可变尺寸边界框，研究人员利用RHOG从检测窗口或边界框图像中提取特征，并在线性支持向量机的辅助下构建学习训练模型。[^24]同样在背景减除后使用HOG特征，以在复杂环境中检测并统计人数。
另一种常用的基于特征的算法是SIFT [^40]。Ozturk等人[^21]利用SIFT特征的光流观测人体和头部的朝向变化。从表II中可以清晰看出，SVM是目前最受欢迎的分类器。表II显示，为了检测图像中的人物，学者们使用了一种名为检测窗口的核或样本（如图14所示）。检测窗口的尺寸会随人物的身高和体宽变化。同理，[^37]使用$80 \times 120$尺寸的检测窗口在俯视图像中检测人体。随着人物与相机距离增加，人物的成像尺寸会发生变化，因此[^15]的作者采用了可变尺寸检测窗口。[^11]的作者结合CoHOG，使用$64 \times 128$尺寸的检测窗口在俯视图像中检测人物。[^41]和[^3]的作者采用了多种尺寸的检测窗口，分别使用$4 \times 4$、$8 \times 8$、$12 \times 12$以及$64 \times 96$、$96 \times 96$在俯视图像中检测头肩区域和人体。
Fig. 14. Example of feature based person detection images taken from [^37][^41] [^3], [^15], respectively.
表II显示，为了降维特征提取后的特征向量，[^11] 使用了PCA方法。为缩小特征向量尺寸，[^41] 采用了均值、标准差这类统计度量。归一化处理中使用了非极大值抑制NMS，提升了所提模型的性能。在基于特征的行人检测技术中，本章研究的大多数工作都以多种方式应用了非极大值抑制（如表II所示）。
---
TABLE II. FEATURE BASED OVERHEAD PERSON DETECTION TECHNIQUES
| S.no | Ref | Year | Purpose | Methodology | Classifier | Field of View | Region of Interest | Color Space | Data Set | Number of Images/Videos |
| 1 | [^21] | 2009 | Detection, Tracking | Motion Vector Detection, SIFT Optical Flow,(Around head region) SOFV vector, particle filter. | | Narrow | Overhead | HSV | Indoor, $1440 \times 10^{80}$\nHD, Human Model( $70 \times 90$) | Video clips |
| 2 | [^37] | 2011 | Detection | HoG, Detection Window Size $80 \times 120$, Sub-cell based interpolation algorithm, Non-maximum suppression | SVM | Narrow | Overhead Body | RGB | Indoor and Semi outdoor images, $320 \times 240$ | Almost 1000 images |
| 3 | [^15] | 2012 | Detection | Rotated HoG, Variable Size Bounding Box, Non-maximum suppression, Disparity Check between actual and detected positions | SVM | Wide | Overhead body | RGB | Indoor, Industrial SCO-VIS data set, $640 \times 480$ | 13000 images |
| 4 | [^38] | 2012 | Detection | Co-HoG, Sliding Window, CoHOG feature vector is very large (200 000), Incremental principal component analysis (IPCA), non maximum-suppression, Detection Window Size $64 \times 128$) | SVM | Narrow | Overhead body | RGB | Indoor images $320 \times 240$, INRIA + & Overhead | 6680 images |
| 5 | [^11] | 2013 | Detection, Tracking | Depth map for local maxima Local Ternary patterns, Non-Maxima Suppression, Head trajectories | SVM | Narrow | Head-shoulder | Depth | Indoor images | 30780 images |
| 6 | [^24] | 2016 | Detection, Counting | HoG, Color gradient Model, Median filter | SVM & KNN | Narrow | Head | Depth | Indoor | 15188 images & 11890 images |
| 7 | [^41] | 2016 | Detection | SLTP and Rauter, Sliding detection window, Statistical Features $4 \times 4$, $8 \times 8$, and $12 \times 12$ | SVM | Narrow | Head-shoulder | Depth | Indoor | 2382 images |
| 8 | [^3] | 2017 | Detection | bounding box, Original Greedy Non-maxima suppression | CNN, R-CNN | Narrow | Overhead body | RGB-Depth | Indoor $800 \times 600$ | 850 images |
| 9 | [^39] | 2018 | Detection | HoG, RHoG, Bounding box, $64 \times 96$, $96 \times 96$, Non-maxima suppression | SVM | Wide | Overhead body | RGB | Indoor SCOVIS industrial, $640 \times 480$ | 26032 |
---
为了提升基于特征的行人检测技术的准确率和效率，Christian Ertler等人[^3] 对当前最优的Faster R-CNN检测模型提出了两点扩展改进。他们在所提模型的不同层对RGB和深度表征进行了融合。实验结果表明，最适合进行融合的层级是中间层。
### III. 讨论
上述讨论表明，现有文献中的大部分研究工作都是基于背景差分技术开展的。大多数研究人员采用简单的背景差分和分割方法检测俯视图像中的ROI，包括人头、头肩和人体。这些技术仅在研究者自行构建的数据集上有效。如表I所示，大多数研究者都自行构建了用于俯视视角行人检测的室内数据集。部分数据集是在约束环境、简单背景下采集得到的，具体可见Fig. 10、Fig. 11和Fig. 12等。部分方法仅能在行人正好位于摄像头正下方时检测出俯视图像中的行人，如Fig. 7、Fig. 8和Fig. 9所示。现有研究大多利用基本的Blob特征，即形状、纹理、颜色和尺寸信息，来检测俯视图像中的行人。由于每种方法都是根据环境要求、光照条件和应用需求，在自行开发的数据集上完成测试，因此无法对各类基于Blob的开发技术的准确率进行比较。
基于背景减除法的方法检测精度较高，但大多会受诸多问题困扰，包括渐变（昼夜）和突变光照变化、背景几何变化（其他物体进入场景）、相机相关问题（相机振动、噪声、运动模糊）、阴影因素、衣物颜色与背景相同，有时还会出现人体自身遮挡的问题。本文还讨论了部分基于特征的俯视人体检测方法。基于特征的方法可以提高俯视人体检测方法的准确率和鲁棒性。但由于这类方法使用的是自制数据集，其准确率的参考价值有限，因为这些数据集大多是在简单背景、低光照条件下采集的。部分研究人员使用了如图14所示的高度复杂工业环境开展研究。近年来，随着使用基于特征的方法提升算法效率成为趋势，且机器学习算法不断发展，该方向研究可以借助深度学习进行扩展，进一步提升方法鲁棒性。基于特征的方法鲁棒性较好，但有时会增加研究人员的计算成本。此外，特征向量的尺寸有时过大，会给研究人员带来复杂性问题。
本文梳理了用于采集俯视图像的各类方法，这些方法采用不同的记录设备：部分方法使用Kinect这类单传感器设备，其他方法则使用多传感器设备。本文梳理的多数研究都采用了如图10、图11和图12所示的窄视场方案。
### IV. 结论
本文对已有文献中不同的俯视角行人检测方法进行了综述，同时也讨论了俯视角行人检测方法面临的各类问题。本文梳理的研究文献按照基于连通域和基于特征分为两类。经分析我们发现，本文梳理的绝大多数研究都基于背景减法，仅有少数研究探讨了基于特征的俯视角行人检测方法。本文还指出了与数据集相关的各类挑战和问题，包括噪声、复杂杂乱背景以及自遮挡。基于上述讨论，我们得出结论：俯视角行人检测领域仍有广阔的全新研究与开发空间。由于现有研究多使用自行录制的数据集，无法对上述方法的准确率进行统一评估。本文同时提出了未来研究的新方向，部分可行方向如下：已开发的技术应当在统一基准数据集上进行验证；现有大多数数据集都是在约束环境和限定光照条件下录制的；未来研究可探索多种基于特征的方法；特征分类可采用先进机器学习算法，让行人检测更高效、更鲁棒。
## 致谢
我们感谢巴基斯坦管理科学研究所与巴基斯坦高等教育委员会（HEC）通过NRPU项目为本研究提供资助，项目编号为5840/KPK/NRPU/RND/HEC/2016。
## 参考文献
[^1]: T. Yahiaoui, C. Meurie, L. Khoudour, and F. Cabestaing, “A people counting system based on dense and close stereovision,” in International Conference on Image and Signal Processing. Springer, 2008, pp. 59–66.
[^2]: P. Vera, S. Monjaraz, and J. Salas, “Counting pedestrians with a zenithal arrangement of depth cameras,” Machine Vision and Applications, vol. 27, no. 2, pp. 303–315, 2016.
[^3]: C. Ertler, H. Posseger, M. Optiz, and H. Bischof, “Pedestrian detection in rgb-d images from an elevated viewpoint,” in 22nd Computer Vision Winter Workshop, 2017.
[^4]: M. Kristoffersen, J. Dueholm, R. Gade, and T. Moeslund, “Pedestrian counting with occlusion handling using stereo thermal cameras,” Sensors, vol. 16, no. 1, p. 62, 2016.
[^5]: F. Malawski, “Top-view people counting in public transportation using kinect,” Challenges of Modern Technology, vol. 5, 2014.
[^6]: A. Burbano, S. Bouaziz, and M. Vasiliu, "3d-sensing distributed embedded system for people tracking and counting," in 2015 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, 2015, pp. 470–475.
[^7]: Z. Zhang, P. L. Venetianer, and A. J. Lipton, “A robust human detection and tracking system using a human-model-based camera calibration,” in The Eighth International Workshop on Visual Surveillance-VS2008, 2008.
[^8]: T.-E. Tseng, A.-S. Liu, P.-H. Hsiao, C.-M. Huang, and L.-C. Fu, "Real-time people detection and tracking for indoor surveillance using multiple top-view depth cameras," in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014, pp. 4077–4082.
[^9]: J. García, A. Gardel, I. Bravo, J. L. Lázaro, M. Martínez, and D. Rodríguez, “Directional people counter based on head tracking,” IEEE Transactions on Industrial Electronics, vol. 60, no. 9, pp. 3991–4000, 2013.
[^10]: I. Ahmed, A. Ahmad, F. Piccialli, A. K. Sangaiah, and G. Jeon, "A robust features-based person tracker for overhead views in industrial environment," IEEE Internet of Things Journal, vol. 5, no. 3, pp. 1598–1605, 2018.
## [^11]: M. Rauter, “Reliable human detection and tracking in top-view depth images,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013, pp. 529–534.
---
[^12]: Q. Lin, C. Zhou, S. Wang, and X. Xu, “Human behavior understanding via top-view vision,” AASRI Procedia, vol. 3, pp. 184–190, 2012.
[^13]: T.-W. Hsu, Y.-H. Yang, T.-H. Yeh, A.-S. Liu, L.-C. Fu, and Y.-C. Zeng, "Privacy free indoor action detection system using top-view depth camera based on key-poses," in 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2016, pp. 004058–004063.
[^14]: D. Ryan, S. Denman, S. Sridharan, and C. Fookes, “An evaluation of crowd counting methods, features and regression models,” Computer Vision and Image Understanding, vol. 130, pp. 1–17, 2015.
[^15]: I. Ahmed and J. N. Carter, “A robust person detector for overhead views,” in Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). IEEE, 2012, pp. 1483–1486.
[^16]: R. Nakatani, D. Kouno, K. Shimada, and T. Endo, “A person identification method using a top-view head image from an overhead camera.” JACIII, vol. 16, no. 6, pp. 696–703, 2012.
[^17]: L. Snidaro, C. Micheloni, and C. Chiavedale, “Video security for ambient intelligence,” IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 35, no. 1, pp. 133–144, 2005.
[^18]: J.-W. Perng, T.-Y. Wang, Y.-W. Hsu, and B.-F. Wu, “The design and implementation of a vision-based people counting system in buses,” in 2016 International Conference on System Science and Engineering (ICSSE). IEEE, 2016, pp. 1–3.
[^19]: I. Cohen, A. Garg, and T. S. Huang, “Vision-based overhead view person recognition,” in Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol. 1. IEEE, 2000, pp. 1119–1124.
[^20]: R. Iguernaissi, D. Merad, and P. Drap, “People counting based on kinect depth data.” in ICPRAM, 2018, pp. 364–370.
[^21]: O. Ozturk, T. Yamasaki, and K. Aizawa, “Tracking of humans and estimation of body/head orientation from top-view single camera for visual focus of attention analysis,” in 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. IEEE, 2009, pp. 1020–1027.
[^22]: T. Van Oosterhout, S. Bakkes, B. J. Kröse et al., “Head detection in stereo data for people counting and segmentation.” in VISAPP, 2011, pp. 620–625.
[^23]: C. Wateosot, N. Suvonvorn et al., “Top-view based people counting using mixture of depth and color information,” in The second asian conference on information systems, ACIS. Citeseer, 2013.
[^24]: C. Gao, J. Liu, Q. Feng, and J. Lv, “People-flow counting in complex environments by combining depth and color information,” Multimedia Tools and Applications, vol. 75, no. 15, pp. 9315–9331, 2016.
[^25]: S. Yu, X. Chen, W. Sun, and D. Xie, “A robust method for detecting and counting people,” in 2008 International Conference on Audio, Language and Image Processing. IEEE, 2008, pp. 1545–1549.
[^26]: S. Mukherjee, B. Saha, I. Jamal, R. Leclerc, and N. Ray, “Anovel framework for automatic passenger counting,” in 2011 18th IEEE
国际图像处理会议。IEEE，2011年，第2969–2972页。
[^27]: S. Velipasalar, Y.-L. Tian, and A. Hampapur, “Automatic counting of interacting people by using a single uncalibrated camera,” in 2006 IEEE International Conference on Multimedia and Expo. IEEE, 2006, pp. 1265–1268.
[^28]: J. Cao, L. Sun, M. G. Odoom, F. Luan, and X. Song, “Counting people by using a single camera without calibration,” in 2016 Chinese Control and Decision Conference (CCDC). IEEE, 2016, pp. 2048–2051.
[^29]: L. Del Pizzo, P. Foggia, A. Greco, G. Percannella, and M. Vento, "A versatile and effective method for counting people on either rgb or depth overhead cameras," in Multimedia & Expo Workshops (ICMEW), 2015 IEEE International Conference on. IEEE, 2015, pp. 1–6.
[^30]: J. Nalepa, J. Szymanek, and M. Kawulok, “Real-time people counting from depth images,” in International Conference: Beyond Databases, Architectures and Structures. Springer, 2015, no 387–307
[^31]: L. Del Pizzo, P. Foggia, A. Greco, G. Percannella, and M. Vento, “Counting people by rgb or depth overhead cameras,” Pattern Recognition Letters, vol. 81, pp. 41–50, 2016.
[^32]: V. Carletti, L. Del Pizzo, G. Percannella, and M. Vento, “An efficient and effective method for people detection from top-view depth cameras,” in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 2017, pp. 1–6.
[^33]: H. Song, S. Sun, N. Akhtar, C. Zhang, J. Li, and A. Mian, “Benchmark data and method for real-time people counting in cluttered scenes using depth sensors,” arXiv preprint arXiv:1804.04339, 2018.
[^34]: Y. Benabbas, N. Ihaddadene, T. Yahiaoui, T. Urruty, and C. Djeraba, “Spatio-temporal optical flow analysis for people counting,” in 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE, 2010, pp. 212–217.
[^35]: L. Maddalena, A. Petrosino, and F. Russo, “People counting by learning their appearance in a multi-view camera environment,” Pattern Recognition Letters, vol. 36, pp. 125–134, 2014.
[^36]: N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in international Conference on computer vision & Pattern Recognition (CVPR'05), vol. 1. IEEE Computer Society, 2005, pp. 886–893.
[^37]: Y. Pang, Y. Yuan, X. Li, and J. Pan, “Efficient hog human detection,” Signal Processing, vol. 91, no. 4, pp. 773–781, 2011.
[^38]: Y. Pang, H. Yan, Y. Yuan, and K. Wang, “Robust cohog feature extraction in human-centered image/video management system,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 42, no. 2, pp. 458–468, 2012.
[^39]: I. Ahmed and A. Adnan, “A robust algorithm for detecting people in overhead views,” Cluster Computing, pp. 1–22, 2017.
[^40]: D. G. Lowe, “Object recognition from local scale-invariant features,” in ICCV. IEEE, 1999, p. 1150.
[^41]: T.-W. Choi, D.-H. Kim, and K.-H. Kim, “Human detection in top-view depth image,” Contemporary Engineering Sciences, vol. 9, no. 11, pp. 547–552, 2016.