Fig. 1. Functional diagram illustrating the common modules in a LIDAR-based perception system for pedestrian detection. Along this paper, the bold blocks will be emphasized.
accuracy (Acc), and whose results are presented in Section V. These are the main contributions of the present work.
### II. DATA PREPROCESSING
The raw range-points have to be processed in advance before the calculation of the feature vector and the subsequent object classification. The tasks performed in this preprocessing module are: pre-filtering, coordinate transformation, and segmentation.
Prefiltering aims to filter the incoming raw data in order to detect isolated/spurious range-points, to discard measurements that occur out a predefined field of interest, and to perform any pertinent task to reduce the complexity and the processing-time. Coordinate transformation, in our case, is a conversion from polar to the Cartesian coordinates, and finally the segmentation stage, which constitutes a critical part in such perception systems, can be performed by means of specific methods as presented in [^6], [^11], [^8].
For allowing a better generalization of the methods presented here, all the range information were considered as two-dimensional measurements; that is the case of the Sick LMS/LD lasers. For multi-layers LIDAR (e.g. Ibeo Alasca), one can project the scanned points to a single reference plane.
Expressing a 2D full scan as a sequence of $N_S$ measurement points in the form $Scan = \{(r_l, \alpha_l) | l = 1, ..., N_S\}$, where $(r_l, \alpha_l)$ denotes the polar coordinates of the $l^{th}$ scan point, a group of scan points that constitute a segment $S_k$ can be expressed as
$$ S_{k}=\{(r_{n},\alpha_{n})\},\quad n\in[l_{i},l_{f}],\quad n=1,\ldots,np $$
where np is the number of points in the current segment, $l_i$ and $l_f$ are the initial and the final scan points that define the segment. A segment can also be defined in Cartesian coordinates $\mathbf{x} = (x_k, y_k)$, where $(x_k = r_n \cos \alpha_n, \, y_k = r_n \sin \alpha_n)$. Henceforth, a segment is explicitly related to a group of range-points related to one, unambiguously, object of interest and expressed by $S_k$.
As the objective of this paper is to demonstrate the feasibility on pedestrian detection based on features calculated from laser segments, all the experiments were conducted using data sets where the segments were extracted under user supervision, avoiding some problems invariably presented on realistic situations, such as: data association errors, oversegmentation, measurements missing, tracking inconsistencies, etc. Therefore, our interest is in the situation where an entity, pedestrian or non-pedestrian, has been already
LIDAR FEATURES FOR PEDESTRIAN DETECTION.
| $f_i$ | Formula | Description |
| 1 | $np \cdot r_{min}$ | $r_{min}$ is the minimum range distance |
| 2 | np | Number of points |
| 3 | $\sqrt{\Delta X^{2} + \Delta Y^{2}}$ | Normalized Cartesian dimension |
| 4 | $\sqrt{\frac{1}{np} \sum_{n=1}^{np} \|\mathbf{x}_n - \bar{\mathbf{x}}\|}$ | Standard deviation: $\mathbf{x}_n$ is the range-points array and $\bar{\mathbf{x}}$ the centroid |
| 5 | Radius $\leftarrow$ fitted circle | Radius: denotes the radius of a circle extracted from $\mathbf{x}_n$ using Guivant's method [^12] |
| 6 | $\frac{1}{np} \sum_{n=1}^{np} \|\mathbf{x}_n - \tilde{\mathbf{x}}\|$ | Mean average deviation from the median $\tilde{\mathbf{x}}$ |
| 7 | IAV | The Inscribed Angle Variance (IAV), proposed by [^4] |
| 8 | std(f7) | Standard deviation of the inscribed angles |
| 9 | $\frac{1}{np} \sum_{n=1}^{np} (\mathbf{x}_n - \hat{\mathbf{x}}_{l,n})^2$ | Linearity (see [^3]), where $\hat{\mathbf{x}}_{l,n}$ corresponds to the fitted line |
| 10 | $\frac{1}{np} \sum_{n=1}^{np} (\mathbf{x}_n - \hat{\mathbf{x}}_{c,n})^2$ | Circularity (see [^3]), with $\hat{\mathbf{x}}_{c,n}$ corresponding to the fitted circle |
| 11 | $\sum_{n=1}^{np} \frac{(\mathbf{x}_n - \mu_x)^2}{np}$ | $2^{th}$ central moment; $\mu_x$ is the mean |
| 12 | $\sum_{n=1}^{np} \frac{(\mathbf{x}_n - \mu_x)^3}{np}$ | $3^{th}$ central moment |
| 13 | $\sum_{n=1}^{np} \frac{(\mathbf{x}_n - \mu_x)^4}{np}$ | $4^{th}$ central moment |
| 14 | $\sum_{n=1}^{np} \|\mathbf{x}_n - \mathbf{x}_{n-1}\|$ | Segment length (see [^3]) |
| 15 | std(f14) | Standard deviation of the segment length |
detected and the referred detected object, characterized as a segment, should be classified.
### III. FEATURE EXTRACTION
Feature extraction from LIDAR data and its utilization for pedestrian detection in urban environment is a subject that has not been investigated significantly, although some works are worth of mention: [^6], [^1], [^10]. Nevertheless, in the mobile robotics field, the work by [^3] is a reference on using purely LIDAR features $^{2}$ for human detection in indoor environments. The components of the feature vector, many of them based on Arras's work, are summarized in Table I.
The feature vector extracted from a segment $S_{k}$ is calculated using only 2D information in polar and/or Cartesian space, hence as said previously for the case of a multi-layer LIDAR the “vertical” information has to be projected on a common 2D plane, which means that all these features can be used in case of single-layer lasers. As a demonstration of a pedestrian detected by the LIDAR, with its corresponding segment and the region of interest (ROI) in the image plane, consider Figure 2. The related laser-features are highlighted in the left part of the Figure.
An important aspect when designing a classifier is to decide the features to be used and, if possible, the combination
---
Fig. 2. An example that illustrates a pedestrian perceived by the laser, as a segment of range points, and the corresponding laser-features. The image is depicted to facilitate the understanding of the scene.
in which it will be applied during the classification process. To prevent redundancy and to take advantage of the diversity among the features, an approach based on mutual information was used for feature selection. We have adopted the method named mRMR (minimum-redundancy maximum-relevancy) [^13] which is based on information theory. Instead of maximizing the dependency $\max D(S, c)$ directly, where S is the feature set which jointly have the largest dependency on the target class c, Peng developed his method based on maximal relevance criteria, which approximates $\max D(S, c)$ using the mean value of the mutual information $I(f_i/c)$ taken from individual features $f_i$:
$$ max D(S,c),D=\frac{1}{|S|}\sum_{f_{i}\in S}I(f_{i}/c) $$
Moreover, together with the max-relevancy, the mRMR method combines another condition to obtain mutually exclusive features. The later criteria is based on the minimal redundancy $\min R(S)$ among the features:
$$ min R(S),R=\frac{1}{|S|^{2}}\sum_{f_{i},f_{j}\in S}I(f_{i},f_{j}) $$
Those features with maximum relevancy and minimal redundancy using the Ibeo laser data set are (refer to Table I): $f_{i}=[1,10,15,6,4,3,9,12,11,5,13,7,14,8,2]$, and those using the Sick laser: $f_{i}=[1,10,15,8,14,11,9,5,12,13,2,6,4,7,3]$. These sequence of features were used during the classifier's validation to determine the number of features with the best performance in terms of the area under ROC curve (AUC) and accuracy (Acc); the results are present in Subsection V-B. It is important to note that these sequences of features are independent, in terms of information theory, of the type of classifier, resulting in a consistent and promising direction for future research.
### IV. PEDESTRIAN DETECTION
Five classifiers, used for pedestrian detection, are discussed in this Section: Naive Bayes, GMMC, MCI-NN, FLDA and RBF-SVM.
### A. Naive Bayes
Based on the assumption that each feature is statistically independent, the probability density function (pdf) that characterizes the object class m is modeled as the product of each feature-based pdf. A one-dimensional Gaussian $\theta_{k}(\mu_{k},\sigma_{k})$ was considered in modeling each pdf,
$$ p(x_{k}|q_{m},\theta_{k})=\frac{1}{\sigma_{k}\sqrt{(2\pi)}}\exp[\frac{-(x_{k}-\mu_{k})^{2}}{2\sigma_{k}^{2}}] $$
where $\mu_{k}$ is the mean and $\sigma_{k}$ is the statistical variance for the $k^{th}$ feature $x_{k}$, and $q_{m}$ corresponds to the “object” class of interest, i.e. pedestrians $q_{1}$ and non-pedestrians $q_{2}$.
For the case $q_{1}$, the likelihood $\mathcal{L}$ is obtained by the normalization
$$ \mathcal{L}(x_{k}|q_{1})=\frac{p(x_{k}|q_{1},\theta_{k})}{p(x_{k}|q_{1},\theta_{k})+p(x_{k}|q_{2},\theta_{k})} $$
and therefore, the combined likelihood is expressed by
$$ \mathcal{L}(x|q_{1})=\prod_{k=1}^{d}\mathcal{L}(x_{k}|q_{1}) $$
where x corresponds to a d-dimensional feature vector.
### B. GMMC
For the GMMC classifier, the likelihood is calculated considering a mixture of M Gaussian pdf, defined by $\Theta_{i}(\rho_{i},\mu_{i},\Sigma_{i})$, where $\rho_{i}$ is a weight-vector, such that $\sum_{i=1}^{M}\rho_{i}=1$, $\mu_{i}$ is the d-dimensional mean vector, and $\Sigma_{i}$ is the covariance matrix. The pdf for a single component i is modeled as
$$ p(x|q_{m},\Theta_{i})=\frac{\exp[-\frac{1}{2}(x-\mu_{i})^{T}(\Sigma_{i})^{-1}(x-\mu_{i})]}{\sqrt{(2\pi)^{d}}} $$
Finally, the likelihood is the linear composition
$$ \mathcal{L}(x|q_{1})=\sum_{i=1}^{M}\rho_{i}\cdot p(x|q_{1},\Theta_{i}) $$
### C. MCI-NN
Minimization of InterClass Interference (MCI) [^14] is a maximum-margin based training algorithm for Neural Networks (NN). MCI aims to create a NN hidden layer output (i.e. feature space) where the patterns have a desirable statistical distribution. Regarding the neural architecture, the linear output layer is replaced by the Mahalanobis kernel in order to improve generalization. MCI is applicable on a neural network model with two sigmoidal hidden layers and one output non-linear layer:
$$ yhf=\varphi(W_{1}\cdot x+b_{1}) $$
$$ yhs=\varphi(W_{2}\cdot yhf+b_{2}) $$
$$ \widehat{y}=\frac{d_{2}-d_{1}}{d_{2}+d_{1}} $$
---
where $y_{hf}$ is the output vector of the first hidden layer, $y_{hs}$ is the output vector of the second hidden layer, $W_k$ ( $k = 1, 2$) is the synaptic weights matrix of the layer k, $b_k$ is the bias vector of layer k, x is the input vector, $\varphi(.)$ is the sigmoid function, $d_m = (y_{hs} - \mu_m)^T \Sigma^{-1}(y_{hs} - \mu_m)$ is the Mahalanobis distance between $y_{hs}$ and $\mu_m$, $\Sigma$ is the covariance matrix over all the output vectors $y_{hs}$, presented by the second hidden layer in response to the training data set, $\mu_m = \frac{1}{N_m} \sum_{i=1}^{N_m} y_{hs_m}(i)$ is the prototype of Class m, $N_m$ is the number of training patterns that belong to Class m, and $y_{hs_m}(i)$ is the second hidden layer output for an input that belongs to Class m. Analyzing (11) we can observe that $\widehat{y}$ varies continuously from -1, for $y_{hs} = \mu_2$, to 1, for $y_{hs} = \mu_1$. This continuous approach enables ROC curves calculation. The MCI creates a hidden space where the Euclidean distance between the prototypes of each class is increased, and the patterns dispersion of each class is decreased. The goal is to maximize the objective function
$$ J=\left(\mu_{1}-\mu_{2}\right)^{T}\left(\mu_{1}-\mu_{2}\right)-\delta_{1}^{2}-\delta_{2}^{2} $$
where, $\delta_{m}^{2} = \sum_{i=1}^{N_{m}}(yh_{m}(i) - \mu_{m})^{T}(yh_{m}(i) - \mu_{m})$ is the deviation of Class m patterns in the hidden space. The weights and biases are updated based on the gradient ascendent algorithm.
### D. FLDA
Let us consider w a vector of adjustable gains and $\{x_c\}$ the set of feature vectors that belong to Class c, $(c=1,2)$ with mean $\mu_c$, and covariance $\sum_c v_c$. The linear combination $w \cdot x_c$ has mean $w \cdot \mu_c$ and covariance $w^T \sum_c w$. The ratio, $J(w)$, of the variance between the classes, $\sigma_b^2$, by the variance within the classes, $\sigma_w^2$, is a suitable measure of separation between these two classes:
$$ J(w)=\frac{\sigma_{\mathbf{b}}^{2}}{\sigma_{\mathbf{w}}^{2}}=\frac{(w\cdot(\mu_{2}-\mu_{1}))^{2}}{w^{T}(\Sigma_{1}+\Sigma_{2})w} $$
To obtain the maximum separation between classes one has to find the vector w which solves the optimization problem
$$ \max_{w}J(w) $$
whose solution is
$$ w=(\Sigma_{1}+\Sigma_{2})^{-1}(\mu_{2}-\mu_{1}) $$
To find the plane that best separates the data, the expression $w^{T}\mu_{1}+b=-(w^{T}\mu_{2}+b)$ has to be solved for the bias b.
### E. SVM
the margin parameter $C^3$, therefore, it is not appropriate to adjust this parameter based on the SVM performance on the test data set, since in this case information from the test data set will be carried to the SVM. The usual approach is to apply K-fold cross validation over the training data set.
Support Vector Machines (SVM) are based on the statistical theory of learning, developed by Vapnik [^15]. This theory provides a set of principles to be followed in order to obtain classifiers with good generalization, defined as its ability to predict correctly the class of new data in the same area where the learning occurred. SVM is very sensitive to
### V. RESULTS
The LIDAR-based pedestrian detection system was evaluated in terms of AUC, accuracy, and ROC curves. The data sets, summarized in Table II, were acquired in the ISR-UC Campus (), under the following configuration:
1) Ibeo-based data set: the LIDAR was mounted approximately 52 cm above the ground, with Field of View (FOV) restricted to 180°, horizontal angular resolution of 0.5°, vertical resolution of $[-1.2^{\circ}, -0.4^{\circ}, 0.4^{\circ}, 1.2^{\circ}]$, and measurement range up to 35m;
2) Sick-based data set: the LIDAR was mounted approximately 64.5 cm above the ground, $180^{\circ}$ of the FOV, horizontal angular resolution of $0.5^{\circ}$, and part of the measurements ranging up to 8m and part up to 25m.
The positives correspond to pedestrians in static or moving behavior, and the negatives consist of trees, vehicles, posts, walls, hydrants, poles, etc. The vehicle where the LIDARs were mounted, shown in Fig. 3, was driven with a maximum speed of 40Km/h approximately. The data sets and the corresponding ground truth, generated under user supervision, are available on the Web $^{4}$ for further comparisons and evaluations.
TABLE II
DATA SETS USED TO TRAIN AND TO EVALUATE THE CLASSIFIERS.
| Ibeo-based data set |
| Designation | Total | N pos. | N neg. | Description |
| Data1_{ISR-UC} | 2614 | 1637 | 977 | Training data set |
| Data2_{ISR-UC} | 1000 | 500 | 500 | Testing data set |
| Sick-based data set |
| Designation | Total | N pos. | N neg. | Description |
| Data3_{ISR-UC} | 2244 | 1184 | 1060 | Training data set |
| Data4_{ISR-UC} | 1000 | 500 | 500 | Testing data set |
### A. Summary of the classifiers
The FLDA and the Naive-Bayes are base classifiers, not requiring parameters adjustments. Eventually, for the case of Naive-Bayes classifiers, the likelihood tends to zero depending on the feature distributions. Based on experiments during the training phase, the number of Gaussian components in the GMMC classifier was set to 4, and the margin parameter used in the RBF-SVM was 1000. Moreover, the MCI-NN was configured with 30 and 200 neurons in the first and second layers respectively.
---
Fig. 3. Electric vehicle and the sensors setup used in the data set acquisition.
### B. Feature analysis
As a demonstration of the impact on the classification performance when the number of features (nfea) varies, consider the bar-graphs, Fig. 4 and Fig. 5, where the AUC and the accuracy (Acc) were used as classification performance metrics. The total number of features, restricted to 10, have been selected using the mRMR method. The best results for each classification method in terms of AUC, Acc and nfea are presented in Table III.
Fig. 4. Number of features vs classifier performance using the data set obtained with the Ibeo AlascaXT laser.
TABLE III
CLASSIFICATION PERFORMANCE: TESTING DATA SET
| Ibeo data set | Sick data set |
| AUC | Acc | nfea | AUC | Acc | nfea |
| FLDA | 0.884 | 0.845 | 9 | 0.890 | 0.80 | 7 |
| Naive-Bayes | 0.903 | 0.843 | 9 | 0.932 | 0.855 | 1 |
| RBF-SVM | 0.928 | 0.796 | 8 | 0.965 | 0.904 | 9 |
| GMMC | 0.950 | 0.894 | 8 | 0.989 | 0.94 | 3 |
| MCI-NN | 0.963 | 0.931 | 8 | 0.968 | 0.943 | 9 |
### C. Classification performance
To allow an appropriate analysis of the classification performance, ROC curves for the Ibeo and Sick data sets are displayed in Fig. 6 and Fig. 7 respectively. The FLDA, Naive-Bayes and RBF-SVM classifiers had, in both data sets, an
Fig. 5. Number of features vs classifier performance using the data set obtained with the Sick LMS200.
analogous performance behavior, excepting the Naive-Bayes which, for the Sick data set, had achieved the best result with just one feature. On the other hand, the GMMC and the MCI-NN did not display much distinction on the performance nature for each data set; actually both classification methods achieved promising results.
Fig. 6. ROC using Ibeo data set.
### VI. CONCLUSIONS
A perception system is indispensable to assess the potential risk for the ego-vehicle and, more important than that, the risk for the vulnerable road users, specifically for the case of pedestrians. In this subject, many researches have addressed coherent solutions for pedestrian classification using, in most of the cases, a LIDAR as a prior object detector, hence the pedestrian classification decision was carried out by vision-based systems.
In this present work we have expanded on this subject and have presented as main contribution a relevant study on
---
Fig. 7. ROC using Sick data set.
pedestrian detection in urban areas using only laser-based features, aiming to explore as much as possible LIDAR sensors, despite of merely using such sensors as prior object detector (object-hypothesis generator). Experiments with five classification techniques using data from an automotive and an industrial LIDAR were presented to demonstrate the effectiveness of such solution, and further comparisons can be made with the data sets that are online available for public usage.
### VII. ACKNOWLEDGMENTS
An appropriate and feasible solution for object detection and classification in urban scenarios where many unproprious and unpredictable circumstances can happen is still an open issue to be solved. In this context, here we have explored the potential of LIDAR in pedestrian classification, as a step forward to realistic applications where a perception system will face real circumstances of cluttered zones, oscillations, vibrations, weather influences, illumination changing, tracking and data association ambiguities, processing time constraints, normative requirements, etc.
This work is supported in part by Fundação para a Ciência e a Tecnologia de Portugal (FCT), under Grant PTDC/EEA-ACR/72226/2006. C.Premebida is supported by FCT under grant SFRH/BD/30288/2006 and O.Ludwig under grant SFRH/BD/44163/2008.
## REFERENCES
[^1]: C. Premebida and U. Nunes. A multi-target tracking and gmm-classifier for intelligent vehicles. In Intelligent Transportation Systems, ITSC. IEEE International Conference on, pages 313–318, Sept. 2006.
[^2]: Tarak Gandhi and Mohan Manubhai Trivedi. Pedestrian protection systems: issues, survey, and challenges. Intelligent Transportation Systems, IEEE Transactions on, 8(3):413–430, 2007.
[^3]: K.O. Arras, O.M. Mozos, and W. Burgard. Using boosted features for the detection of people in 2d range data. In Robotics and Automation, IROS. IEEE International Conference on, pages 3402–3407, April 2007.
[^4]: J. Xavier, M. Pacheco, D. Castro, A. Ruano, and U. Nunes. Fast line, arc/circle and leg detection from laser scan data in a player driver. In Robotics and Automation, ICRA. IEEE International Conference on, pages 3930–3935, April 2005.
[^5]: K. Dietmayer, J. Sparbert, and D. Streller. Model based object classification and object tracking in traffic scenes from range images. In Intelligent Vehicles Symposium, IVS. IEEE, Tokyo, Japan 2001.
[^6]: D. Streller and K. Dietmayer. Object tracking and classification using a multiple hypothesis approach. In Intelligent Vehicles Symposium, IVS. IEEE, pages 808–812, June 2004.
[^7]: S. Gidel, P. Checchin, C. Blanc, T. Chateau, and L. Trassoudaine. Pedestrian detection method using a multilayer laserscanner: Application in urban environment. Intelligent Robots and Systems, IROS. IEEE/RSJ International Conference on, pages 173–178, Sept. 2008.
[^8]: L. Spinello and R. Siegwart. Human detection using multimodal and multidimensional features. In Robotics and Automation, ICRA. IEEE International Conference on, pages 3264–3269, May 2008.
[^9]: Cristiano Premebida, Oswaldo Ludwig, and Urbano Nunes. Lidar and vision-based pedestrian detection system. Journal of Field Robotics, 26(9), 2009.
[^10]: B. Douillard, D. Fox, and F. Ramos. A spatio-temporal probabilistic model for multi-sensor object recognition. In Intelligent Robots and Systems, IROS. IEEE/RSJ International Conference on, pages 2402–2408, 29 2007-Nov. 2 2007.
[^11]: C. Premebida and U. Nunes. Segmentation and geometric primitives extraction from 2d laser range data for mobile robot applications. In Proc. 5th National Festival of Robotics, Scientific Meeting (ROBOTICA), Coimbra, Portugal, 2005.
[^12]: Jose E. Guivant, Favio R. Masson, and Eduardo M. Nebot. Simultaneous localization and map building using natural features and absolute information. Robotics and Autonomous Systems, 40(2-3):79 – 90, 2002.
[^13]: Hanchuan Peng, Fuhui Long, and C. Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(8):1226–1238, Aug. 2005.
[^14]: O. Ludwig and U. Nunes. Improving the generalization properties of neural networks: an application to vehicle detection. In Intelligent Transportation Systems, ITSC. IEEE International Conference on, pages 310–315, Oct. 2008.
[^15]: Vladimir N. Vapnik. Statistical Learning Theory. John Wiley, 1998.
[^16]: Shigeo Abe. Support Vector Machines for Pattern Classification (Advances in Pattern Recognition). Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2005.