# Stationary Human Detection Method Using 2D LiDAR
Haruki Mochizuki
Graduate School of Kanagawa
Institute of Technology
Atsugi, Japan
Ryozo Kiyohara
Kanagawa Institute of Technology
Atsugi, Japan
Abstract—Since the COVID-19 pandemic, Japanese industry has been facing a labor shortage in various fields. In particular, the security sector is experiencing a serious labor shortage problem. University security operations are also being affected by the lack of security guards. On the other hand, robot technology is developing and it is becoming possible to perform not only simple tasks but also tasks that involve interaction with humans. In particular, robots are expected to be introduced in patrol security, which requires a large number of personnel, as much of the work involves confirming that there are no problems. A security guard should go there only if it cannot be confirmed that there is no problem. Therefore, the accuracy of confirmation and the cost of introducing and operation become issues. The technologies for recognizing human using cameras mounted on 3D-LiDAR (Light Detection And Raging) on robots has been established. However, how to use it in a dark place and recognition technology by the low-cost 2D-LiDAR have not been established. In this paper, we propose a human recognition method using 2D-LiDAR that extends the technologies of previous research, and present the results of our evaluation.
Keywords—Human recognition, LiDAR, Robot, Security Guard
### I. INTRODUCTION
Since the COVID-19 pandemic, Japanese industry has been facing a labor shortage in various fields [^1]. The security sector, in particular, has a severe labor shortage due to the harsh working environment, which involves working early in the morning, at night, and outdoors. Due to the effects of COVID-19, facilities have been closed, price competition among security companies has also had an impact, and salaries are low and career development is difficult. University security operations are also being affected by the lack of security guards. Therefore, it is necessary to reduce the patrol duties of security guards as much as possible.
On the other hand, robot technology is developing and it is becoming possible to perform not only simple tasks but also tasks that involve interaction with humans. In particular, robots are expected to be introduced in patrol security, which requires a large number of personnel, as much of the work involves confirming that there are no problems. In Japan, for example, robots that serve food at restaurants move around in the flow of people [^2].
In terms of security, if it cannot be confirmed by robots that there is no problem, a security guard is required to go and check. In order to reduce such cases, improving the accuracy of confirmation is an absolute requirement. In addition, it is necessary to reduce the cost of introducing and operating the robot as much as possible. Technology for recognizing people using cameras mounted on robots and 3D-LiDAR (Light Detection And Raging) has already been established and is in use $^{[3][4]}$. However, no technology has been established for use in poorly lit areas or for low-cost 2D-LiDAR recognition.
Several methods were proposed for recognizing people using 2D-LiDAR[^5][^6]. We already proposed several methods[^7][^8]. However, there were some conditions and problems with accuracy. In this paper, we propose a person recognition method using 2D-LiDAR, which is an extension of the technology of our previous research.
### II. LiDAR
Several devices are currently available for human detection. Cameras are the most commonly used. Many studies have been conducted on artificial intelligence (AI) technologies for image processing in human detection. However, a camera requires a light source. Therefore, camera devices are not suitable for nighttime. Autonomous vehicles use high-sensitivity cameras that require only a small amount of light. However, this method is expensive to use with robots as a security guard. We experimented with human detection at night using a low-cost web camera, and even with some light available, it was difficult to detect humans located a short distance away. A thermal sensor camera can be used at night. However, it is difficult to detect humans near heat obstacles. Therefore, we adopt LiDAR technologies. The 3D-LiDAR must be installed on robots for simultaneous localization and mapping (SLAM), which creates maps for autonomous movement. Therefore, if they could also be used for human detection, sensor costs would not be required.
The sensing data of LiDAR are point-cloud data that show obstacles relative to the LiDAR. The amount of point-cloud data from 3D-LiDAR is too large. Therefore, the calculation costs are too large $^{[9]}$. We then propose a human detection method using part of the point-cloud data from 3D-LiDAR as well as multiple 2D LiDAR.
LiDAR emits light around itself and detects the light reflected from objects. Then, the LiDAR calculates the distance using the phase difference of the light. In other words, LiDAR can get 3D data that the image data from the camera does not have. Fig. 1 shows the point-cloud data from 3D-LiDAR and
---
Fig. 2 shows the point-cloud data from 2D-LiDAR. The amount of point-cloud data from the 3D-LiDAR are too large. If the two legs and two arms of a human feature can be detected using a small amount of point-cloud data, the calculation cost can be reduced.
### III. ROBOTS AS A SECURITY GUARD
### A. Patrol in KAIT Campus
In this section, we describe the aims of the patrol used on our campus. Fig.3 shows the KAIT campus. There are many buildings on campus. Buildings no. 2 and no. 5 are only used for lectures. Other buildings have lecture rooms, personal rooms for professors, and rooms for office workers.
During the day, the main purpose of a patrol is finding suspicious objects included illegal parking. Because there are many students, teachers, office workers, and neighborhood residents on campus during the daytime, they are not required to go through an admission and confirmation process to enter the campus. However, all rooms except lecture halls are locked, and only students and teachers with approval can enter them.
During the nighttime, only teachers and students with permission can stay on campus. Therefore, a patrol service is required. The purposes of such a patrol were mentioned in
Fig1 Example of point-cloud data (3D-LiDAR)
Fig2 Example of point-cloud data (2D-LiDAR)
Fig. 3 Map of KAIT campus
section 1. Currently, security patrols are provided outside the campus buildings several times at night. Moreover, they patrol inside the buildings for other purposes. Therefore, they cannot conduct a patrol if a problem occurs.
Thus, we studied a way to introduce patrol robots on our campus. The robot works as a delivery and guide robot during the daytime. At night, the robot works as a patrol robot. The main purpose of introducing such robots are to reduce costs. Another purpose is a safe patrol. When finding a person who has fallen, the patrol robot might have to put on a gas mask. If the robots have a gas sensor, they can send such information to the control center. Fig. 4 shows the goal of our study.
The control center controls the parking, charging, and communicating space through WiFi. There are many buildings and many objects between buildings (e.g., benches and trash bins). We are planning to put 5G equipment between each building. In these areas, the possibility of finding a human is greater than in other areas. Therefore, robots should be able to be controlled by the center using 5G.
Fig. 4 Image of our campus
---
It is difficult to find suspicious people. However, robots may encounter non-malicious people. Because, they do not think he is not illegal. Then, we hope that the robot can deter and find non-malicious people. Examples of non-malicious people include aged wanderers and children.
### B. Current Patrol
Currently, many patrol systems consist of human patrollers and monitoring cameras. These cameras show the surrounding area and information for security guards. Moreover, the camera is used for deterrence. However, security guards must decide what action they should take for every case when applying this system. This means that the guards are constrained by time. Moreover, it is not easy to analyze the still images from monitoring cameras at night or under poor weather conditions.
### C. Robots
We are planning to introduce robots for many different purposes. The robots are based on small automated vehicles, and have many sensors:
GPS
2D and 3D LiDAR for object detection
Infrared camera
Moreover, robots can communicate through a network such as LTE, WiFi, Bluetooth, and 5G.
We assume the robots are a type of autonomous vehicle. Therefore, they have a campus map. Moreover, they have 3D LiDAR (VLP-16) [^4] and GPS. In addition, they can update the map dynamically using their own 3D LiDAR and can compare the previous information with the current information.
### IV. RERALATED AND OUR PREVIOUS STUDIES
Moreover, we assume there are no steps outside of the buildings on our campus. The robot speed is almost the velocity of a walking human.
Numerous studies have been conducted regarding human detection. Human detection using image processing is one of the most commonly used methods. Recent camera and AI technologies can detect humans with high accuracy [^10][^11]. These studies have focused on face or action recognition. Studies on pedestrian detection by monocular cameras are surveyed [^3]. However, in the case of many campuses, image-processing technologies with cameras may be difficult to apply for human detection. The campus had a mixture of bright and dark fields. Processing images using low-cost cameras is challenging.
There are some studies on human detection with 3D-LiDAR. Pedestrian recognition using 3D-LiDAR for autonomous vehicles [^4] uses a support vector machine (SVM) for human detection. Other approaches use 3D LiDARs as multiple 2D-LiDARs [^5]. This study gathered point-cloud data from many different heights and achieved a classification rate of up to 96%. An example of using reduced point-cloud data is dividing 3D point clouds into the target into parts, that is, the trunk and legs, to achieve robust in situ pedestrian recognition [^6].
All these studies used 3D point-cloud data, which requires a powerful CPU. Because 3D LiDAR can acquire a point-cloud of an entire human image, it is possible to discriminate a human regardless of its shape, such as clothes. However, it is not necessary to have multiple human shapes that can be acquired from locations at similar heights. Therefore, in this study, we propose a human detection method that uses multiple 2D-LiDARs to reduce the calculation costs.
In a previous study [^7], we proposed a human detection method based on low-resolution 2D-LiDAR and a PointNet-based autoencoder [^12]. 2D-LiDAR was used to gather data from a height of 90 cm, which is considered the typical waist height of humans, both frontally and laterally. The proposed method successfully detected humans within 5 m of LiDAR.
In a previous study [^8], we proposed the human recognition method by multiple 2D-LiDARs. However, both methods require to recognize from exact front. In this paper, we propose a method for recognizing people regardless of the sensing angle.
### V. PROPOSED METHOD
In many cases, it's unclear from which angle the robot obtained the pointcloud data relative to the target object. This study focuses on elderly individuals with dementia entering facilities without malicious intent. Since dementia rates increase with age, most affected individuals are elderly, suggesting that 2D-LiDAR should be installed at a height matching that of the elderly.
Table 1 shows the height of Japanese elderly people by age. By focusing on individuals between 135 and 175 cm tall, there's a higher chance of detecting elderly individuals with dementia entering without malicious intent.
The human body comprises the head, torso, arms, and legs. Given the head's smaller size compared to the rest of the body, fixed LiDAR heights may not capture its geometric features. Therefore, this study concentrates on the shape characteristics of the torso, arms, and legs. To focus on individuals between 135 and 175 cm tall, LiDAR heights are set at 95 cm and 40 cm to ensure feature acquisition.
LiDAR mounted on a robot does not necessarily acquire pointcloud information from the front of the target. There is a possibility that point cloud information is obtained in various orientations from $0^{\circ}$ to $360^{\circ}$ of humans relative to the robot. However, the acquired pointcloud data of human have the characteristic that the shape remains the same even in different orientations.
Table 1 Average height of Japanese elderly people
| Age | Height (cm) |
| Male | Female |
| 60-69 | 167.4 | 154.0 |
| 70-79 | 163.1 | 149.4 |
| 80+ | 161.1 | 146.6 |
---
Fig. 5 and 6 illustrate human waist pointcloud data at $0^{\circ}$ and $180^{\circ}$ relative to the robot's LiDAR, respectively. The same shape is observed at opposite directions, such as $0^{\circ}$ and $180^{\circ}$, $45^{\circ}$ and $225^{\circ}$, $90^{\circ}$ and $270^{\circ}$, etc.
Therefore, if pointcloud data of human can be determined using only pointcloud data in half of the orientation from $0^{\circ}$ to $180^{\circ}$ with respect to LiDAR, it can also be determined in the other half of the orientation. Therefore, we obtain point cloud information from $0^{\circ}$ to $180^{\circ}$.
Fig. 7 shows the flow of patrol security in which a robot equipped with LiDAR at 95 cm and 40 cm detects a suspicious person and notifies the security guard. First, a robot equipped with LiDAR and a computer patrols and collects pointcloud data of the surrounding area. Background subtraction processing is performed on the computer equipped with the acquired pointcloud data.
Next, the extracted pointcloud data of obstacles such as people and objects are sent to the server through the wireless network, and the server side uses PointNet to identify human or obstacle. If the target is recognized as a human, a security guard
Fig 5 Pointcloud data in front
Fig 6 Pointcloud data in behind
Fig. 7 Flow of proposed method
is notified, the security guard heads to the robot's location, and the robot tracks the human.
On the server, in order to perform person recognition using the acquired pointcloud data, they are classified using machine learning technologies. To build a system for classification, use "Classify pointclouds using PointNet," which is published in the MATLAB library. This is a method that uses PointNet to compress and restore pointcloud data and extract only the features for classification.
Classification using PointNet is supervised learning, so it is necessary to learn pointcloud data to be classified in advance. Therefore, it is learned pointcloud data about humans and trees, and classify the acquired pointcloud data into humans and trees. Since the number and shape of pointcloud data differ depending on the size of the target object, learning is performed by acquiring point cloud information of various types of humans and trees. Furthermore, the number and shape of pointcloud data that can be acquired differs depending on the orientation of the person. When humans learn one direction, it is difficult to recognize directions that have not been learned.
In the case of front facing, pointcloud data of two human legs are obtained. Pointcloud data for the two legs are obtained in the same way even in orientations other than landscape. Therefore, by learning the front facing direction, it is possible to recognize human with a certain degree of accuracy in situations other than those facing sideways. Moreover, since the learned orientation can be determined in the case of landscape orientation, it can be determined by learning the landscape orientation.
Therefore, we think that by learning two directions, frontal and sideways, the range of human orientations that can be determined will be increased, making it possible to identify human even in a variety of orientations.
### VI. EXPERIMENT
Two 2D-LiDARs have been installed at heights of 95cm and 40cm to obtain pointcloud data. Pointcloud data have been acquired using RPLiDAR A1M8 [^13]. Table 2 shows the specification of this LiDAR. Considering the maximum measurement distance of LiDAR which is 6m, the measurement range to be acquired has been set to 5m, and the point cloud has been acquired by measuring from 1m to 5m at 1m intervals.
First, the necessary pointcloud data have been gotten from the acquired pointcloud data, and since 2D-LiDAR can only acquire x and y coordinates, when inputting it to PointNet, input the actually acquired heights of 95 cm and 40 cm as z coordinates. Then, 3D pointcloud data has been created.
Table 2 Specification of RPLiDAR-A1M8
| Name | RPLiDAR A1M8 |
| Range | 6m radius |
| Viewing Angle | 360° |
| Revolution Per Second | 6r/s～8r/s |
---
Next, Using the created pointcloud data, we have used “Classify point clouds using PointNet,” which is published by MATLAB, to compress and restore the pointcloud and extract features. The extracted features have been classified into classes and human has been recognized.
### A. Recognition Accuracy Based on Arm Position
We have verified the difference in recognition accuracy depending on the position of the human arm in the front. Therefore, we have acquired four types of pointcloud data at different positions of the human arm. The four types of arm positions obtained has been: arms down to the side, arms up, arms crossed in front, and arms crossed behind the back. The pointcloud data have been approximately 2m away from the 2D-LiDAR, and 150 pieces of pointcloud data were prepared for each rotation of the 2D-LiDAR. Moreover, similar number of pointcloud data of tree have been obtained as incorrect data. Table 3 shows the results of this experiment.
The recognition result shows that the accuracy of human recognition was high even when the position of the arm is different. From this result, there was no effect of the arm position for human recognition of our method.
### B. Recognition Accuracy for Each Angle
We have confirmed differences in human recognition accuracy based on differences in characteristics of pointcloud data from each angle. In order to determine the recognition rate of classification for each angle, pointcloud data have been acquired at a distance of approximately 2 m from the LiDAR. When classifying, we prepared 150 pointcloud data for each angle as learning data, one rotation of the 2D-LiDAR for frontal and sideways human data. Moreover we have prepared 150 pointcloud data of trees at 2m. Total of 300 pieces of pointcloud data of front-facing and side-facing humans and trees have been trained in advance, with the number of learning times being 100.
For the test data, we have been prepared 150 pieces of human data for each $10^{\circ}$ from $0^{\circ}$ to $180^{\circ}$ as correct data, which is different from the training data. Moreover, we have prepared tree data for the incorrect answers, which has been different from the training data. Table 4 shows an example of the F value for human recognition for each angle.
From the recognition results, we have been able to obtain high accuracy in each orientation by learning frontal and sideways orientations. Moreover, the F value decreased slightly in the cases of $90^{\circ}$, $120^{\circ}$, and $150^{\circ}$ compared to other orientations. However, since each F value is quite close to 1, it can be gotten enough accuracy rate.
Table 3 F value for arm positions
| Test | Recall | Precision | F |
| Arms down at the side | 1.00 | 1.00 | 1.00 |
| Arms up | 1.00 | 1.00 | 1.00 |
| Arms crossed in front | 1.00 | 1.00 | 1.00 |
| Arms crossed back | 1.00 | 1.00 | 1.00 |
Table 4 F value for each angle
| Angle | Recall | Precision | F |
| 0 | 1.00 | 1.00 | 1.00 |
| 30 | 1.00 | 1.00 | 1.00 |
| 45 | 1.00 | 1.00 | 1.00 |
| 60 | 1.00 | 1.00 | 1.00 |
| 90 | 0.98 | 1.00 | 0.99 |
| 120 | 0.93 | 1.00 | 0.97 |
| 150 | 0.91 | 1.00 | 0.95 |
| 180 | 1.00 | 1.00 | 1.00 |
Therefore, by learning pointcloud data of front-facing and side-facing humans, the system has been able to identify human even when the humans have been in various orientations.
From these experimental results, in order to identify human in various orientations, it is possible to recognize by learning two orientations: frontal and sideways. Within the range of features learned through two-direction, various orientations can be determined.
### C. Recognition Accuracy by Distance
We have confirmed the difference in the accuracy of human recognition when the number of pointcloud data differs depending on the distance. Pointcloud data have been acquired from 2D-LiDAR in a distance range of approximately 1 m to 5 m. Moreover, one rotation of the 2D-LiDAR has been treated as pointcloud data for a human, and 150 pieces of pointcloud data have been acquired for each distance.
When classifying into classes, we have prepared pointcloud data from 2m distance in front and sideways orientations as learning data. Moreover, we have prepared 150 pieces of pointcloud data from 2m distance of tree, and have trained 100 times to learn a total of 300 pieces of pointcloud data of humans and trees.
For the test data, pointcloud data of human which have separated from the training data have been prepared as correct data, with 1 m interval each in the range of 1 m to 5 m for frontal, diagonal, and sideways orientations. Moreover, pointcloud data of a tree different from the training data have been similarly prepared for the incorrect data. When identifying human using the prepared pointcloud data of human and tree, we have used the same distance from each other. Tables 5, 6, and 7 show the F values for recognizing human facing forward, diagonally, and sideways for each distance.
From the recognition results, we have been able to obtain an F value that has been quite close to 1 even when the distance changed due to frontal and sideways learning. Moreover, we have been able to obtain F values close to the learned orientations of frontal and sideways, except for the learned diagonal orientation.
Since the accuracy of person recognition differs depending on the distance, it is possible to recognize people with high accuracy even at different distances. It is possible to recognize a person in unlearned orientations with the same accuracy as in a learned orientation within the LiDAR measurement range of
---
Table 5 F value at $0^{\circ}$
| Recall | Precision | F |
| 1m | 1.00 | 1.00 | 1.00 |
| 2m | 1.00 | 1.00 | 1.00 |
| 3m | 1.00 | 1.00 | 1.00 |
| 4m | 0.93 | 0.85 | 0.89 |
| 5m | 1.00 | 0.98 | 0.99 |
Table 7 F value at $45^{\circ}$
| Recall | Precision | F |
| 1m | 1.00 | 1.00 | 1.00 |
| 2m | 1.00 | 1.00 | 1.00 |
| 3m | 1.00 | 1.00 | 1.00 |
| 4m | 0.98 | 0.86 | 0.92 |
| 5m | 1.00 | 0.98 | 0.99 |
Table 8 F value at $90^{\circ}$
| Recall | Precision | F |
| 1m | 1.00 | 1.00 | 1.00 |
| 2m | 0.98 | 1.00 | 0.99 |
| 3m | 0.98 | 1.00 | 0.99 |
| 4m | 0.81 | 0.83 | 0.82 |
| 5m | 1.00 | 0.98 | 0.99 |
1m to 5m. Moreover, based on these experimental results, in order to recognize human in various orientations and distances, it is sufficient to learn two orientations: frontal and sideways. Learning in two directions increases the range of features that can be recognized, making it possible to recognize in different orientation or distance.
from various angles. As a result of experiments, we have found that it has been possible to identify stationary people from various angles: frontal and sideways. We have proposed a method that can identify a stationary human by learning a small number of angles, regardless of the angle of the human.
### VII. CoCLUSION
In order to solve the problem of labor shortage in the security industry, we have verified whether it is possible to identify human from various angles using a security robot equipped with 2D-LiDAR. Using 2D-LiDAR, we focused on the characteristic shapes of the human waste and shins and learned the frontal and sideways orientations to identify human recognition when we perform not only on trees on the Kanagawa University campus, but also on objects similar to the shapes of human waste and shins. Furthermore, there is a possibility of misrecognition if the human to be recognized is not standing upright, it is necessary to verify the accuracy of human recognition for various standing positions.
## REFERENCES
[^1]: S. Kotera, J. M. Schmittmann, “The Japanese Labor Market During the COVID-19 Pandemic,” International Monetary Fund, 2022.
[^2]: T. Shimmura, R. Ichikari, T. Okuma, H. Ito, K. Okada, T. Nonaka, “Service robot introduction to a restaurant enhances both labor productivity and service quality,” Procedia CIRP, Vol.88, pp.589-594, 2020.
[^3]: M. Enzweiler and D. M. Gavrila, "Monocular Pedestrian Detection: Survey and Experiments," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2179-2195, Dec. 2009
[^4]: H. Wang, B. Wang, B. Liu, X. Meng, G. Yang, “Pedestrian recognition and tracking using 3D LiDAR for autonomous vehicle,” Robotics and Autonomous Systems, Vol.88, pp.71-78, 2017
[^5]: L. Spinello, K. Arras, R. Triebel, and R. Siegwart, “A Layered Approach to People Detection in 3D Range Data”, AAAI, vol. 24, no. 1, pp. 1625-1630, 2010.
[^6]: N. Kirchner, A. Alempijevic and A. Virgona, "Head-to-shoulder signature for person recognition," 2012 IEEE International Conference on Robotics and Automation, pp. 1226-1231, 2012
[^7]: Y. Nagai, Y. Sawano, Y. Terashima, T. Suzuki and R. Kiyohara, "A Method for Detecting Human by 2D-LiDAR," IEEE International Conference on Consumer Electronics (ICCE), pp. 1-2, 2022
[^8]: T. Watanabe, Y. Terashima and R. Kiyohara, "Standing Human Detection Method Using 2D-LiDARs," 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), pp. 1260-1265. 2023
[^9]: Q. Wang, Y. Tan, Z. Mei," Computational Methods of Acquisition and Processing of 3D Point Cloud Data for Construction Applications," Archives of Computational Methods in Engineering, Vol.27, pp.479-499, 2020
[^10]: R. Chellappa, C. L. Wilson and S. Sirohey, "Human and machine recognition of faces: a survey," in Proceedings of the IEEE, vol. 83, no. 5, pp. 705-741, 1995,
[^11]: R. Poppe, "A survey on vision-based human action recognition," Image and Vision Computing, Vol 28, Issue 6, pp. 976-990, 2010.
[^12]: C. R. Qi, H. Su, K. Mo, L. J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
[^13] SLAMTEC,