# Advancing Autonomy: LiDAR-Powered Human Detection and Tracking for Safe and Efficient Systems Pavan Kumar B N IIIT Sri City Andhra Pradesh, India Chethana B Aptiv, Bengaluru, India Bharatesh Chakravarthi Arizona State University Tempe, USA Raju Imandi IIIT Sri City Andhra Pradesh, India Gyanendra Prasad Joshi Kangwon National University South Korea Abstract—This research articulates the development and implementation of an advanced LiDAR-based framework for the detection and tracking of humans within autonomous systems, significantly enhancing situational awareness in dynamic environments. Leveraging high-resolution 3D point cloud data, the study introduces a novel algorithmic approach that integrates ground plane removal, Euclidean and DBSCAN clustering, and principal component analysis (PCA) to robustly identify and track human figures across varying conditions and occlusions. This framework is validated through a series of field tests with a Velodyne HDL-32 LiDAR system, demonstrating improved accuracy and computational efficiency in real-time human tracking. The findings underscore LiDAR's pivotal role in augmenting the safety and reliability of autonomous vehicles, robotics, and surveillance systems by effectively managing the complexities of real-world operational settings. Our pilot study provides a scalable model for future enhancements in autonomous navigational technologies. Index Terms—LiDAR, Object Detection, Autonomous Driving, Human Detection ### I. INTRODUCTION LiDAR (Light Detection and Ranging) technology has revolutionized the perception systems of autonomous vehicles and robots, providing high-resolution, three-dimensional environmental data essential for navigation and object detection. This technology functions by emitting laser beams and measuring the time taken for these beams to bounce back after hitting various surfaces. This method allows for the creation of detailed 3D maps of the surroundings, which are crucial for the safe and efficient operation of autonomous systems such as vehicles, drones, and robots. These capabilities enable such systems to accurately detect and track objects, particularly humans, which is critical for ensuring safety and operational efficiency [^1] [^2] [^3]. With recent advancements, LiDAR technology has seen significant improvements in both reliability and precision, broadening its applications beyond traditional roles like autonomous navigation to encompass fields such as forest management, urban planning, and remote environmental monitoring. Despite these advancements, the dynamic nature of human movement and the complex environments in which autonomous systems operate present persistent challenges. These include handling occlusions—where objects are partially obscured—and managing the system's response to rapidly changing environmental conditions. These are pivotal concerns that impact the technology's reliability in detecting and tracking human subjects. Addressing these challenges, this paper introduces a sophisticated, cutting-edge framework that capitalizes on enhanced LiDAR data processing algorithms to significantly improve human detection and tracking across various operational contexts. Our approach diverges from existing models by integrating advanced computational techniques that effectively manage occlusions and adapt to environmental variability. This focus not only bridges a significant gap in current LiDAR applications but also enhances the system's performance in complex, real-world scenarios. Figure 1 illustrates ![Image](images/d3718c82a6c77740bed839f50601d60e5d0e512151d105e1b5f4455e77b28c46.jpg)
Fig. 1. Overview of the proposed pilot human detection and tracking platform
the comprehensive step-by-step methodology employed in our LiDAR-based human detection and tracking system. This figure delineates each phase of the process, starting from the initial data acquisition where 3D point cloud data is captured by the LiDAR sensors. Following this, the data undergoes several processing steps: firstly, ground plane removal to isolate human figures from the ground; secondly, data segmentation into clusters using both Euclidean and DBSCAN clustering methods to identify potential human targets; and finally, applying PCA for precise tracking and movement analysis. This structured approach ensures that each element of the surrounding environment is analyzed systematically. --- enabling accurate and reliable tracking of human movements within complex and dynamic settings. The principal contribution of this study lies in the development of an algorithmic framework designed to optimize the accuracy and efficiency of human detection and tracking using LiDAR. We conduct rigorous field testing across diverse settings to validate our methods, and our findings reveal considerable enhancements in situational awareness for autonomous systems. These improvements significantly bolster the safety and operational reliability of autonomous vehicles, robotics, and surveillance systems, marking a significant step forward in the deployment and scalability of LiDAR-based technologies. ### II. RELATED WORKS Automated person identification and property estimation by machines are not recent developments. Our research provides a surface-level exploration of these practices, tracing their origins back to the early advancements in camera technology. This evolution has been augmented by the integration of artificial intelligence into intelligent systems, bestowing them with remarkable capabilities. ### A. Video Camera-based Detection and Tracking Video surveillance systems are incredibly important today. They are used by the military, defense, public security, and private organizations to keep things safe. The main focus of modern research on these systems is tracking and finding objects, like people [^4]. When it comes to tracking people in videos, it's quite challenging because they can be seen from different angles and lighting conditions [^5] [^6] [^7] [^8]. It's also hard because people can be hidden behind other things or have different poses and sizes. Another problem with using videos is that they only show a limited area, so often many cameras are needed. Tracking people in real-time is another tricky issue. Video-based tracking and figuring out things about them, like estimating their properties, is hard. When you have a network of cameras, there's a lot of data coming in all the time, which can be really tough to handle. Plus, there are other problems like things blocking the view, people changing their poses, and not staying in the same posture, which can make the real-time system not work as well. ### B. Infrared Sensor-based Detection and Tracking The inception of infrared radiation dates back to the early 1700s when Sir FW Herschel, renowned for discovering the planet Uranus, unveiled its existence. This unique form of radiation serves various purposes such as communication and distance measurement. Presently, the innovation of a sensor enables the discernment of properties in close proximity. The utilization of infrared sensors for tracking individuals is a novel concept with emerging applications. An instance of this innovation is depicted in a study from 2005 that showcases a Bayesian network-based approach to person monitoring using infrared sensors [^9]. This implementation exhibited resilience in the face of environmental variations, facilitating remote tracking within the ambit of tight security surveillance. Certain scholars explored the integration of low-resolution thermal infrared (I.R.) sensors [^10] for person tracking across multimodal sensor arrays. By integrating a cost-effective I.R. sensor, they achieved computational efficiency, operational affordability, and a heightened level of precision. The fusion of this setup resulted in improved accuracy when contrasted with standalone RGB camera-based tracking mechanisms. Utilizing an infrared sensor setup facilitates person tracking, while concurrently enabling the extraction of human behaviors [^11] through the accumulation of extensive temporal data. Their proposed framework lays out a methodology to analyze individuals' habitual behaviors based on data from infrared sensors. This approach underscores the necessity for long-term data collection to effectively observe genuine human conduct. However, it's important to acknowledge that accurately tracking multiple targets within a budget-friendly network proves challenging when dealing with more than one individual of interest. ### C. Thermal Camera-based Detection and Tracking Thermal imagery is obtained by capturing radiations reflected, transmitted, and emitted from objects within a space. These thermal cameras are especially valued for their passive operation, as they engage with an object's infrared radiation. Yet, generating a thermal image from the surroundings presents challenges due to diverse radiation sources. Thermal imaging holds particular relevance in point-based imaging, enabling the detection of temperature disparities across an area. As a result, it finds utility in regions with elevated temperatures, and its application in military scenarios, including night vision, underscores its surveillance prowess. With diminishing costs, thermal cameras have garnered widespread adoption across diverse applications. Comprehensive review articles have explored its applications, delineating strategic placements and promising prospects on the horizon [^12]. Thermal cameras have also proven their mettle in-person tracking and precise detection [^13], [^14], [^15] solidifying their presence as well-established tools. However, the system encounters challenges in aspects like change detection and background subtraction [^16]. One ongoing challenge is the requirement for a training period, which isn't universally feasible for all use cases. ### D. LiDAR-based Detection and Tracking Laser Imaging Detection and Ranging (LiDAR) sensors have emerged as dependable and presently affordable range sensors, furnishing insights into dynamic environments at commendable rates (around 20-30 Hz) that are feasible for real-time processing. Each scan comprises a concise array of a few hundred integers. These LiDAR sensors are classified into two categories, with 3D LiDAR being particularly adept at capturing visual data. However, limited research pertains to 2D LiDAR sensors. --- The utility of 2D LiDAR sensors is primarily attributed to their cost-effectiveness and ease of processing. These sensors are economical and straightforward to manage. Additionally, privacy considerations come into play. While 3D LiDAR captures visual information that can compromise privacy, 2D LiDAR sensors solely record distance values within a 2D plane. This ensures that personal details remain concealed, and the risk of any breaches is minimal. It's worth noting that the initiatives discussed earlier predominantly revolve around cameras or 3D LiDAR sensors. ### III. IMPLEMENTATION AND EVALUATION OF THE PROPOSED PILOT PLATFORM Our human detection and tracking platform is detailed in Figure 1, which breaks down our methodology into structured logical blocks. Figure 2 further illustrates this process with a flowchart, demonstrating how each module processes the 3D point cloud data to perform human detection and tracking. The entire system ingests sequences of 3D point cloud data collected from ground-mounted 3D LiDAR and outputs information pertinent to human tracking. ![Image](images/f5add7a541904afd10d10267902e942cd0999491c9c3402911fcff188788426d.jpg)
Fig. 2. Flow diagram of the proposed platform showing the connections between each module.
Reading LiDAR Data In our approach, we employ the Velodyne HDL 32 LiDAR system for the accurate detection and tracking of humans, with data stored in PCAP format. Each scan generates a detailed 3D point cloud, essential for the effective processing and rapid indexing within our sensor processing pipeline. To efficiently manage this data, we utilized MATLAB's pointCloud object, which organizes the data using a K-d tree structure. The velodyneFileReader is instrumental in forming organized pointCloud representations for each LiDAR scan. The Location property of the pointCloud object records the X, Y, and Z coordinates in meters, while the Intensity property captures the intensity of the points. Dynamic visualization of streaming point cloud data is achieved using MATLAB's pcplayer object, which allows us to specify and display the region around the vehicle, enhancing the visualization of data captured by the LiDAR system. Ground Plane Detection and Removal The first step in our data processing pipeline is the detection and removal of the ground plane, a critical task for isolating and identifying potential obstacles. This is accomplished using MATLAB's segmentGroundFromLidarData function, which leverages a connected components algorithm to label and separate ground points from other elements in the data. This segmentation allows subsequent analyses to focus exclusively on points representing potential obstacles. Segmenting/Clustering Point Cloud Data After the removal of ground points, the remaining point clouds are then segmented into distinct clusters. To efficiently organize these clusters, we employ two advanced clustering techniques: Euclidean clustering [^16] and DBSCAN clustering [^17]. Euclidean clustering groups points based on their proximity, using a specified distance threshold to form clusters. Conversely, DBSCAN is adept at identifying clusters of varying shapes and sizes based on density, effectively segregating densely packed points from sparse ones. This dual approach allows us to adaptively structure the point cloud data into well-defined, coherent clusters that can be further analyzed. Each cluster undergoes a rigorous analysis using Principal Component Analysis (PCA). PCA is instrumental in examining the spatial consistency of clusters across sequential frames. Clusters that exhibit no significant change in position or composition are classified as static, signifying immovable objects within the environment. On the other hand, clusters that appear or change position in subsequent frames are marked as dynamic, indicating potential movement. This meticulous classification aids in the precise tracking and categorization of both stationary and moving objects, enhancing the system's ability to monitor dynamic environments accurately. Human Detection and Tracking Our system meticulously processes the continuous data stream from the LiDAR sensor, analyzing each frame for changes. When an object is detected in one frame at a specific location, designated as $(x_{n-1}, y_{n-1}, z_{n-1})$, it is anticipated to be present in the subsequent frame n at a new position $(x_n, y_n, z_n)$. This positional data is crucial as it forms the basis for estimating the object's trajectory. By systematically marking these positions frame by frame, our system constructs a detailed trajectory for each detected object. This trajectory mapping is vital for understanding the movement patterns of humans and other dynamic objects within the monitored area, providing valuable insights into their behavior and interactions with the surrounding environment. ### IV. EXPERIMENTAL RESULTS In this pilot study, we implemented and tested the proposed LiDAR-based human detection and tracking framework under real-world conditions. Utilizing the comprehensive 3D point cloud data provided by the LiDAR sensor, our approach accurately identifies and tracks human subjects across a variety of operational scenarios. A key goal of this study was to enhance the safety and reliability of autonomous vehicles, robotics, and surveillance systems by ensuring robust detection and tracking capabilities. To rigorously evaluate the framework's effectiveness, we conducted a field study with the LiDAR system mounted on the ground to capture environmental data. This setup allowed us to simulate realistic scenarios that autonomous --- and surveillance systems are likely to encounter, providing a practical testbed for our methods. Figures 3 through 6 showcase the results of these experiments, highlighting the capability of our method to detect and track objects within complex environments. The findings confirm that our framework can effectively distinguish humans from background elements, maintain accurate real-time tracking, and adapt to diverse conditions such as occlusions and environmental clutter. These results affirm the practical viability of our system for real-world deployments and pave the way for future enhancements and broader applications. It is important to note that, as this was a preliminary pilot project, the experiments were conducted under relatively ideal conditions. We plan to extend this research to include more challenging and volatile environments, which will help in refining the technology further and ensuring its efficacy in a wider range of conditions. ![Image](images/14b27b2cb97a28497ccf9f193488162f443b0465aefd2613f15cf1bbc1ab22ea.jpg)
Fig. 3. Visualization the LiDAR captured raw point cloud data.
![Image](images/9809657bd1c9b0ab4cb580b72e8dfd4536b97afc0e100738a9953bb72ef18ee8.jpg)
Fig. 4. Visualization of the point cloud data after removing the ground data.
![Image](images/a9212651ac4245ca18325dd9e69b7c095a8bc322757ea92caa205f65be384ea4.jpg)
Fig. 5. Visualization of the point cloud data after forming the clusters.
![Image](images/718e7050d36f241357ccf0a4ab7a20440edc89fbe3bfe6435d3922248d0c6ac2.jpg)
Fig. 6. Visualization of the detected moving objects.
### V. CONCLUSION This pilot study examined the effectiveness of LiDAR technology in detecting and tracking moving objects, particularly humans, within autonomous systems. The results demonstrated that our LiDAR-based approach successfully identified and tracked targets in diverse environments. While this work primarily focused on the detection and tracking aspects, future enhancements will integrate machine learning techniques to further differentiate object classes—especially humans—from other moving entities. These improvements have the potential to significantly increase the reliability and safety of self-driving vehicles, robotics, and surveillance applications. Consequently, this research lays the groundwork for more advanced LiDAR-based autonomous systems capable of operating robustly under a wide range of real-world conditions. --- ## REFERENCES [^1]: Pierzchała, M.; Giguère, P.; Astrup, R. Mapping forests using an unmanned ground vehicle with 3D LiDAR and graph-SLAM. Comput. Electron. Agric. 2018, 145, 217–225. [^2]: Zhu, Q.; Chen, L.; Li, Q.; Li, M.; Nüchter, A.; Wang, J. 3D LIDAR point cloud based intersection recognition for autonomous driving. In Proceedings of the 2012 IEEE Intelligent Vehicles Symposium, Madrid, Spain, 3–7 June 2012; pp. 456–461. [^3]: VanVoorst, B.R.; Hackett, M.; Strayhorn, C.; Norfleet, J.; Honold, E.; Walczak, N.; Schewe, J. Fusion of LIDAR and video cameras to augment medical training and assessment. In Proceedings of the 2015 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), San Diego, CA, USA, 14–16 September 2015; pp. 345–350. [^4]: Hou, Li, Wanggen Wan, Jenq-Neng Hwang, Rizwan Muhammad, Mingyang Yang, and Kang Han. "Human tracking over camera networks: a review." EURASIP Journal on Advances in Signal Processing 2017, no. 1 (2017): 1-20. [^5]: Bergmann, Philipp, Tim Meinhardt, and Laura Leal-Taixe. "Tracking without bells and whistles." In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 941-951. 2019. [^6]: Bochinski, Erik, Volker Eiselein, and Thomas Sikora. "High-speed tracking-by-detection without using image information." In 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS), pp. 1-6. IEEE, 2017. [^7]: Henschel, Roberto, Yunzhe Zou, and Bodo Rosenhahn. "Multiple people tracking using body and joint detections." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0. 2019. [^8]: Zhang, Yifu, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. "Bytetrack: Multi-object tracking by associating every detection box." In European Conference on Computer Vision, pp. 1-21. Cham: Springer Nature Switzerland, 2022. [^9]: T. Hosokawa, M. Kudo, 2005. Person Tracking with Infrared Sensors. In: Khosla R., Howlett R.J., Jain L.C. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2005. Lecture Notes in Computer Science, vol 3684. Springer, Berlin, Heidelberg. 10.1007/11554028_95. [^10]: Kumar, Suren, Tim K. Marks, and Michael Jones. "Improving person tracking using an inexpensive thermal infrared sensor." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 217-224. 2014. [^11]: Honda, Seiichi, Ken-ichi Fukui, Koichi Moriyama, Satoshi Kurihara, and Masayuki Numao. "Extracting human behaviors with infrared sensor network." In 2007 Fourth International Conference on Networked Sensing Systems, pp. 122-125. IEEE, 2007. [^12]: Gade, R. and Moeslund, T.B., 2014. Thermal cameras and applications: a survey. Machine vision and applications, 25, pp.245-262. [^13]: Berg, A., Ahlberg, J., Felsberg, M. (2015). A thermal infrared dataset for evaluation of short-term tracking methods [^14]: Berg, Amanda, Kristoffer Ofjäll, Jörgen Ahlberg, and Michael Felsberg. "Detecting rails and obstacles using a train-mounted thermal camera." In Image Analysis: 19th Scandinavian Conference, SCIA 2015, Copenhagen, Denmark, June 15-17, 2015. Proceedings 19, pp. 492-503. Springer International Publishing, 2015. [^15]: Berg, Amanda, Jorgen Ahlberg, and Michael Felsberg. "Channel coded distribution field tracking for thermal infrared imagery." In Proceedings of the IEEE Conference on computer vision and pattern recognition workshops, pp. 9-17. 2016. [^16]: T oyama, Kentaro, John Krumm, Barry Brumitt, and Brian Meyers. "Wallflower: Principles and practice of background maintenance." In Proceedings of the seventh IEEE international conference on computer vision, vol. 1, pp. 255-261. IEEE, 1999. [^17]: Wen, Long, Lei He, and Zhenhai Gao. "Research on 3D point cloud distortion algorithm and its application on Euclidean clustering." IEEE Access 7 (2019): 86041-86053. [^18]: Tran, Thanh N., Klaudia Drab, and Michal Daszykowski. "Revised DBSCAN algorithm to cluster data with dense adjacent clusters." Chemometrics and Intelligent Laboratory Systems 120 (2013): 92-96.