Fig. 1. Approaches employed for HAR.
multiple output (MIMO) radar that provides more accessible and straightforward point cloud data to capture human movement information. Point cloud data refer to a set of vectors with geometric positions in a spatial coordinates system. They are sampling points that scan objects and record them as point data, each of which contains 3-D coordinates, and the number of points is large and dense which means point cloud data.
The main contributions of this article are as follows.
1) We construct a large-scale point cloud benchmark dataset for HAR. It is a publicly available, high-resolution dataset that contains more classes and more subjects than the current HAR datasets.
2) In contrast to previous research, we explored state-of-the-art methods in the domains of the HAR field. We have discussed the technology from traditional handcrafted features to recently developed deep learning.
3) We evaluate the advanced detection algorithm to process this benchmark point cloud dataset under consistent experimental conditions. This provides the literature with extensive baseline results for future research on HAR tasks.
In this article, Section I has a brief introduction to HAR systems. Section II describes the human activities and data collection method of the raw-sensor device. Section III provides a case study and evaluates the results, which shows the benchmark challenges in activity recognition. Finally, Section IV discusses the HAR benchmark datasets and compares some related works to prove our contributions. Section V has given a conclusion and indicates directions for future research.
### II. MATERIALS AND METHODS
For daily activities of humans, the complexity of the actions may be different, and human movements are classified according to various situations, including the number of activities [^13], the types of activities [^14], data collection energy consumption [^15], and significance [^16].
Fig. 2 shows that human activities can be divided into three basic types: 1) static activities [^17]; 2) dynamic activities [^18]; and 3) activities with posture transitions [^4]. Static activities generally refer to activities, such as sitting and standing, which basically do not involve human movement. Compared with static activities, dynamic activities have a motion trend and it represents more intense environmental interactions, for example, walking, falling, and other short-lived behaviors.
---
Fig. 2. Different types of human activities.
which are the interaction between human body posture and behavior [^1]. It is usually some high-frequency behaviors of the human being in daily life. Activities with posture transition are combinatorial behaviors that are aggregated through multiple basic activities, and multiple users may participate during the behavior [^19].
Static activities (such as sitting, standing, etc.) are more accessible to recognize than regular activities (walking and picking up some things, and so on). However, highly similar posture transitions, such as sitting and standing, will cause great complexity in the case of separation in that the activity feature space has obvious overlap. In the static activities of sitting and standing, which means completed action without movements, and dynamic activities or posture transitions for sitting down and standing up which means action in progress, it is continuous movement. In addition, dynamic activities (standing up and sitting down) with a high degree of similarity in the activity feature space are also difficult to distinguish due to related activity patterns.
In most cases, the correspondence between the activities performed is not similar during the entire activity period, which makes the HAR task more difficult. For example, sitting and standing are very similar, so it is hard to tell them differently from frame data. However, they are remarkably different from walking and are easy to separate from activity data at this point. Therefore, transition activities can be further divided into four types: 1) static to static posture transition (stand to sit); 2) static to dynamic (stand to walk); 3) dynamic to static (walk to stand); and 4) dynamic to dynamic (walk to fall).
Several variations have been built that represent various difficulty levels of the human activities dataset based on the selected activity objects. This allows researchers to explore the robustness of existing classification methods on benchmark datasets in more extreme real-world scenarios.
### A. Proposed Benchmark Dataset Overview
Fig. 3 shows the experimental setup for data collection. Fig. 3(a) shows the human movement data that has been acquired by the NodeNs sensor on the fixed table that is 2 m from the volunteer. Fig. 3(b) shows how to set up four subjects in the data collection environment. The time duration of all human activities has been locked on for 5 s, which is a unified time axis setting for easy normalization of data length. The dataset was divided into four classes, that is, a total of 1011 data files, where each file is a sample that represents a particular number of subjects (from one subject to four subjects that are four classes) and activities. It means that each file is one individual sample input to algorithm train and test. The main data folder is subdivided into 15 folders corresponding to the 15 classes. The detail about the benchmark dataset are shown in Table I.
### III. CASE STUDY
For a clearer picture of the maturity of the point cloud-based HAR task to be designed benchmark dataset, this work uses representative methods to benchmark test. The goal is to identify the contributions and limitations of the current work on the HAR task. In this case study, the first is data preprocessing to the point cloud data through density-based spatial clustering of applications with noise (DBSCAN) [^20] for clustering computing and then training and testing the HAR task on the long short-term memory (LSTM) neural network for classification to demonstrate the functionality of the benchmark dataset. Meanwhile, to prove that training on datasets with accurate attributes can help improve classification performance. The evaluation of LSTM is based on the labeled dataset that is randomly divided into two subsets: 1) training set (80%) and 2) test set (20%), and make sure that the training set and test set contain objects from different scenes. The workflow is shown in Fig. 4.
### A. Point Cloud Data Preprocessing
Point cloud data analysis often needs to segment the point cloud data and extract the region of interest. The methods of point cloud segmentation include model fitting, graph-based computing, deep learning, and clustering which is popularly deployed. For 3-D point cloud data, its feature attributes are usually used for clustering, and feature extraction or transformation is performed for each point space or local space. In this way, various attributes can be obtained, such as normal vector, density, distance, elevation, and intensity, which segments point clouds with different attributes. For example, for density clustering, a cluster is defined as the most extensive set of density-connected points, dividing regions with high density into clusters and finding clusters of arbitrary shapes in the spatial database of noise.
At this point, DBSCAN is a density-based clustering algorithm, which generally assumes that the tightness of the sample distribution can determine the point cloud categories. For the same category samples, the points between them are closely connected. Therefore, there must be points of the same category around the sample points in this category. So, it is a density-based spatial clustering calculation. There are advantages of antinoise as well as no need to specify the number of categories and clustering of arbitrary shapes in spatial data, which are suitable for clustering point cloud data.
The core of the DBSCAN algorithm is to find the most extensive set of densely connected points. It describes the tightness of the sample by two parameters: 1) epsilon and 2) minPoints. Epsilon is the radius of the scanning circle for each data point that checks the point density. It is the radius of the neighborhood, which is used to express the neighborhood
---
| Number of Subjects | Activity | Number of Subjects and Corresponding Activity | Description | Number of samples |
| 1 | Sit down, Stand up, Walking, Fall, Pick up. | • P1 (All activities) | • One subject performing each Action single | 101 / Activity class * 5 = 505 |
| 2 | • Sit down + Stand up, • Sit down + Pick up, • Stand up + Pick up | • P2(Sit) + P1(Stand), • P1(Sit) + P2(Stand), • P2(Sit) + P1 (Pick), • P1(Sit) + P2 (Pick) | • One subject performing the action of "Sitting" and one subject performing the action of "Standing", at the same time. • One subject performing the action of "Sitting" and one subject performing the action of "Pick up a box", at the same time. • One subject performing the action of "Standing" and one subject performing the action of "Pick up a box", at the same time | 40 + 40 + 39 = 119 |
| 3 | • Sit down + Stand up + Pick up, • Stand up + Pick up + Sit down, • Pick up + Sit down + Stand up. | • P3(Sit) + P2(Stand) + P1(Pick), • P1(Sit) + P3(Stand) + P2(Pick), • P2(Sit) + P1(Stand) + P3(Pick), • P1(Sit) + P3(Stand) + P2(Pick), • P3(Sit) + P2(Stand) + P1(Pick), • P2(Sit) + P1(Stand) + P3(Pick), • P3(Sit) + P2(Stand) + P1(Pick) | • One subject performing the action of "Sitting", one subject performing the action of "Standing" and one subject performing the action of "Pick up a box", at the same time. • One subject performing the action of "Standing", at the same time. • One subject performing the action of "Sitting" and one subject performing the action of "Pick up a box", at the same time | 62 + 93 + 62 = 217 |
| 4 | • Sit down + Stand up + Pick up + Sit down, • Sit down + Sit down + Stand up + Pick up, • Pick up + Sit down + Sit down + Stand up, • Stand up + Pick up + Sit down + Sit down. | • P3(Sit) + P2(Stand) + P4(Pick) + P1(Sit), • P1(Sit) + P3(Stand) + P2(Pick) + P4(Sit), • P4(Sit) + P1(Stand) + P2(Sit) + P3(Sit), • P3(Sit) + P2(Stand) + P4(Sit) + P1(Pick), • P3(Pick) + P2(Sit) + P4(Sit) + P2(Stand) + P2(Pick) + P4(Sit) + P1(Stand) + P2(Stand) + P2(Pick) + P3(Sit) + P4(Sit) | • One subject performing the action of "Standing", one subject performing the action of "Standing", one subject performing the action of "Standing", one subject performing the action of "Standing", one subject performing the action of "Standing", one subject performing the action of "Standing" and one subject performing the action of "Standing" and one subject performing the action of "Pick up a box", at the same time. • One subject performing the action of "Standing", one subject performing the action of "Standing", one subject performing the action of "Standing", one subject performing the action of "Standing" and one subject performing the action of "Sitting" and one subject performing the action of "Standing" and one subject performing the action of "Standing" and one subject performing the action of "Sitting" and one subject performing the action of "Standing" and one subject performing the action of "Standing" and one subject performing the action of "Sitting" and | |
---
distance threshold of the current point. minPoints is the minimum number of data points required inside the scanning circle, which the data point will judge as a core point. It is used to represent the minimum number of data points in the neighborhood. The algorithm can select the most suitable parameters by iterating these two parameter variables in loop computing. The flow and parameter setting of the DBSCAN Algorithm 1 are shown in pseudo-code [^20].
```pseudo
// Algorithm 1: DBSCAN Algorithm
1: Input: D is a dataset containing n objects, $\varepsilon$ is a radius parameter, M is a Neighborhood density threshold.
2: Output: A set of density-based clusters
Require: :
3: Initial Dataframe to Zero, and then load data D as Dataframe in the algorithm
4: Setting $\varepsilon$: epsilon radius parameter to 0.4
5: Setting minPoints M: Neighborhood density threshold to 1
6: Mark all objects as unvisited
7: while Objects marked unvisited do
8: Randomly select an unvisited object p:
9: Mark p as visited
10: if There are at least M objects in the $\varepsilon$-neighborhood of p then
11: Create a new cluster C, and then add p to C
12: Let N as p's $\varepsilon$-neighborhood for the set of objects.
13: for Each point p' in N do
14: if p' is unvisited then
15: Mark p' as unvisited;
16: if The $\varepsilon$-neighborhood of p' has at least M points then
17: Add these points to N
18: end if
19: if p' is still not a member of any cluster then
20: add p' to C:
21: end if
22: end if
23: end for
24: Output C:
25: else Mark p' as noise
26: end if
27: end while
```
The algorithm first traverses the point cloud data point by point. If the point is not a core point, then it is considered a noise point and ignored, and the noise point may be classified into clusters by the core point later. If it is a core point, then it creates a new cluster, and all neighborhood points are added to the cluster. The core point in the neighborhood point is added to the cluster recursively. Following this way, until no point can be added to the cluster, and then start working for new points to establish a new cluster.
In general, if the dataset is a dense and not a convex set, then using the DBSCAN algorithm will be better than other clustering algorithms, such as K-means. The K-means clustering algorithm can only deal with spherical clusters that are solid mass clusters. This is a limitation of the algorithm based on calculating the average distance to the cluster. If the dataset is not dense, it is not recommended to use DBSCAN for clustering. Compared with the traditional K-means algorithm, the most significant difference of DBSCAN is that there is no need to input the number of categories K. Meanwhile, DBSCAN's most significant advantage is finding clusters of arbitrary shapes rather than K-means, which is generally only used for convex sample set clustering. Finally, the DBSCAN algorithm can also find outliers while clustering, similar to the balanced iterative reducing and clustering using hierarchies (BIRCH) clustering algorithm.
### B. Human Activity Recognition
As the recurrent neural network (RNN) has the problem of gradient explosion and gradient disappearance in dealing with nonlinear time problems, a temporal recursive neural network LSTM is widely used in the time-series data. Due to the advantage of the network structure, it can process and predict highly time-dependent and strongly coupled events.
Traditional neural networks are affected by short-term memory. If the time-series data is too long, it will be difficult to transfer information from the earlier time step to the later time step. For the time-series data, the RNN may miss important information from the start. Because during backpropagation, the RNN will face the problem of gradient disappearance. The gradient is used to update the weight value of neural networks, and the problem of gradient disappearance is that the gradient decreases as the gradient spreads over time. If the gradient becomes very small, the learning cannot be continued.
Compared with the classical method, the LSTM using the LSTM unit can directly connect the feature data of DBSCAN's preprocessing data, input it into the neural network processing, and model the data properly. Diversified with the hidden state of the original RNN, the LSTM adds a cell state. For the human being point cloud data after DBSCAN clustering, it is the time-series structure, and the data is divided into t parts and fed to the LSTM. Finally, the LSTM outputs the classified result about the HAR.
### C. Evaluation and Results
Evaluate the LSTM algorithm training and testing performance after DBSCAN feature extraction on point cloud data. Fig. 5 illustrates confusion matrices from a single subject to multiple subjects. Fig. 5(a) is for a single subject of five activities classification results that is for fall, pick up, sit down, stand up, and walk, and the accuracy is 95.75%. Fig. 5(b) is for two subjects of three activities, which include sit down + stand up, sit down + pick up, and stand up + pick up, and the accuracy is 97.59%. Fig. 5(c) is for three subjects of three activities, which include sit down + stand up + pick up, stand up + pick up + sit down, and pick up + sit down + stand up, and the accuracy is for 98.01%. Fig. 5(d) is for four subjects of four activities, and there is arrangement and combination from sit down, stand up, and pick up. The activities sequence are sit down + stand up + pick up + sit down, sit down + sit down + stand up + pick up, pick up + sit down + sit down + stand up, and stand up + pick up + sit down + sit down, the accuracy of which is 96.90%. This result reflects
---
Fig. 3. Experimental setup data collection. (a) Human movement data acquired by the NodeNs sensor on the fixed table that is 2 m from the volunteer. (b) How to set up four subjects in the data collection environment.
Fig. 4. Experimental workflow for the HAR task.
the performance evaluation of popular LSTM algorithms in the activity recognition task.
### IV. Discussion
In this article, the focus has been on the processing of point cloud data for human movement classification. This is largely motivated by the prominence of HAR datasets, which have been greatly expanding and improving in recent years. Recent work in this area and corresponding benchmark datasets have been discussed. Targeting point cloud data in the HAR field, the early ideas were developed by deep learning and extension of image-processing concepts, for example, multiple view images [^25] or convolutional calculations on 3-D voxel grids [^26] for point cloud data processing. With processing improvements, convolution operations evolved from 2-D convolution kernels to 3-D convolution kernels on the neural network. However, running convolutional calculations on point cloud data is a complicated matter. The main difficulty stems from the fact that the point cloud data does not clearly define the point order for performing convolution calculations [^27]. Qi et al. [^28] solved this problem by using symmetric functions with constant order of points and using it to learn the global characteristics of point cloud data. Alternatively, some other methods are proposed to learn local features from convolution [^29], and algorithms such as autoencoders are used to remedy the defects [^30]. Further research uses data fusion methods, based on point cloud data and multiview projection data to learn features jointly [^31]. Meanwhile, Han et al. [^32] demonstrated serializing the point cloud processing and views to analyze data. Finally, some studies directly use the unsupervised learning method [^33] to complete the HAR task.
Here, the work has conducted a comparative analysis of some HAR benchmark datasets shown in Table II. For example, Taylor et al. [^21] designed a benchmark dataset that includes two activities collected by Universal Software Radio Peripheral (USRP). It uses a machine-learning model with the random forest algorithm to analyze channel state information (CSI) from the radio waves. On their benchmark dataset, the HAR task achieved an accuracy of 92.47%. On the other hand, Khan et al. [^22] proposed a benchmark dataset based on the Kinect V2 sensor that captured 12 classes of human skeleton joints data. It is based on a hybrid approach to combine the convolutional neural network (CNN) and LSTM algorithms and then recognizes human activity with the accuracy of August 16, 2026 at 13:55:09 UTC from IEEE Xplore. Restrictions apply.
---
Fig. 5. Confusion matrices for a single subject to multiple subjects. (a) Single subject of five activities classification results. (b) Two subjects of three activities. (c) Three subjects of three activities. (d) Four subjects of four activities.