# Challenges in Sensor-based Human Activity Recognition and a Comparative Analysis of Benchmark Datasets: A Review

Anindya Das Antar, Masud Ahmed, and Md Atiqur Rahman Ahad $^{*}$

Department of Electrical and Electronic Engineering, University of Dhaka.

Email: $*<atiqahad@du.ac.bd>$

Abstract—Human Activity Recognition using embedded sensors has lately made renowned development and is drawing growing attention in numerous application domains including machine learning, pattern recognition, context awareness, and human-centric sensing. Due to the lacking of a prominent analysis of this topic that can acquaint concomitant communities of the research avant-garde, there are still vital perspectives that, if pleaded, would create a vital turn in the way of interaction among people and mobile devices. In this paper, we have presented a comprehensive survey along with the prevailing state of various challenges of human activity recognition based on wearable, environmental, and smartphone sensors. Firstly, we have shown numerous factors to be considered for the data pre-processing part regarding noise filtering and segmentation methods. Besides, we have made a list of sensing devices, sensors, and applications that can be used for collecting activity data along with a discussion on sensor position and requirements. Moreover, we have made a comprehensive analysis of some benchmark datasets, which includes information about sensors, attributes, activity classes, etc. Finally, we have shown an analysis of activity recognition approaches on some of the benchmark datasets based on existing works.

Index Terms—Benchmark datasets, data pre-processing, human activity recognition, smartphone sensors, survey, wearable sensors.

### I. INTRODUCTION

Over the last few years, human activity recognition (HAR) is one of the most prominent topics in numerous areas including machine learning, mobile computing, context-aware computing, security based on surveillance, and ambient assistive living $^{[1]}$. The aim of HAR is to understand daily behaviours of people through the analysis of observations obtained from people and their neighboring environments of living. This information is collected from different sensors embedded in smartphones, wearable devices, and home settings. Cameras and video-based systems are also used in computer vision to capture daily human activities and automatic recognition is done based on the sequence of images $^{[2]}$. Nowadays, with the advancement of microelectronics and computer systems, there has been a comprehensive development in low-power, high-capacity, low-cost, and miniaturized sensors along with wired and wireless communication networks $^{[3]}$. This is the primary reason people are interacting with the devices and technologies as part of their regular sustenance.

We can use the activity recognition process in daily physical exercise monitoring. Furthermore, the recognition of static activities along with dynamic activities with postural transitions will help to monitor worker's health and working rate in working environments. Moreover, these systems can be essential in sustaining a healthful lifestyle among people by recommending minute behaviour corrections. For security and safety purposes of soldiers in strategic situations, it is required to gather precise information about their activities, health conditions, and locations. This information can be highly advantageous in case of decision making in battle and training situations. Smart homes [^4], for daily activity monitoring is an example of external sensing for monitoring moderately complicated activities (e.g., working on a computer, brushing teeth, eating lunch, etc.). Finally, camera-based systems are eminently proper for surveillance (e.g., intervention disclosure) and interactive purposes. These systems are also used for action recognition using motion history images and computer vision [^6].

### A. Approaches to Human Activity Recognition

In order to achieve the goal of human activity recognition, we require activity recognition systems with sensing capabilities. Two most used methods for this purpose are video-based and sensor-based activity recognition. We can also categorize the sensor-based activity recognition as shown in Figure 1.

Though video-based approach often works good indoor, it fails in obtaining the same precision outdoor or in real life situations [^7], [^8], [^5]. Environmental sensor-based systems passively monitor their occupants. However, these sensors are infrastructure dependent. Wearable sensors are able to regulate physiological parameters, which is difficult to measure using ambient or camera sensors as wearable sensors are attached to the monitoring subject and are independent of the infrastructure [^9], [^10]. Unfortunately, the main constraints of body sensors include patient discomfort while wearing [^11]. Some techniques of self-calibration of sensors in harsh environments

---

have been described in [^12] to solve this problem. In the research of innovative alternatives for the recuperation of data directly from the users, smartphones can play a vital role. One of the benefits of the current smartphone technologies is that they are consolidating inertial sensors that can be exploited for human activity recognition. One issue of using smartphones for HAR is related to the quick energy loss of the smartphone's battery.

In this paper, Section I provides a brief introduction to the concepts of human activity recognition systems along with its importance. In Section II, we have briefly described the basic structure of HAR and several ways of numerous preprocessing and segmentation steps of raw-sensor data. We have discussed the challenges in activity recognition along with sensor information in Section III and IV. Moreover, we have provided the necessary pieces of information about some benchmark datasets in Section V. Following that in Section VI, we have also compared some works based on feature and classifier selection for some of these benchmark datasets. Finally, Section VII draws the main conclusion from the literature and provides directions for future research.

![Image](images/573e524b6511daf88f000747bf4491047e09a6979d8db09b5b80bd0c70ad1a43.jpg)

<div style="text-align: center;">Fig. 1. Approaches employed for human activity recognition.</div>

### II. BASIC STRUCTURE FOR HUMAN ACTIVITY RECOGNITION SYSTEMS

There are many established ways to extract information regarding daily human activities from raw sensor data. We can summarize the fundamental steps as pre-processing of raw sensor data, segmentation, extracting features, and classification.

Pre-processing of raw sensor data: The first step of data pre-processing is noise filtering. Normally, four filters namely median filter, Butterworth low-pass filter, discrete wavelet package shrinkage, and Kalman filter are used to filter acceleration and gyroscope noise [^13]. But in case of choosing one or more filters, we need to be careful about signal-to-noise ratio (SNR), correlation coefficient (R) between filtered signal and the reference signal, cut-off frequency, waveform delay, filter order, window length, etc. Kalman filters show larger SNR and R-value. Performance parameters of the described four filters have been summarized in Table I.

<div style="text-align: center;">TABLE I</div>

<div style="text-align: center;">PERFORMANCE COMPARISON OF FOUR FILTERS.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Filter</td><td style='text-align: center; word-wrap: break-word;'>Real-time performance</td><td style='text-align: center; word-wrap: break-word;'>Waveform delay</td><td style='text-align: center; word-wrap: break-word;'>SNR</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Median</td><td style='text-align: center; word-wrap: break-word;'>Related to window length  N. It has to wait $\frac{N}{2}$ future data points for filtering.</td><td style='text-align: center; word-wrap: break-word;'>Little</td><td style='text-align: center; word-wrap: break-word;'>Moderate</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Butterworth (low-pass)</td><td style='text-align: center; word-wrap: break-word;'>Best real-time performance.</td><td style='text-align: center; word-wrap: break-word;'>Large (depends on filter order and cut-off frequency)</td><td style='text-align: center; word-wrap: break-word;'>Lowest (with waveform delay) Highest (without waveform delay)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Kalman</td><td style='text-align: center; word-wrap: break-word;'>Moderate.</td><td style='text-align: center; word-wrap: break-word;'>Short delay</td><td style='text-align: center; word-wrap: break-word;'>Better than Median and Wavelet filter.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Wavelet</td><td style='text-align: center; word-wrap: break-word;'>Depends on decomposition level  J. It needs to wait $2 \cdot J$ future data points to filter noise.</td><td style='text-align: center; word-wrap: break-word;'>Little</td><td style='text-align: center; word-wrap: break-word;'>Moderate</td></tr></table>

Segmentation: For the segmentation process, some factors need to be considered to select window length. A previous study [^15] showed that extracted features from smaller windows show better quality but it delays the recognition result for the end user. On the contrary, if the activities are performed for short duration, there is a high chance of failure of recognition in case of longer windows. The optimal window length should be chosen based on the activity being performed. In earlier studies, window size has been ranged from 0.25s to 6.7s, whereas some studies suggested a constant percentage of overlapping between adjacent windows [^16], [^17]. From [^18], we have discovered that on an average, features extracted from window lengths of one and two seconds achieve imperceptibly larger precision values than those with other window lengths for human activity recognition.

Feature extraction and classification: After the segmentation stage, we need to focus on extracting more relevant features from segmented data, which can be useful in differentiating more similar activities. We can extract features in the time domain and frequency domain. Besides, Features extracted from the activity graph and energy-based features can also play an important role for activity classification. Mean value, standard deviation, median absolute deviation, minimum and maximum values of the samples in a frame, Signal Magnitude Area, Inter-quartile range, Signal entropy, Auto-regression coefficients are common time domain features. Fast Fourier Transform (FFT) is used to derive the features in the frequency domain. Different features perform different important roles to make the right classification of activities. We can select the features based on importance factor, which can be found using Random Forest attribute. We can also use other feature selection steps like univariate feature selection, removing feature with lower variance and higher correlation, recursive feature elimination, and L1-based feature selection.

Finally, we can apply numerous classification methods based on data type, amount of data, similarities of activities, the number of activities, number of classes, etc. Most common classifiers, for example, Linear Discriminant Analysis (LDA), Logistic Regression (LR), K-Nearest Neighbour (KNN), Random Forest (RnF), Support Vector Machine (SVM), etc. can be used for the classification of dynamic and static activities. But the choice of kernels for SVM, K-value (number of neighbors) and distance calculation method for KNN, number of trees for

---

RnF, tolerance value for LDA, etc. factors play an essential role in precision and accuracies of these classifiers, which needs to be considered with care. We can also use deep learning networks for classifying the human activities. In this case, there is a tradeoff between processing time in real time and performance for deep neural networks as they are costly in terms of processing time.

### III. CHALLENGES IN ACTIVITY RECOGNITION

Human activity recognition algorithms can be assessed on the footing of the recognition complexity and challenges in the field of sensor-based activity recognition. The complexity of the activities can differ and depends on various circumstances including the activity numbers, activity types, choice of sensors, energy consumption, obtrusiveness, and data collection protocols for those activities. Ambulation activities can be classified into three basic types: static activities, dynamic activities, and activities with postural transitions as shown in Figure 2. Static activities along with postural transitions (lying, sitting, etc.) can be recognized easily than periodic activities (running, walking, etc.). However, highly similar postures (sitting and standing) create great complexities in case of separation due to notable overlapping in feature space. Furthermore, dynamic activities (walking upstairs and walking downstairs), which share high similarity in the feature space are also very hard to discriminate because of related action patterns.

![Image](images/1e2fd0638b4462899f944b6c880be5ec2e69d3a7fbc64e2b4be46091fd33cb59.jpg)

<div style="text-align: center;">Fig. 2. Different types of daily human activities.</div>

In most cases, correspondence among performed activities is not similar during the entire period of activities, which makes the recognition even harder. For instance, sitting and standing are much related (difficult to separate), however, they are very distinct from walking (easily separable). Transitional activities can be further divided into four types: static to static postural transition (stand to sit, lie to sit, etc.), static to dynamic (still to walk), dynamic to static (walk to still) and dynamic to dynamic (walk to jog, jog to run, etc.).

Sensor requirements: The complexity of the recognition algorithm can increase significantly by the number of sensors of smartphone used, the type of sensors used, and location of the smartphone while taking data from users.

The number of sensors to choose: A smartphone contains many sensors: such as Wi-Fi, pressure, Bluetooth, accelerometers, magnetometers, barometers, gyroscopes, proximity sensors, temperature and humidity sensors, ambient light sensors, cameras, and microphones. But we need to choose the number of sensors carefully as a recognition system employed with a small set of sensors make the process easier and convenient in real-life application. In the case of wearable sensors, the number of sensors plays an important role because of user comfort. Carrying multiple sensors is difficult for users. This is a tradeoff between the number of sensors and performance which should be dealt with care.

Location of smartphone and wearable sensors: Normally users keep smartphones in their shirt pocket, pant pocket or keep it in hand. As accelerometers handle axis-based motion sensing and gyroscope determines the orientation, the position of smartphone needs to be considered while taking data, as data can alter for various sensor locations on a person's body, even for the same activity. Otherwise, due to the position of smartphones and wearable sensors, wrong recognition of a particular activity can deteriorate the recognition accuracy. Another problem is associated with the fact that some users may not keep their phone with them while they are in the home, making it impossible to track their activity. In this case, a wearable sensor can be a good option though it comes with the problem of discomfort for many users to wear it all day while performing activities.

### IV. AVAILABLE SENSING DEVICES AND TOOLS

For human activity data collection, wearable devices and sensors, merged on home appliances have been used most of the times. But nowadays, smartphones also contain necessary sensors, which can be used for monitoring activities at any place any time. In this section, we have described wearable devices and sensor modules along with some applications and tools that can be utilized for human activity recognition.

Motion Node: Motion Node (<http://www.motionnode.com>) is tiny and lightweight with a 6-DOF inertial measurement unit, which is specially intended for motion sensing purposes. It integrates accelerometer, gyroscope, and magnetometer.

LoRaWAN: LoRaWAN [^22] is a sensor device, which can be interfaced with a wide range of sensors. This is a high-performing device with highest measurement accuracy. LoRaWAN sensor has up to 10 years of system lifetime.

ECO sensors: ECO sensors [^19] are small in dimension without any battery. Each of the ECOs includes an accelerometer sensor intended for the distinct task of infant motion monitoring.

μParts: μParts [^20] is a low-cost small sensor node, which requires a large group of comparatively low sampling rate sensors. This system includes a light, temperature, and a ball switch sensor for motion detection.

TMote Sky: This is an ultra-low power IEEE 802.15.4 compliant wireless sensor module (<http://www.moteiv.com>). It contains humidity, light, and temperature sensors with USB, which facilitates a broad range of mesh network applications.

---

Smart-Its: This is a secured platform for smart objects. This reflects the theory of the “Disappearing Computer”, which sets computing in the framework of people’s interaction with their surrounding environment [^21].

Luna Nurse: Luna Nurse (<http://www.g-mark.org>) is a bed leaving sensor, which can detect three different states of care recipient on the bed-rising up, sitting on the edge of the bed, and leaving the bed by sensing the movement of that person. Device Analyzer: This is an app that works on Android devices and collects usage statistics in the background while the phone is in use (<http://deviceanalyzer.cl.cam.ac.uk>). It has an analytics option to extract data, monitoring our everyday work.

HASC tool and HASC logger: HASC Tool (<http://hasc.jp/tools/hasctool-en.html>) is a tool for the action information processing developed with Eclipse Plugin. Bellabeat: Bellabeat (<http://gadgetsandwearables.com/bellabeat>) is a wellness-oriented outfit dedicated to creating attractive connected wearables for women. This smart piece of jewellery is designed for tracking health conditions. This can be worn as a necklace or bracelet with accessories or items.

UCI machine-learning repository: University of California Irvine machine learning repository hosts datasets from different domains, which are typically in CSV format. Datasets in this repository contain information regarding attribute types, missing values, target domain, etc. The main advantages are flexibility in data donation and diversity of data. In Table II, we have summarized publicly available smartphone and wearable sensor-based activity recognition datasets from this repository.

<div style="text-align: center;">TABLE II</div>

### V. BENCHMARK DATASETS

Researchers have to face many difficulties including technical challenges, subject privacy issue, organizational authority, etc. in the process of data collection. Because of these difficulties, there are currently few datasets available in sensor-based activity recognition. In this section, we have gathered benchmark datasets from well-known repositories with relevant information like dataset characteristics, attribute characteristics, number of attributes and classes, etc.

<div style="text-align: center;">LIST OF UCI ML REPOSITORY DATASETS.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Dataset</td><td style='text-align: center; word-wrap: break-word;'>Subjects (Activities)</td><td style='text-align: center; word-wrap: break-word;'>Year</td><td style='text-align: center; word-wrap: break-word;'>Devices</td><td style='text-align: center; word-wrap: break-word;'>Sensors/Modules</td><td style='text-align: center; word-wrap: break-word;'>Missing values</td><td style='text-align: center; word-wrap: break-word;'>Dataset characteristics</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>HAR [^23]</td><td style='text-align: center; word-wrap: break-word;'>30 (6)</td><td style='text-align: center; word-wrap: break-word;'>2012</td><td style='text-align: center; word-wrap: break-word;'>Smartphone</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer and gyroscope</td><td style='text-align: center; word-wrap: break-word;'>No</td><td style='text-align: center; word-wrap: break-word;'>Continuous</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>HAPT [^24]</td><td style='text-align: center; word-wrap: break-word;'>30 (12)</td><td style='text-align: center; word-wrap: break-word;'>2015</td><td style='text-align: center; word-wrap: break-word;'>Smartphone</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer and gyroscope</td><td style='text-align: center; word-wrap: break-word;'>No</td><td style='text-align: center; word-wrap: break-word;'>Continuous</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Single Chest [^25]</td><td style='text-align: center; word-wrap: break-word;'>15 (7)</td><td style='text-align: center; word-wrap: break-word;'>2014</td><td style='text-align: center; word-wrap: break-word;'>Device on chest</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer at 52 Hz</td><td style='text-align: center; word-wrap: break-word;'>No</td><td style='text-align: center; word-wrap: break-word;'>Continuous</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>OPPOR-TUNITY [^26]</td><td style='text-align: center; word-wrap: break-word;'>4 (35)</td><td style='text-align: center; word-wrap: break-word;'>2012</td><td style='text-align: center; word-wrap: break-word;'>Wearable object</td><td style='text-align: center; word-wrap: break-word;'>Accelerometers, motion, and ambient sensors</td><td style='text-align: center; word-wrap: break-word;'>Yes</td><td style='text-align: center; word-wrap: break-word;'>Isolated</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>REAL-DISP [^27]</td><td style='text-align: center; word-wrap: break-word;'>17 (33)</td><td style='text-align: center; word-wrap: break-word;'>2014</td><td style='text-align: center; word-wrap: break-word;'>Wearable device</td><td style='text-align: center; word-wrap: break-word;'>Accelerometers, gyroscopes, and magnetic sensor.</td><td style='text-align: center; word-wrap: break-word;'>No</td><td style='text-align: center; word-wrap: break-word;'>Continuous</td></tr></table>

N.A: Not Available.

Wearable sensor-based datasets: Wearable devices need to be designed considering the flexibility of users. Light-weight, fashionable and comfortable wearable devices with embedded sensors for activity monitoring had been used in multiple datasets. In Table III, some of these publicly available datasets have been discussed with relevant information.

<div style="text-align: center;">TABLE III</div>

<div style="text-align: center;">LIST OF WEARABLE SENSOR-BASED DATASETS.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Dataset</td><td style='text-align: center; word-wrap: break-word;'>Subjects (Activities)</td><td style='text-align: center; word-wrap: break-word;'>Applications</td><td style='text-align: center; word-wrap: break-word;'>Year</td><td style='text-align: center; word-wrap: break-word;'>Devices</td><td style='text-align: center; word-wrap: break-word;'>Sensors/ Module</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>USC-HAD [^28]</td><td style='text-align: center; word-wrap: break-word;'>14 (12)</td><td style='text-align: center; word-wrap: break-word;'>Healthcare</td><td style='text-align: center; word-wrap: break-word;'>2012</td><td style='text-align: center; word-wrap: break-word;'>Motion Node</td><td style='text-align: center; word-wrap: break-word;'>accelerometer and gyroscope.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>WARD [^29]</td><td style='text-align: center; word-wrap: break-word;'>20 (13)</td><td style='text-align: center; word-wrap: break-word;'>Human action recognition</td><td style='text-align: center; word-wrap: break-word;'>2009</td><td style='text-align: center; word-wrap: break-word;'>DexterNet</td><td style='text-align: center; word-wrap: break-word;'>accelerometer and gyroscope.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>SKODA [^30]</td><td style='text-align: center; word-wrap: break-word;'>1 (10)</td><td style='text-align: center; word-wrap: break-word;'>car maintenance</td><td style='text-align: center; word-wrap: break-word;'>2007</td><td style='text-align: center; word-wrap: break-word;'>Wearable system</td><td style='text-align: center; word-wrap: break-word;'>accelerometer</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>HCI [^31]</td><td style='text-align: center; word-wrap: break-word;'>1 (5)</td><td style='text-align: center; word-wrap: break-word;'>Human-computer interaction</td><td style='text-align: center; word-wrap: break-word;'>2009</td><td style='text-align: center; word-wrap: break-word;'>Wearable system</td><td style='text-align: center; word-wrap: break-word;'>accelerometer</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>PPS Grouping [^32]</td><td style='text-align: center; word-wrap: break-word;'>10 (2)</td><td style='text-align: center; word-wrap: break-word;'>Detecting walking group</td><td style='text-align: center; word-wrap: break-word;'>2009</td><td style='text-align: center; word-wrap: break-word;'>Wearable System.</td><td style='text-align: center; word-wrap: break-word;'>accelerometer</td></tr></table>

Medical activities related datasets: Medical activities recognition has numerous application for the remote monitoring of patients, pregnant women and elderly people with heart diseases. In Table IV, we have listed some datasets related to medical activities, where Dophnet FoG and Predicting Parkinson's Disease datasets are publicly available.

<div style="text-align: center;">TABLE IV</div>

<div style="text-align: center;">LIST OF MEDICAL ACTIVITIES RELATED DATASETS.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Dataset</td><td style='text-align: center; word-wrap: break-word;'>Subjects (Activities)</td><td style='text-align: center; word-wrap: break-word;'>Applications</td><td style='text-align: center; word-wrap: break-word;'>Year</td><td style='text-align: center; word-wrap: break-word;'>Devices</td><td style='text-align: center; word-wrap: break-word;'>Sensors/ Module</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Daphnet FoG [^33]</td><td style='text-align: center; word-wrap: break-word;'>10 (3)</td><td style='text-align: center; word-wrap: break-word;'>Monitoring PD patients&#x27; walk and detecting sporadic freezing of gait</td><td style='text-align: center; word-wrap: break-word;'>2010</td><td style='text-align: center; word-wrap: break-word;'>Wearable device</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Nursing Activity [^34]</td><td style='text-align: center; word-wrap: break-word;'>82 (25)</td><td style='text-align: center; word-wrap: break-word;'>Monitoring nursing activities in the hospital</td><td style='text-align: center; word-wrap: break-word;'>2016</td><td style='text-align: center; word-wrap: break-word;'>iPod</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Predicting Parkinson&#x27;s Disease [^35]</td><td style='text-align: center; word-wrap: break-word;'>9 (2)</td><td style='text-align: center; word-wrap: break-word;'>Monitoring and measure symptoms of PD disease</td><td style='text-align: center; word-wrap: break-word;'>2011</td><td style='text-align: center; word-wrap: break-word;'>Smartphone</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer, compass ambient light, proximity, battery level, GPS, and audio sensors</td></tr></table>

Smartphone sensor-based datasets: Smartphones have become a daily companion of most of the people throughout the day nowadays. So, it is easy to monitor user activities using embedded sensors data from smartphones. Because of the availability of sensors and cost-effective approaches, smartphone sensor-based activity recognition has gained many researcher's interest. In Table V, we have listed some publicly available datasets of this approach but HASC-IPSC is not publicly available.

<div style="text-align: center;">TABLE V</div>

<div style="text-align: center;">LIST OF SMARTPHONE SENSOR-BASED AMBULATION ACTIVITY DATASETS.</div>

<table border=1 style='margin: auto; word-wrap: break-word;'><tr><td style='text-align: center; word-wrap: break-word;'>Dataset</td><td style='text-align: center; word-wrap: break-word;'>Subjects (Activities)</td><td style='text-align: center; word-wrap: break-word;'>Applications</td><td style='text-align: center; word-wrap: break-word;'>Year</td><td style='text-align: center; word-wrap: break-word;'>Devices</td><td style='text-align: center; word-wrap: break-word;'>Sensors/ Module</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>WISDM [^36]</td><td style='text-align: center; word-wrap: break-word;'>29</td><td style='text-align: center; word-wrap: break-word;'>Activity monitoring</td><td style='text-align: center; word-wrap: break-word;'>2012</td><td style='text-align: center; word-wrap: break-word;'>Android phones</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>Smart Devices [^37]</td><td style='text-align: center; word-wrap: break-word;'>N.A (11)</td><td style='text-align: center; word-wrap: break-word;'>Human behaviour recognition</td><td style='text-align: center; word-wrap: break-word;'>2017</td><td style='text-align: center; word-wrap: break-word;'>Smartphone, smartwatch</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer, gyroscope, EOG, and pressure sensor.</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>HASC 2010 Corpus [^38]</td><td style='text-align: center; word-wrap: break-word;'>540 (6)</td><td style='text-align: center; word-wrap: break-word;'>Activity recognition</td><td style='text-align: center; word-wrap: break-word;'>2010</td><td style='text-align: center; word-wrap: break-word;'>Smartphone, smartwatch, and smartglass</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer. Sampling rate: (10-100 Hz)</td></tr><tr><td style='text-align: center; word-wrap: break-word;'>HASC IPSC [^39]</td><td style='text-align: center; word-wrap: break-word;'>107</td><td style='text-align: center; word-wrap: break-word;'>Indoor position and building structure estimation</td><td style='text-align: center; word-wrap: break-word;'>2014</td><td style='text-align: center; word-wrap: break-word;'>Smartphone</td><td style='text-align: center; word-wrap: break-word;'>Accelerometer, wifi, pressure sensor, angular velocity, and geomagnetism sensor.</td></tr></table>

---

### VI. DISCUSSION

### A. Classification approaches of previous works

In this section, we have made a comparative analysis of the classification approaches of previous works for some of the benchmark datasets. In [^40], they developed a system by collecting data from 6 subjects that included 12 daily activities using a waist-mounted accelerometer. This work proposed a method to deliver the maximum amount of signal processing inside the board of the wearable unit. Their proposal also includes the use of embedded intelligence and real-time classification system implementation. They achieved an overall accuracy of 90.8%, whereas, recognition of postural orientation was 94.1% accurate and possible fall detection was made with 95.6% accuracy. On the other hand, in the paper [^41], 1D Haar-like filtering techniques are proposed as these are not only new techniques of feature extraction but also lower calculation is required. Their proposed method achieved 93.91% recognition accuracy while diminishing calculation cost to 21.22% compared to preceding methods. In [^42], features were further processed by a Kernel Principal Component Analysis and Linear Discriminant Analysis after extraction to make them more robust. Finally, they used a Deep Belief Network for training the features. They found 89.61% accuracy, which outperformed typical multiclass Support Vector Machine (82.02% accuracy) and Artificial Neural Network (65.31%).

In recent years, there have been various research works in the field of sensor-based activity recognition using deep learning and general classification methods. Semi-supervised machine learning based activity recognition technique based on self-training procedure was implemented by [^43], for a very short amount of labeled training data to overcome the problem of generating training dataset with wrong labeling. In [^44], K-Means clustering algorithm based unsupervised machine learning method was used to identify human activities. But the performance of these methods is limited when there is a mixture of static and dynamic activity in the dataset. Research work in [^45] started new research about the estimation of the user's route by sensing data. They stated that successful development of this route estimation technology from sensing data can improve lifelogging, context-aware services, or location-aware services. To improve mobile activity recognition with incorrect segments in which the starting and ending timestamps of uniform and continuous activities have inaccurate boundaries, EM+Sparse (EM algorithms and sparse segmentation) and EM+Dense (EM algorithms and dense segmentation) methods have been used in [^46] for HASC and UCI HAR datasets.

Inoue et al. [^47], on the other hand, proposed a method with high throughput from raw accelerometer data using a deep recurrent neural network, which ensured shorter recognition time than other naive methods. Research work [^48] used a multiclass hardware-friendly Support Vector Machine (k=8 bits) using 17 features in time and frequency domain that can help to release system resources and it could lessen energy expenditures for smartphones as Multi-Class Hardware-friendly Support Vector Machine approach makes use of fixed-point arithmetic for the identification of activities instead of the conventionally used floating-point arithmetic algorithms. Ref. [^49] proposed a post processing technique named Mod technique along with Random Forest classifiers for the classification of locomotion and transportation activities. In other work [^50], Adaboost ensemble classifier is used to recognize activity data taken from body sensors by using a weighted combination of several classifiers.

### B. Future scopes

In future research work, analysis of resource consumption should be done, which includes memory, CPU, number of sensors, and most importantly, the battery usage. The most common trade-off among the recognition accuracy, precision, and resource usage should be explored more deeply to get the best possible outcome. Besides, for the correct recognition of the most similar activities like sitting and standing or walking, walking upstairs, and walking downstairs, a group of classifier based method can be used. Most of the existing works have not succeeded with one classifier to distinguish similar activities precisely. It should also be included in future plans to work with a manually labeled training data that is not correctly labeled such as walking activity has been wrongly labeled as jogging activity due to human fault. Incorporation of video data with sensor data can be an important area to explore.

### VII. CONCLUSION

With the emerging growth and technological advancement in the field of sensors, activity recognition has become the latest wave of context-aware personalized applications in numerous emerging computing areas. But the fact is, being a new field of research there are not much survey works in sensor-based activity recognition. Most of the time researchers fail to find benchmark datasets, which make their work difficult. This paper surveys the state-of-the-art human activity recognition where we have gathered some benchmark datasets on daily activities using wearable and smartphone sensors. Detailed information related to attributes, activity classes, types of sensors, and devices used for these datasets have been given. Besides, we have made a list that includes all kinds of sensing devices and application tools that can be used for generating a new dataset. Several noise filtering methods, choice of filters, segmentation methods, and factors to be considered to choose the window length have been described in detail. Moreover, a summary and comprehensive analysis of previous activity recognition techniques have been given. Finally, numerous plans are offered for future research to elongate this field to more practical and pervasive scenarios.

## REFERENCES

[^1]: M. Philipose, K. P. Fishkin, M. Perkowitz, D. J. Patterson, D. Fox, H. Kautz, and D. Hahnel, “Inferring activities from interactions with objects,” in IEEE pervasive computing, pp. 50–57, 2004.

[^2]: P. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea, “Machine recognition of human activities: A survey,” in IEEE Trans. on Circuits and Systems for Video technology, pp. 1473–1488, 2008.

---

[^3]: D. Ding, R. A. Cooper, P. F. Pasquina, and L. Fici-Pasquina, “Sensor technology for smart homes,” in Maturitas, pp. 131–136, 2011.

[^4]: A. Tolstikov, X. Hong, J. Biswas, C. Nugent, L. Chen, and G. Parente, “Comparison of fusion methods based on dst and dbn in human activity recognition,” in Journal of Control Theory and Applications, pp. 18–27, 2011.

[^5]: M. A. R. Ahad, in Computer vision and action recognition, Springer, 2011.

[^6]: M. A. R. Ahad, in Motion History Images for Action Recognition and Understanding, Springer, 2012.

[^7]: T. Tan, M. Gochoo, S. Huang, Y. Liu, S. Liu, and Y. Huang, "Multi-resident activity recognition in a smart home using rgb activity image and dcnn," in IEEE Sensors Journal, 2018.

[^8]: J. Liu, A. Shahroudy, D. Xu, A. C. Kot, and G. Wang, “Skeleton-based action recognition using spatio-temporal lstm network with trust gates,” in IEEE Trans. on PAMI, pp. 3007–3021, 2018.

[^9]: S. C. Mukhopadhyay, “Wearable sensors for human activity monitoring: A review,” in IEEE sensors journal, pp. 1321–1330, 2015.

[^10]: M. Z. Uddin and M. M. Hassan, “Activity recognition for cognitive assistance using body sensors data and deep convolutional neural network,” in IEEE Sensors Journal, pp. 1–1, 2018.

[^11]: O. D. Lara, M. A. Labrador in et al., "A survey on human activity recognition using wearable sensors." in IEEE Communications Surveys and Tutorials, pp. 1192–1209, 2013.

[^12]: L. Gou, D. Peng, X. Chen, L. Wu, and Q. Tang, “A self-calibration method for angular displacement sensor working in harsh environments,” in IEEE Sensors Journal, pp. 1–1, 2018.

[^13]: M. Huang, G. Zhao, L. Wang, and F. Yang, “A pervasive simplified method for human movement pattern assessing,” ICPADS, IEEE, pp. 625–628, 2010.

[^14]: E. K. Antonsson and R. W. Mann, “The frequency content of gait,” in Journal of biomechanics, pp. 39–47, 1985.

[^15]: A. M. Khan, “Human activity recognition using a single tri-axial accelerometer,” in Department of Computer Engineering, Graduate School, Kyung Hee University, Seoul, Korea, 2011.

[^16]: S. J. Preece, J. Y. Goulermas, L. P. Kenney, D. Howard, K. Meijer, and R. Crompton, "Activity identification using body-mounted sensors: a review of classification techniques," in Physiological measurement, 2009.

[^17]: L. Bao and S. S. Intille, “Activity recognition from user-annotated acceleration data,” in in Int. Conf. on Pervasive Computing, Springer, pp. 1–17, 2004.

[^18]: T. Huynh and B. Schiele, “Analyzing features for activity recognition,” in Proc. of the 2005 joint conf. on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies, ACM, pp. 159–163, 2005.

[^19]: C. Park, J. Liu, and P. H. Chou, “Eco: an ultra-compact low-power wireless sensor node for real-time motion monitoring,” in Proc. of the 4th int. symp. on Info. processing in sensor networks, IEEE, 2005.

[^20]: M. Beigl, C. Decker, A. Krohn, T. Riedel, and T. Zimmer, “ $\mu$parts: Low cost sensor networks at scale,” in UbiComp 2005, 2005.

[^21]: M. Beigl and H. Gellersen, “Smart-its: An embedded platform for smart objects,” in Smart Objects Conf. (sOc), 2003.

[^22]: T. Hossain, M. A. R. Ahad, T. Tazin, and S. Inoue, “Activity recognition by using lorawan sensor,” UbiComp Adjunct, pp. 58–61, 2018.

[^23]: D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, “A public domain dataset for human activity recognition using smartphones.” in ESANN, 2013.

[^24]: J.-L. Reyes-Ortiz, L. Oneto, A. Samà, X. Parra, and D. Anguita, “Transition-aware human activity recognition using smartphones,” in Neurocomputing, pp. 754–767, 2016.

[^25]: P. Casale, O. Pujol, and P. Radeva, “Personalization and user verification in wearable systems using biometric walking patterns,” in Personal and Ubiquitous Computing, pp. 563–580, 2012.

[^26]: R. Chavarriaga, H. Sagha, A. Calatroni, S. T. Digumarti, G. Tröster, J. d. R. Millán, and D. Roggen, “The opportunity challenge: A benchmark database for on-body sensor-based activity recognition,” in Pattern Recognition Letters, pp. 2033–2042, 2013.

[^27]: O. Baños, M. Damas, H. Pomares, I. Rojas, M. A. Tóth, and O. Amft, "A benchmark dataset to evaluate sensor displacement in activity recognition," UbiComp, pp. 1026–1035, 2012.

[^28]: M. Zhang and A. A. Sawchuk, “Usc-had: a daily activity dataset for ubiquitous activity recognition using wearable sensors,” UbiComp, pp. 1036–1043, 2012.

[^29]: A. Y. Yang, R. Jafari, S. S. Sastry, and R. Bajcsy, “Distributed recognition of human actions using wearable motion sensor networks,” in Journal of Ambient Intelligence and Smart Environments, pp. 103–115, 2009.

[^30]: T. Stiefmeier, D. Roggen, and G. Troster, “Fusion of string-matched templates for continuous activity recognition,” in Wearable Computers, 2007 11th IEEE International Symposium on. IEEE, 2007, pp. 41–44.

sinner self-

there robustness

agains.

urable Computers, 2009.

ISWC'09. Inte.

... IEEE, 2009, pp. 77–84.

[^32]: M. Wirz, D. Roggen, and G. Troster, “Decentralized detection of group formations from wearable acceleration sensors,” Int. Conf. on Computational Science and Engineering, pp. 952–959, 2009.

[^33]: M. Bachlin, M. Plotnik, D. Roggen, I. Maidan, J. M. Hausdorff, N. Giladi, and G. Troster, “Wearable assistant for parkinson's disease patients with the freezing of gait symptom,” in IEEE Transactions on Information Technology in Biomedicine, pp. 436–446, 2010.

[^34]: S. Inoue, N. Ueda, Y. Nohara, and N. Nakashima, “Recognizing and understanding nursing activities for a whole day with a big dataset,” in Journal of Info. Processing, pp. 853–866, 2016.

[^35]: “Predicting parkinson's disease progression with smartphone data,” <https://www.kaggle.com/c/3300/download/Participant%20Description.xls>, accessed on 2019-01-27.

[^36]: J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity recognition using cell phone accelerometers,” in ACM SigKDD Explorations Newsletter, pp. 74–82, 2011.

[^37]: S. Faye, N. Louveton, S. Jafarnejad, R. Kryvchenko, and T. Engel, “An open dataset for human activity analysis using smart devices,” 2017.

[^38]: “Hasc2010 corpus.” <http://hasc.jp>, accessed on 2019-02-12.

[^39]: K. Kaji, H. Watanabe, R. Ban, and N. Kawaguchi, “Hasc-ipsc: indoor pedestrian sensing corpus with a balance of gender and age for indoor positioning and floor-plan generation researches,” UbiComp, pp. 605–610, 2013.

[^40]: D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and B. G. Celler, “Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring,” in IEEE trans. on information technology in biomedicine, pp. 156–167, 2006.

[^41]: Y. Hanai, J. Nishimura, and T. Kuroda, “Haar-like filtering for human activity recognition using 3d accelerometer,” DSP/SPE, pp. 675–678, 2009.

[^42]: M. M. Hassan, M. Z. Uddin, A. Mohamed, and A. Almogren, “A robust human activity recognition system using smartphone sensors and deep learning,” in Future Generation Computer Systems, pp. 307–313, 2018.

[^43]: M. Stikic, K. Van Laerhoven, and B. Schiele, “Exploring semi-supervised and active learning for activity recognition,” 2008.

[^44]: I. P. Machado, A. L. Gomes, H. Gamboa, V. Paixão, and R. M. Costa, “Human activity data discovery from triaxial accelerometer sensor: Non-supervised learning sensitivity to feature extraction parametrization,” in Information Processing & Management, pp. 204–214, 2015.

[^45]: N. Kawaguchi, H. Watanabe, T. Yang, N. Ogawa, Y. Iwasaki, K. Kaji, T. Terada, K. Murao, H. Hada, and S. Inoue, “Hasc2012corpus: Large scale human activity corpus and its application,” in Proc. of the IPSN, 2012.

[^46]: T. Toda, S. Inoue, and N. Ueda, “Mobile activity recognition through training labels with inaccurate activity segments,” in Proc. of the 13th Int. Conf. on Mobile and Ubiquitous Systems: Computing, Networking and Services. ACM, pp. 57–64, 2016.

[^47]: M. Inoue, S. Inoue, and T. Nishida, “Deep recurrent neural network for mobile human activity recognition with high throughput,” in Artificial Life and Robotics, pp. 173–185, 2018.

[^48]: D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, “A public domain dataset for human activity recognition using smartphones.” in ESANN, 2013.

[^49]: A. D. Antar, M. Ahmed, M. S. Ishrak, and M. A. R. Ahad, “A comparative approach to classification of locomotion and transportation modes using smartphone sensor data,” UbiComp Adjunct, pp. 1497–1502, 2018.

[^50]: A. Subasi, D. H. Dammas, R. D. Alghamdi, R. A. Makawi, E. A. Albiety, T. Brahimi, and A. Sarirete, “Sensor based human activity recognition using adaboost ensemble classifier,” in Procedia Computer Science, pp. 104–111, 2018.
