Smartphone-based Activity Recognition Research Articles

A novel smartphone-based activity recognition modeling method for tracked equipment in forest operations.

Activity recognition modelling using smartphone Inertial Measurement Units (IMUs) is an underutilized resource defining and assessing work efficiency for a wide range of natural resource management tasks. This study focused on the initial development and validation of a smartphone-based activity recognition system for excavator-based mastication equipment working in Ponderosa pine (Pinus ponderosa) plantations in North Idaho, USA. During mastication treatments, sensor data from smartphone gyroscopes, accelerometers, and sound pressure meters (decibel meters) were collected at three sampling frequencies (10, 20, and 50 hertz (Hz)). These data were then separated into 9 time domain features using 4 sliding window widths (1, 5, 7.5 and 10 seconds) and two levels of window overlap (50% and 90%). Random forest machine learning algorithms were trained and evaluated for 40 combinations of model parameters to determine the best combination of parameters. 5 work elements (masticate, clear, move, travel, and delay) were classified with the performance metrics for individual elements of the best model (50 Hz, 10 second window, 90% window overlap) falling within the following ranges: area under the curve (AUC) (95.0% - 99.9%); sensitivity (74.9% - 95.6%); specificity (90.8% - 99.9%); precision (81.1% - 98.3%); F1-score (81.9% - 96.9%); balanced accuracy (87.4% - 97.7%). Smartphone sensors effectively characterized individual work elements of mechanical fuel treatments. This study is the first example of developing a smartphone-based activity recognition model for ground-based forest equipment. The continued development and dissemination of smartphone-based activity recognition models may assist land managers and operators with ubiquitous, manufacturer-independent systems for continuous and automated time study and production analysis for mechanized forest operations.

Smartphone-Based Activity Recognition in a Pedestrian Navigation Context.

In smartphone-based pedestrian navigation systems, detailed knowledge about user activity and device placement is a key information. Landmarks such as staircases or elevators can help the system in determining the user position when located inside buildings, and navigation instructions can be adapted to the current context in order to provide more meaningful assistance. Typically, most human activity recognition (HAR) approaches distinguish between general activities such as walking, standing or sitting. In this work, we investigate more specific activities that are tailored towards the use-case of pedestrian navigation, including different kinds of stationary and locomotion behavior. We first collect a dataset of 28 combinations of device placements and activities, in total consisting of over 6 h of data from three sensors. We then use LSTM-based machine learning (ML) methods to successfully train hierarchical classifiers that can distinguish between these placements and activities. Test results show that the accuracy of device placement classification (97.2%) is on par with a state-of-the-art benchmark in this dataset while being less resource-intensive on mobile devices. Activity recognition performance highly depends on the classification task and ranges from 62.6% to 98.7%, once again performing close to the benchmark. Finally, we demonstrate in a case study how to apply the hierarchical classifiers to experimental and naturalistic datasets in order to analyze activity patterns during the course of a typical navigation session and to investigate the correlation between user activity and device placement, thereby gaining insights into real-world navigation behavior.

Optimal Feature Set for Smartphone-based Activity Recognition

Human activity recognition using wearable and mobile devices is used for decades to monitor humans’ daily behaviours. In recent years as smartphones being widely integrated into our daily lives, the use of smartphone’s built-in sensors in human activity recognition has been receiving more attention, in which smartphone accelerometer plays the main role. However, in comparison to the standard machine, when developing human activity recognition using a smartphone, the limitations such as processing capability and energy consumption should be taken into consideration, and therefore, a trade-off between performance and computational complexity should be considered. In this paper, we shed light on the importance of feature selection and its impact on simplifying the activity classification process, which enhances the computational complexity of the system. The novelty of this work is related to identifying the most efficient features for the detection of each individual activity uniquely. In an experimental study with human users and using different smartphones, we investigated how to achieve an optimal feature set, using which the system complexity can be decreased while the activity recognition accuracy remains high. For that, in the considered scenario, we instructed the participants to perform different activities, including static, dynamic, going up and down the stairs, and walking fast and slow while freely holding a smartphone in their hands. To evaluate the obtained optimal feature set implementing two major classification algorithms, the decision tree and the Bayesian network, we investigated activity recognition accuracy for different activities. We further evaluated the optimal feature set by comparing the performance of the activity recognition system using the optimal feature set and three feature sets taken from the state-of-the-art. The experimental results demonstrated that replacing a large number of conventional features with an optimal feature set has only a negligible impact on the overall activity recognition system performance while it can significantly decrease the system’s complexity, which is essential for smartphone-based systems.

Deep Learning for Activity Recognition in Older People Using a Pocket-Worn Smartphone.

Activity recognition can provide useful information about an older individual’s activity level and encourage older people to become more active to live longer in good health. This study aimed to develop an activity recognition algorithm for smartphone accelerometry data of older people. Deep learning algorithms, including convolutional neural network (CNN) and long short-term memory (LSTM), were evaluated in this study. Smartphone accelerometry data of free-living activities, performed by 53 older people (83.8 ± 3.8 years; 38 male) under standardized circumstances, were classified into lying, sitting, standing, transition, walking, walking upstairs, and walking downstairs. A 1D CNN, a multichannel CNN, a CNN-LSTM, and a multichannel CNN-LSTM model were tested. The models were compared on accuracy and computational efficiency. Results show that the multichannel CNN-LSTM model achieved the best classification results, with an 81.1% accuracy and an acceptable model and time complexity. Specifically, the accuracy was 67.0% for lying, 70.7% for sitting, 88.4% for standing, 78.2% for transitions, 88.7% for walking, 65.7% for walking downstairs, and 68.7% for walking upstairs. The findings indicated that the multichannel CNN-LSTM model was feasible for smartphone-based activity recognition in older people.

Using Smartphone Accelerometer for Human Physical Activity and Context Recognition in-the-Wild

Adaptation of smart devices is frequently rising, where a new generation of smartphones is growing into an emerging platform for personal computing, monitoring, and private data processing. Smartphone sensing allows collecting data from immediate environments and surroundings to recognize human daily living activities and behavioral contexts. Although smartphone-based activity recognition is universal; however, there is a need for coinciding recognition of in-the-wild human physical activities and the associated contexts. This research work proposes a two-level scheme for in-the-wild recognition of human physical activities and the corresponding contexts based on the smartphone accelerometer data. Different classifiers are used for experimentation purposes, and the achieved results validate the efficiency of the proposed scheme.

Feature extraction and feature selection in smartphone-based activity recognition

Nowadays, smartphones are gradually being integrated in our daily lives, and they can be considered powerful tools for monitoring human activities. However, due to the limitations of processing capability and energy consumption of smartphones compared to standard machines, a trade-off between performance and computational complexity must be considered when developing smartphone-based systems. In this paper, we shed light on the importance of feature selection and its impact on simplifying the activity classification process which enhances the computational complexity of the system. Through an in-depth survey on the features that are widely used in state-of-the-art studies, we selected the most common features for sensor-based activity classification, namely conventional features. Then, in an experimental study with 10 participants and using 2 different smartphones, we investigated how to reduce system complexity while maintaining classification performance by replacing the conventional feature set with an optimal set. For this reason, in the considered scenario, the users were instructed to perform different static and dynamic activities, while freely holding a smartphone in their hands. In our comparison to the state-of-the-art approaches, we implemented and evaluated major classification algorithms, including the decision tree and Bayesian network. We demonstrated that replacing the conventional feature set with an optimal set can significantly reduce the complexity of the activity recognition system with only a negligible impact on the overall system performance.

Smartphone-Based Activity Recognition for Indoor Localization Using a Convolutional Neural Network.

In the indoor environment, the activity of the pedestrian can reflect some semantic information. These activities can be used as the landmarks for indoor localization. In this paper, we propose a pedestrian activities recognition method based on a convolutional neural network. A new convolutional neural network has been designed to learn the proper features automatically. Experiments show that the proposed method achieves approximately 98% accuracy in about 2 s in identifying nine types of activities, including still, walk, upstairs, up elevator, up escalator, down elevator, down escalator, downstairs and turning. Moreover, we have built a pedestrian activity database, which contains more than 6 GB of data of accelerometers, magnetometers, gyroscopes and barometers collected with various types of smartphones. We will make it public to contribute to academic research.

WOODY: A Post-Process Method for Smartphone-Based Activity Recognition

In the past decade, the rapid popularization of smartphone has provided a promising direction for human activity recognition. Despite identifying a variety of movements without any complicated wearable device, the smartphone-based activity recognition is still deeply affected by the differences between users and phone locations. To overcome this problem, post-process attempts to correct the errors in the classified activity sequence. In consideration of both the activity sequence continuity and the recognition result confidence, we propose WOODY, a novel post-process method that locates and corrects the errors in a classified activity sequence just like Woody Woodpecker pecking holes to catch the pests. In our method, the recognition result is considered as the weighted observation state, and a weighted observation hidden Markov model (WOHMM) is built to model the classified activity sequence. Consequently, a sequence labeling algorithm of the WOHMM is also designed to modify those recognition results with low confidence. To validate the effectiveness of WOODY, we make a series of contrast experiments on two public data sets collected from real scenarios. The results show that WOODY is not only able to improve the recognition accuracy but also significantly enhance the robustness.

Smartphone-Based Patients' Activity Recognition by Using a Self-Learning Scheme for Medical Monitoring.

Smartphone based activity recognition has recently received remarkable attention in various applications of mobile health such as safety monitoring, fitness tracking, and disease prediction. To achieve more accurate and simplified medical monitoring, this paper proposes a self-learning scheme for patients' activity recognition, in which a patient only needs to carry an ordinary smartphone that contains common motion sensors. After the real-time data collection though this smartphone, we preprocess the data using coordinate system transformation to eliminate phone orientation influence. A set of robust and effective features are then extracted from the preprocessed data. Because a patient may inevitably perform various unpredictable activities that have no apriori knowledge in the training dataset, we propose a self-learning activity recognition scheme. The scheme determines whether there are apriori training samples and labeled categories in training pools that well match with unpredictable activity data. If not, it automatically assembles these unpredictable samples into different clusters and gives them new category labels. These clustered samples combined with the acquired new category labels are then merged into the training dataset to reinforce recognition ability of the self-learning model. In experiments, we evaluate our scheme using the data collected from two postoperative patient volunteers, including six labeled daily activities as the initial apriori categories in the training pool. Experimental results demonstrate that the proposed self-learning scheme for activity recognition works very well for most cases. When there exist several types of unseen activities without any apriori information, the accuracy reaches above 80% after the self-learning process converges.

Energy Efficient Smartphone-Based Activity Recognition Using Fixed-Point Arithmetic

In this paper we propose a novel energy efficient approach for the recog- nition of human activities using smartphones as wearable sensing devices, targeting assisted living applications such as remote patient activity monitoring for the disabled and the elderly. The method exploits fixed-point arithmetic to propose a modified multiclass Support Vector Machine (SVM) learning algorithm, allowing to better pre- serve the smartphone battery lifetime with respect to the conventional floating-point based formulation while maintaining comparable system accuracy levels. Experiments show comparative results between this approach and the traditional SVM in terms of recognition performance and battery consumption, highlighting the advantages of the proposed method.

Exploratory Data Analysis of Acceleration Signals to Select Light-Weight and Accurate Features for Real-Time Activity Recognition on Smartphones

Smartphone-based activity recognition (SP-AR) recognizes users' activities using the embedded accelerometer sensor. Only a small number of previous works can be classified as online systems, i.e., the whole process (pre-processing, feature extraction, and classification) is performed on the device. Most of these online systems use either a high sampling rate (SR) or long data-window (DW) to achieve high accuracy, resulting in short battery life or delayed system response, respectively. This paper introduces a real-time/online SP-AR system that solves this problem. Exploratory data analysis was performed on acceleration signals of 6 activities, collected from 30 subjects, to show that these signals are generated by an autoregressive (AR) process, and an accurate AR-model in this case can be built using a low SR (20 Hz) and a small DW (3 s). The high within class variance resulting from placing the phone at different positions was reduced using kernel discriminant analysis to achieve position-independent recognition. Neural networks were used as classifiers. Unlike previous works, true subject-independent evaluation was performed, where 10 new subjects evaluated the system at their homes for 1 week. The results show that our features outperformed three commonly used features by 40% in terms of accuracy for the given SR and DW.