Sensor-based Human Activity Recognition Research Articles

A general framework for sensor-based human activity recognition

Today's wearable devices like smartphones, smartwatches and intelligent glasses collect a large amount of data from their built-in sensors like accelerometers and gyroscopes. These data can be used to identify a person's current activity and in turn can be utilised for applications in the field of personal fitness assistants or elderly care. However, developing such systems is subject to certain restrictions: (i) since more and more new sensors will be available in the future, activity recognition systems should be able to integrate these new sensors with a small amount of manual effort and (ii) such systems should avoid high acquisition costs for computational power.We propose a general framework that achieves an effective data integration based on the following two characteristics: Firstly, a smartphone is used to gather and temporally store data from different sensors and transfer these data to a central server. Thus, various sensors can be integrated into the system as long as they have programming interfaces to communicate with the smartphone. The second characteristic is a codebook-based feature learning approach that can encode data from each sensor into an effective feature vector only by tuning a few intuitive parameters. In the experiments, the framework is realised as a real-time activity recognition system that integrates eight sensors from a smartphone, smartwatch and smartglasses, and its effectiveness is validated from different perspectives such as accuracies, sensor combinations and sampling rates.

On the Generality of Codebook Approach for Sensor-Based Human Activity Recognition

With the recent spread of mobile devices equipped with different sensors, it is possible to continuously recognise and monitor activities in daily life. This sensor-based human activity recognition is formulated as sequence classification to categorise sequences of sensor values into appropriate activity classes. One crucial problem is how to model features that can precisely represent characteristics of each sequence and lead to accurate recognition. It is laborious and/or difficult to hand-craft such features based on prior knowledge and manual investigation about sensor data. To overcome this, we focus on a feature learning approach that extracts useful features from a large amount of data. In particular, we adopt a simple but effective one, called codebook approach, which groups numerous subsequences collected from sequences into clusters. Each cluster centre is called a codeword and represents a statistically distinctive subsequence. Then, a sequence is encoded as a feature expressing the distribution of codewords. The extensive experiments on different recognition tasks for physical, mental and eye-based activities validate the effectiveness, generality and usability of the codebook approach.

Training Classifiers with Shadow Features for Sensor-Based Human Activity Recognition.

In this paper, a novel training/testing process for building/using a classification model based on human activity recognition (HAR) is proposed. Traditionally, HAR has been accomplished by a classifier that learns the activities of a person by training with skeletal data obtained from a motion sensor, such as Microsoft Kinect. These skeletal data are the spatial coordinates (x, y, z) of different parts of the human body. The numeric information forms time series, temporal records of movement sequences that can be used for training a classifier. In addition to the spatial features that describe current positions in the skeletal data, new features called ‘shadow features’ are used to improve the supervised learning efficacy of the classifier. Shadow features are inferred from the dynamics of body movements, and thereby modelling the underlying momentum of the performed activities. They provide extra dimensions of information for characterising activities in the classification process, and thereby significantly improve the classification accuracy. Two cases of HAR are tested using a classification model trained with shadow features: one is by using wearable sensor and the other is by a Kinect-based remote sensor. Our experiments can demonstrate the advantages of the new method, which will have an impact on human activity detection research.

Wearable Sensor-Based Human Activity Recognition Method with Multi-Features Extracted from Hilbert-Huang Transform.

Wearable sensors-based human activity recognition introduces many useful applications and services in health care, rehabilitation training, elderly monitoring and many other areas of human interaction. Existing works in this field mainly focus on recognizing activities by using traditional features extracted from Fourier transform (FT) or wavelet transform (WT). However, these signal processing approaches are suitable for a linear signal but not for a nonlinear signal. In this paper, we investigate the characteristics of the Hilbert-Huang transform (HHT) for dealing with activity data with properties such as nonlinearity and non-stationarity. A multi-features extraction method based on HHT is then proposed to improve the effect of activity recognition. The extracted multi-features include instantaneous amplitude (IA) and instantaneous frequency (IF) by means of empirical mode decomposition (EMD), as well as instantaneous energy density (IE) and marginal spectrum (MS) derived from Hilbert spectral analysis. Experimental studies are performed to verify the proposed approach by using the PAMAP2 dataset from the University of California, Irvine for wearable sensors-based activity recognition. Moreover, the effect of combining multi-features vs. a single-feature are investigated and discussed in the scenario of a dependent subject. The experimental results show that multi-features combination can further improve the performance measures. Finally, we test the effect of multi-features combination in the scenario of an independent subject. Our experimental results show that we achieve four performance indexes: recall, precision, F-measure, and accuracy to 0.9337, 0.9417, 0.9353, and 0.9377 respectively, which are all better than the achievements of related works.

A Triaxial Accelerometer-Based Human Activity Recognition via EEMD-Based Features and Game-Theory-Based Feature Selection

In recent years, sensor-based human activity recognition has attracted lots of studies. This paper presents a single wearable triaxial accelerometer-based human activity recognition system, which can be used in the real life of activity monitoring. The sensor is attached around different parts of the body: waist and left ankle, respectively. In order to improve the accuracy and reduce the computational complexity, the ensemble empirical mode decomposition (EEMD)-based features and the feature selection (FS) method are introduced, respectively. Considering the feature interaction, a game theory-based FS method is proposed to evaluate the features. Relevant and distinguished features that are robust to the placement of sensors are selected. In the experiment, the data acquired from the two different parts of the body, waist and ankle, are utilized to evaluate the proposed FS method. To verify the effectiveness of the proposed method, k-nearst neighbor and support vector machine are used to recognize the human activities from waist and ankle. Experiment results demonstrate the effectiveness of the introduced EEMD-based features for human activity recognition. Compared with the representative FS methods, including Relief-F and minimum-redundancy maximum-relevance, the proposed FS approach selects fewer features and provides higher accuracy. The results also show that the triaxial accelerometer around the waist produces optimal results.

Complex activity recognition using time series pattern dictionary learned from ubiquitous sensors

Sensor-based human activity recognition has become an important research field within pervasive and ubiquitous computing. Techniques for recognizing atomic activities such as gestures or actions are mature for now, but complex activity recognition still remains a challenging issue. In this paper, we address the problem of complex activity recognition using time series extracted from multiple sensors. We first build a dictionary of time series patterns, called shapelets, to represent atomic activities, then present three shapelet-based models to recognize sequential, concurrent, and generic complex activities. We use the datasets collected from three different labs to evaluate our shapelet-based approach and the results show that our approach can handle complex activity recognition effectively. Our experimental results also show that the shapelet-based approach outperforms other competing approaches in terms of recognition accuracy and system usage.

Sensor-based human activity recognition system with a multilayered model using time series shapelets

Human activity recognition can be exploited to benefit ubiquitous applications using sensors. Current research on sensor-based activity recognition is mainly using data-driven or knowledge-driven approaches. In terms of complex activity recognition, most data-driven approaches suffer from portability, extensibility and interpretability problems, whilst knowledge-driven approaches are often weak in handling intricate temporal data. To address these issues, we exploit time series shapelets for complex human activity recognition. In this paper, we first describe the association between activity and time series transformed from sensor data. Then, we present a recursively defined multilayered activity model to represent four types of activities and employ a shapelet-based framework to recognize various activities represented in the model. A prototype system was implemented to evaluate our approach on two public datasets. We also conducted two real-world case studies for system evaluation: daily living activity recognition and basketball play activity recognition. The experimental results show that our approach is capable of handling complex activity effectively. The results are interpretable and accurate, and our approach is fast and energy-efficient in real-time.

A Model for Abnormal Activity Recognition and Alert Generation System for Elderly Care by Hidden Conditional Random Fields Using R-Transform and Generalized Discriminant Analysis Features

The growing population of elderly people living alone increases the need for automatic healthcare monitoring systems for elderly care. Automatic vision sensor-based systems are increasingly used for human activity recognition (HAR) in recent years. This study presents an improved model, tested using actors, of a sensor-based HAR system to recognize daily life activities of elderly people at home and generate an alert in case of abnormal HAR. Datasets consisting of six abnormal activities (falling backward, falling forward, falling rightward, falling leftward, chest pain, and fainting) and four normal activities (walking, rushing, sitting down, and standing up) are generated from different view angles (90°, -90°, 45°, -45°). Feature extraction and dimensions reduction are performed by R-transform followed by generalized discriminant analysis (GDA) methods. R-transform extracts symmetric, scale, and translation-invariant features from the sequences of activities. GDA increases the discrimination between different classes of highly similar activities. Silhouette sequences are quantified by the Linde-Buzo-Gray algorithm and recognized by hidden conditional random fields. Experimental results provide an average recognition rate of 94.2% for abnormal activities and 92.7% for normal activities. The recognition rate for the highly similar activities from different view angles shows the flexibility and efficacy of the proposed abnormal HAR and alert generation system for elderly care.

Wearable sensor-based human activity recognition from environmental background sounds

Understanding individual’s activities, social interaction, and group dynamics of a certain society is one of fundamental problems that the social and community intelligence (SCI) research faces. Environmental background sound is a rich information source for identifying individual and social behaviors. Therefore, many power-aware wearable devices with sound recognition function are widely used to trace and understand human activities. The design of these sound recognition algorithms has two major challenges: limited computation resources and a strict power consumption requirement. In this paper, a new method for recognizing environmental background sounds with a power-aware wearable sensor is presented. By employing a novel low calculation one-dimensional (1-D) Haar-like sound feature with hidden Markov model (HMM) classification, this method can achieve high recognition accuracy while still meeting the wearable sensor’s power requirement. Our experimental results indicate an average recognition accuracy of 96.9 % has been achieved when testing with 22 typical environmental sounds related to personal and social activities. It outperforms other commonly used sound recognition algorithms in terms of both accuracy and power consumption. This is very helpful and promising for future integration with other sensor(s) to provide more trustworthy activity recognition results for the SCI system.

An Incremental Learning Method Based on Probabilistic Neural Networks and Adjustable Fuzzy Clustering for Human Activity Recognition by Using Wearable Sensors

Human activity recognition by using wearable sensors has gained tremendous interest in recent years among a range of health-related areas. To automatically recognize various human activities from wearable sensor data, many classification methods have been tried in prior studies, but most of them lack the incremental learning abilities. In this study, an incremental learning method is proposed for sensor-based human activity recognition. The proposed method is designed based on probabilistic neural networks and an adjustable fuzzy clustering algorithm. The proposed method may achieve the following features. 1) It can easily learn additional information from new training data to improve the recognition accuracy. 2) It can freely add new activities to be detected, as well as remove existing activities. 3) The updating process from new training data does not require previously used training data. An experiment was performed to collect realistic wearable sensor data from a range of activities of daily life. The experimental results showed that the proposed method achieved a good tradeoff between incremental learning ability and the recognition accuracy. The experimental results from comparison with other classification methods demonstrated the effectiveness of the proposed method further.

A Pattern Mining Approach to Sensor-Based Human Activity Recognition

Recognizing human activities from sensor readings has recently attracted much research interest in pervasive computing due to its potential in many applications, such as assistive living and healthcare. This task is particularly challenging because human activities are often performed in not only a simple (i.e., sequential), but also a complex (i.e., interleaved or concurrent) manner in real life. Little work has been done in addressing complex issues in such a situation. The existing models of interleaved and concurrent activities are typically learning-based. Such models lack of flexibility in real life because activities can be interleaved and performed concurrently in many different ways. In this paper, we propose a novel pattern mining approach to recognize sequential, interleaved, and concurrent activities in a unified framework. We exploit Emerging Pattern-a discriminative pattern that describes significant changes between classes of data-to identify sensor features for classifying activities. Different from existing learning-based approaches which require different training data sets for building activity models, our activity models are built upon the sequential activity trace only and can be applied to recognize both simple and complex activities. We conduct our empirical studies by collecting real-world traces, evaluating the performance of our algorithm, and comparing our algorithm with static and temporal models. Our results demonstrate that, with a time slice of 15 seconds, we achieve an accuracy of 90.96 percent for sequential activity, 88.1 percent for interleaved activity, and 82.53 percent for concurrent activity.

Sensor-Based Abnormal Human-Activity Detection

With the availability of affordable sensors and sensor networks, sensor-based human activity recognition has attracted much attention in artificial intelligence and ubiquitous computing. In this paper, we present a novel two-phase approach for detecting abnormal activities based on wireless sensors attached to a human body. Detecting abnormal activities is a particular important task in security monitoring and healthcare applications of sensor networks, among many others. Traditional approaches to this problem suffer from a high false positive rate, particularly when the collected sensor data are biased towards normal data while the abnormal events are rare. Therefore, there is a lack of training data for many traditional data mining methods to be applied. To solve this problem, our approach first employs a one-class support vector machine (SVM) that is trained on commonly available normal activities, which filters out the activities that have a very high probability of being normal. We then derive abnormal activity models from a general normal model via a kernel nonlinear regression (KNLR) to reduce false positive rate in an unsupervised manner. We show that our approach provides a good tradeoff between abnormality detection rate and false alarm rate, and allows abnormal activity models to be automatically derived without the need to explicitly label the abnormal training data, which are scarce. We demonstrate the effectiveness of our approach using real data collected from a sensor network that is deployed in a realistic setting.