Smartphone-based Human Activity Recognition Research Articles

DiabSense: early diagnosis of non-insulin-dependent diabetes mellitus using smartphone-based human activity recognition and diabetic retinopathy analysis with Graph Neural Network

Non-Insulin-Dependent Diabetes Mellitus (NIDDM) is a chronic health condition caused by high blood sugar levels, and if not treated early, it can lead to serious complications i.e. blindness. Human Activity Recognition (HAR) offers potential for early NIDDM diagnosis, emerging as a key application for HAR technology. This research introduces DiabSense, a state-of-the-art smartphone-dependent system for early staging of NIDDM. DiabSense incorporates HAR and Diabetic Retinopathy (DR) upon leveraging the power of two different Graph Neural Networks (GNN). HAR uses a comprehensive array of 23 human activities resembling Diabetes symptoms, and DR is a prevalent complication of NIDDM. Graph Attention Network (GAT) in HAR achieved 98.32% accuracy on sensor data, while Graph Convolutional Network (GCN) in the Aptos 2019 dataset scored 84.48%, surpassing other state-of-the-art models. The trained GCN analyzed retinal images of four experimental human subjects for DR report generation, and GAT generated their average duration of daily activities over 30 days. The daily activities in non-diabetic periods of diabetic patients were measured and compared with the daily activities of the experimental subjects, which helped generate risk factors. Fusing risk factors with DR conditions enabled early diagnosis recommendations for the experimental subjects despite the absence of any apparent symptoms. The comparison of DiabSense system outcome with clinical diagnosis reports in the experimental subjects was conducted using the A1C test. The test results confirmed the accurate assessment of early diagnosis requirements for experimental subjects by the system. Overall, DiabSense exhibits significant potential for ensuring early NIDDM treatment, improving millions of lives worldwide.

An ensemble maximal feature subset selection for smartphone based human activity recognition

Smartphone-based Human Activity Recognition (HAR) uses spatiotemporal time series data collected from a smartphone’s in-built accelerometer, gyroscope, and magnetometer sensor. Real-time HAR datasets such as UCI-HAR and UCI-HAPT extract time and frequency domain statistical features for activity recognition from the collected time series signals. Various Machine Learning techniques have been proposed in the perspective of feature selection through ensemble, traditional, hybrid, and meta-heuristic-based techniques to improve the recognition performance. Though the feature selection techniques have shown promising results, the computational time and selection of valid significant features for better classification performance still need to be included. Moreover, these techniques are highly parameter-dependent and critical to threshold values used for experimentation. This research proposes a novel method of parameter-free ensemble feature selection by combining traditional feature selection algorithms with the concept of association rule mining to obtain the Maximal Feature Subset (MFS), which is used for training and testing the classification model for HAR. Mathematical association rule mining is framed rather than the traditional one to select the associated frequent feature(s) from the feature set evolved by traditional techniques. Experimentation has been carried out on UCI-HAR and UCI-HAPT, and the proposed method has increased the classification efficiency by selecting significant features and reduces computation time significantly.

Smartphone-based human activity recognition using lightweight multiheaded temporal convolutional network

Smartphone-based human activity recognition using lightweight multiheaded temporal convolutional network

Edge-Fog-Cloud Data Analysis for eHealth-IoT

Thanks to advancements in artificial intelligence and the Internet of Things (IoT), eHealth is becoming an increasingly attractive area for researchers. However, different challenges arise when sensor-generated information is stored and analyzed using cloud computing. Latency, response time, and security are critical concerns that require attention. Fog and Edge Computing technologies have emerged in response to the requirement for resources near the network edge where data is collected, to minimize cloud challenges. This paper aims to assess the effectiveness of Machine Learning (ML) and Deep Learning (DL) techniques when executed in Edge or Fog nodes within the eHealth data. We compared the most efficient baseline techniques from the state-of-the-art on three eHealth datasets: Human Activity Recognition (HAR), University of Milano Bicocca Smartphone-based Human Activity Recognition (UniMiB SHAR), and MIT-BIH Arrhythmia. The experiment showed that for the HAR dataset, the Support Vector Machines (SVM) model was the best performer among the ML techniques, with low processing time and an accuracy of 96%. In comparison, the K-Nearest Neighbors (KNN) performed 94.43, and 96%, respectively, for SHAR and MIT-BIH datasets. Among the DL techniques, the Convolutional Neural Network with Fourier (CNNF) model performed the best, with accuracies of 94.49% and 98.72% for HAR and MIT-BIH. In comparison, CNN achieved 96.90% for the SHAR dataset.

The use of deep learning for smartphone-based human activity recognition.

The emerging field of digital phenotyping leverages the numerous sensors embedded in a smartphone to better understand its user's current psychological state and behavior, enabling improved health support systems for patients. As part of this work, a common task is to use the smartphone accelerometer to automatically recognize or classify the behavior of the user, known as human activity recognition (HAR). In this article, we present a deep learning method using the Resnet architecture to implement HAR using the popular UniMiB-SHAR public dataset, containing 11,771 measurement segments from 30 users ranging in age between 18 and 60 years. We present a unified deep learning approach based on a Resnet architecture that consistently exceeds the state-of-the-art accuracy and F1-score across all classification tasks and evaluation methods mentioned in the literature. The most notable increase we disclose regards the leave-one-subject-out evaluation, known as the most rigorous evaluation method, where we push the state-of-the-art accuracy from 78.24 to 80.09% and the F1-score from 78.40 to 79.36%. For such results, we resorted to deep learning techniques, such as hyper-parameter tuning, label smoothing, and dropout, which helped regularize the Resnet training and reduced overfitting. We discuss how our approach could easily be adapted to perform HAR in real-time and discuss future research directions.

Coupling of Dimensionality Reduction and Stacking Ensemble Learning for Smartphone-Based Human Activity Recognition

Human activity recognition (HAR) plays a vital role in the field of ambient assisted living (AAL) for the welfare of the elders who live alone in the home. AAL provides service through ambient sensors, vision systems, smartphone devices, and wearable sensors. Smartphone devices are familiar, portable, cost-effective, and make the process of monitoring easier. Various research works have proposed smartphone-based HAR systems to recognize basic and complex activities. However, the results are not satisfactory for the case of postural transitions such as stand-to-sit, sit-to-sleep, etc. To improve the recognition rate, this paper couples principal component analysis with stacking ensemble learning for dimensionality reduction and classification respectively. Extensive experimentation of UCI repository datasets such as UCI-HAR has been performed and the performances are measured using familiar metrics such as accuracy, precision, and recall.

An Integration of feature extraction and Guided Regularized Random Forest feature selection for Smartphone based Human Activity Recognition

Human Activity Recognition (HAR) is an eminent area of research due to its extensive scope of applications in remote health monitoring, sports, smart home, and many more. Smartphone-based HAR systems use sensor data to infer human physical activities. Extraction of pertinent and essential features to identify human activities is a crucial but challenging task. Researchers continuously endeavor to extract pertinent and non-redundant features without compromising the classification accuracy. Smartphone sensor data generates high dimensional feature sets to recognize human physical activities. This work aims to build an efficient HAR model that extracts the most relevant features from the 3-axial accelerometer and gyroscope signal data and enhances the HAR system’s classification accuracy without data loss using time–frequency domain features. After feature extraction, we apply the Guided Regularized Random Forest (GRRF) feature selection method to determine the most pertinent and non-redundant features to reduce the time to recognize the human activities efficiently. After selecting the most relevant features, Support Vector Machine (SVM), Naive Bayes (NB), Random Forest (RF), and Decision Tree (DT) are used to identify various human physical activities. Using two different datasets, we investigate that GRRF selects relevant feature sets compared to two benchmark feature selection methods such as Relief-F and GRF, without compromising the recognition accuracy. This integration model with GRRF shows improved performance using all methods mentioned above. Our proposed strategy achieves higher accuracy values of 99.10% and 99.30% for SVM using two different datasets.

Guided regularized random forest feature selection for smartphone based human activity recognition.

Human activity recognition (HAR) is an eminent area of research due to its extensive scope of applications in remote health monitoring, sports, smart home, and many more. Smartphone-based HAR systems use high-dimensional sensor data to infer human physical activities. Researchers continuously endeavor to select pertinent and non-redundant features without compromising the classification accuracy. In this work, our aim is to build an efficient HAR model that not only extracts the most relevant features from the 3-axial accelerometer and gyroscope signal data but also enhances the classification accuracy of the HAR system, without data loss using time-frequency domain features. We propose a feature selection method based on guided regularized random forest (GRRF) to determine the most pertinent and non-redundant features to reduce the time to recognize the human activities efficiently. After selecting the most relevant features, a support vector machine (SVM) is used to identify various human physical activities. The UCI public dataset and a self-collected dataset are used to assess the generalization capability and performance of the proposed feature selection method. Eventually, the accuracy reached 99.10% and 99.30% on the self-collected and UCI HAR datasets, respectively.

ConvAE-LSTM: Convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition

The self-regulated recognition of human activities from time-series smartphone sensor data is a growing research area in smart and intelligent health care. Deep learning (DL) approaches have exhibited improvements over traditional machine learning (ML) models in various domains, including human activity recognition (HAR). Several issues are involved with traditional ML approaches; these include handcrafted feature extraction, which is a tedious and complex task involving expert domain knowledge, and the use of a separate dimensionality reduction module to overcome overfitting problems and hence provide model generalization. In this article, we propose a DL-based approach for activity recognition with smartphone sensor data, i.e., accelerometer and gyroscope data. Convolutional neural networks (CNNs), autoencoders (AEs), and long short-term memory (LSTM) possess complementary modeling capabilities, as CNNs are good at automatic feature extraction, AEs are used for dimensionality reduction and LSTMs are adept at temporal modeling. In this study, we take advantage of the complementarity of CNNs, AEs, and LSTMs by combining them into a unified architecture. We explore the proposed architecture, namely, “ConvAE-LSTM”, on four different standard public datasets (WISDM, UCI, PAMAP2, and OPPORTUNITY). The experimental results indicate that our novel approach is practical and provides relative smartphone-based HAR solution performance improvements in terms of computational time, accuracy, F1-score, precision, and recall over existing state-of-the-art methods.

DCNN based human activity recognition framework with depth vision guiding

The smartphone-based human activity recognition (HAR) provides abundant healthcare guidance for telemedicine and clinic treatment. The continually increasing daily activities cause many difficulties for recognition and labeling. Although multimodal data fusion and artificial intelligence (AI) techniques can solve these problems, big data collection and labeling are still heavy. This paper proposes a remarkable depth data-guided framework based on smartphones for complex HAR and automatic labeling. The hardware platform is utilized to collect information of depth vision from the Microsoft Kinect camera and Inertial Measurement Unit (IMU) signals from the smartphone simultaneously. This framework consists of five clustering layers and deep learning (DL) based classification model to identify 12 complex daily activities. The results show that the hierarchical k-medoids (Hk-medoids) algorithm obtains the labels with high accuracy (93.89%). Furthermore, the performance evaluation of the DCNN model is evaluated better by comparing it with other machine learning (ML) and DL methods.

Performance Analysis of Machine Learning Algorithms for Smartphone-Based Human Activity Recognition

Performance Analysis of Machine Learning Algorithms for Smartphone-Based Human Activity Recognition

PSDRNN: An Efficient and Effective HAR Scheme Based on Feature Extraction and Deep Learning

Recently, in this article, due to pervasive usability, smartphone-based human activity recognition (HAR) has witnessed significant development in smart health. Meanwhile, deep recurrent neural network (DRNN) shows a strong ability to automatically extract features from the time-series data, and therefore, DRNN-based HAR schemes have achieved more effective recognition (i.e., recognition accuracy) than those adopting the traditional machine learning. However, the efficiency in training and recognition (in terms of running time) has not fully taken into account, especially for resource-constraint smartphones. To solve the above issue, we propose the PSDRNN and tri-PSDRNN schemes that employ the explicit feature extraction before DRNN. Specifically, considering that the power spectral density (PSD) feature can capture the frequency characteristics and meanwhile retain the successive time characteristics of data gathered from smartphone accelerometer, PSD feature vectors are, respectively, extracted from linear accelerations and triaxle accelerations and explicitly used as the input to the following DRNN classification model. Thorough experiments based on a real dataset demonstrate that the PSDRNN can achieve the comparable effectiveness as the xyz -DRNN (the most accurate DRNN-based HAR scheme only using acceleration data), and the average recognition and training time were reduced by 56% and 80%, respectively. Moreover, tri-PSDRNN advantages over the xyz -DRNN in terms of recognition accuracy, and the running time is still lower than the xyz -DRNN. Besides, our proposed PSDRNN scheme achieved superiority in the recognition of complex transition activities.

Multi-task Self-Supervised Learning for Human Activity Detection

Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Specifically, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. Therefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classification result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning settings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs significantly better than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by attaining a kappa score between 0.7 - 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a different data source. Self-supervision drastically reduces the requirement of labeled activity data, effectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas.

Efficient FPGA Implementation of Multilayer Perceptron for Real-Time Human Activity Classification

The smartphone-based human activity recognition (HAR) systems are not capable to deliver high-end performance for challenging applications. We propose a dedicated hardware-based HAR system for smart military wearables, which uses a multilayer perceptron (MLP) algorithm to perform activity classification. To achieve the flexible and efficient hardware design, the inherent MLP architecture with parallel computation is implemented on FPGA. The system performance has been evaluated using the UCI human activity dataset with 7767 feature samples of 20 subjects. The three combinations of a dataset are trained, validated, and tested on ten different MLP models with distinct topologies. The MLP design with the 7-6-5 topology is finalized from the classification accuracy and cross entropy performance. The five versions of the final MLP design (7-6-5) with different data precision are implemented on FPGA. The analysis shows that the MLP designed with 16-bit fixed-point data precision is the most efficient MLP implementation in the context of classification accuracy, resource utilization, and power consumption. The proposed MLP design requires only 270 ns for classification and consumes 120 mW of power. The recognition accuracy and hardware results performance achieved are better than many of the recently reported works.

Smartphone-Based Human Activity Recognition Using Bagging and Boosting

In today’s healthcare applications, the use of mobile technologies brings together physicians and patients for intelligent and automatic monitoring of daily clinical activities, remote life assistants, and preventive care, especially for the elderly and those under medical control. As smartphones become an important part of our everyday life, they are ever more employed in human activities recognition (HAR) including the monitoring of personal health care and wellbeing. However, HAR is complex and it is important to use the best technology and learn about human activity using machine learning. The purpose of this paper is to develop a HAR system based on the smartphone sensors’ data using Bagging and Adaboost ensemble classifiers. The experimental results for the HAR data have been evaluated after performing different data mining techniques. For each subject, the total classification accuracy, the F-measure, and the ROC area were calculated. Adaboost ensemble classifiers algorithm improved significantly the performance of smartphone-based HAR, combined with SVM, it reached 97.44% accuracy compared to the rest of the classifiers. The proposed algorithm of Adaboost SVM can lead to an accurate HAR for elderly and disabled patients who need continuous care as well as it is a tool that supports the decisions of all medical practitioners.

A Robust Deep Learning Approach for Position-Independent Smartphone-Based Human Activity Recognition.

Recently, modern smartphones equipped with a variety of embedded-sensors, such as accelerometers and gyroscopes, have been used as an alternative platform for human activity recognition (HAR), since they are cost-effective, unobtrusive and they facilitate real-time applications. However, the majority of the related works have proposed a position-dependent HAR, i.e., the target subject has to fix the smartphone in a pre-defined position. Few studies have tackled the problem of position-independent HAR. They have tackled the problem either using handcrafted features that are less influenced by the position of the smartphone or by building a position-aware HAR. The performance of these studies still needs more improvement to produce a reliable smartphone-based HAR. Thus, in this paper, we propose a deep convolution neural network model that provides a robust position-independent HAR system. We build and evaluate the performance of the proposed model using the RealWorld HAR public dataset. We find that our deep learning proposed model increases the overall performance compared to the state-of-the-art traditional machine learning method from 84% to 88% for position-independent HAR. In addition, the position detection performance of our model improves superiorly from 89% to 98%. Finally, the recognition time of the proposed model is evaluated in order to validate the applicability of the model for real-time applications.

A robust convolutional neural network for online smartphone-based human activity recognition

The online smartphone-based human activity recognition (HAR) has a variety of applications such as fitness tracking, healthcare… etc. Currently, the signals generated from smartphone-embedded sensors are used for HAR systems. The smartphone-embedded sensors are utilized in order to provide an unobtrusive platform for HAR. In this paper, we propose a deep convolution neural network (CNN) model that provides an effective and efficient smartphone-based HAR system. For automatic local features extraction from the raw time-series data, we use the CNN while simple time-domain statistical features are used to extract more distinguishable features. Furthermore, we explore the impact of a novel data augmentation on the recognition accuracy of the proposed model. The performance of the proposed method is evaluated using two public data sets (UCI and WISDM) which are collected using smartphones

Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone.

Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of such datasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigate the robustness of common supervised classification approaches under instances of noisy data. We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generated labels achieved an 84% f-score (using Neural Networks and Random Forests); however, results also demonstrated how the presence of label noise could lower the f-score up to 64–74% depending on the classification approach (Nearest Centroid and Multi-Class Support Vector Machine).

Learning Hardware-Friendly Classifiers Through Algorithmic Stability

Most state-of-the-art machine-learning (ML) algorithms do not consider the computational constraints of implementing the learned model on embedded devices. These constraints are, for example, the limited depth of the arithmetic unit, the memory availability, or the battery capacity. We propose a new learning framework, the Algorithmic Risk Minimization (ARM), which relies on Algorithmic-Stability, and includes these constraints inside the learning process itself. ARM allows one to train advanced resource-sparing ML models and to efficiently deploy them on smart embedded systems. Finally, we show the advantages of our proposal on a smartphone-based Human Activity Recognition application by comparing it to a conventional ML approach.

Evaluation of a smartphone human activity recognition application with able-bodied and stroke participants.

BackgroundMobile health monitoring using wearable sensors is a growing area of interest. As the world’s population ages and locomotor capabilities decrease, the ability to report on a person’s mobility activities outside a hospital setting becomes a valuable tool for clinical decision-making and evaluating healthcare interventions. Smartphones are omnipresent in society and offer convenient and suitable sensors for mobility monitoring applications. To enhance our understanding of human activity recognition (HAR) system performance for able-bodied and populations with gait deviations, this research evaluated a custom smartphone-based HAR classifier on fifteen able-bodied participants and fifteen participants who suffered a stroke.MethodsParticipants performed a consecutive series of mobility tasks and daily living activities while wearing a BlackBerry Z10 smartphone on their waist to collect accelerometer and gyroscope data. Five features were derived from the sensor data and used to classify participant activities (decision tree). Sensitivity, specificity and F-scores were calculated to evaluate HAR classifier performance.ResultsThe classifier performed well for both populations when differentiating mobile from immobile states (F-score > 94 %). As activity recognition complexity increased, HAR system sensitivity and specificity decreased for the stroke population, particularly when using information derived from participant posture to make classification decisions.ConclusionsHuman activity recognition using a smartphone based system can be accomplished for both able-bodied and stroke populations; however, an increase in activity classification complexity leads to a decrease in HAR performance with a stroke population. The study results can be used to guide smartphone HAR system development for populations with differing movement characteristics.