Human Activity Tracking Research Articles

Recognizing salat activity using deep learning models via smartwatch sensors

In this study, we focus on human activity recognition, particularly aiming to distinguish the activity of praying (salat) from other daily activities. To achieve this goal, we have created a new dataset named HAR-P (Human activity recognition for Praying), which includes eight different activities: walking, running, sitting, standing, walking upstairs, walking downstairs, typing with a keyboard, and praying (salat). The HAR-P dataset was collected from 50 male individuals, who wore smartwatches on their dominant wrists. We compare the activity classification performance using three state-of-the-art algorithms from the literature: Long Short-Term Memory, Convolutional Long Short-Term Memory, and Convolutional Neural Network—Long Short-Term Memory. To assess the influence of sensors, data from accelerometer, gyroscope, linear acceleration sensor, and magnetic field sensor were utilized. The impact of individual sensor data as well as combinations thereof was investigated. The highest classification accuracy within single sensor groups, reaching 95.7%, was achieved using the accelerometer data with the Convolutional Long Short-Term Memory method. Combining two sensor groups resulted in an increase in accuracy of up to 9%. The highest accuracy of 96.4% was obtained by utilizing three sensor groups together with the Convolutional Neural Network—Long Short-Term Memory method. Furthermore, the evaluation of sensor and model performance was conducted using the stratified k-fold cross-validation method with 5-folds. These findings contribute significantly to evaluating the performance of sensor combinations and different algorithms in activity classification. This study may provide an effective foundation for the automatic recognition and tracking of human activities and offer an applicable model, particularly for the recognition of religious practices such as praying.

Human activity tracking using wearable sensors in loose-fitting clothes: survey

Human activity tracking using wearable sensors in loose-fitting clothes: survey

Loose Garments Effects on Wearable Sensors in Human Activity Tracking Applications

Loose Garments Effects on Wearable Sensors in Human Activity Tracking Applications

A new approach for physical human activity recognition from sensor signals based on motif patterns and long-short term memory

Numerous studies have been carried out in recent years on the recognition, tracking, and discrimination of human activities. Automatic recognition of physical activities is often referred to as human activity recognition (HAR). There are generally vision-based and sensor-based approaches for activity recognition. The computer vision-based approach generally works well in laboratory conditions, but it can fail in real-world problems due to clutter, variable light intensity, and contrast. Sensor-based HAR systems are realized by continuously monitoring and analyzing physiological signals measured from heterogeneous sensors connected to the person's body. In this study, the Motif Patterns (MP) approach, which extracts features from sensor signals, is proposed for HAR. The success of the HAR systems depends on the effectiveness of the features extracted from the signals. The LSTM network is a special kind of recurrent neural network that has been used to make very successful predictions on time series data where long-term dependencies are. The LSTM network type offers a successful solution approach to solving long-term dependencies problems such as human activity recognition. The classification process was carried out with Long-Short Term Memory (LSTM) using MP features extracted from accelerometer, gyroscope, and magnetometer sensor signals. A large dataset of 9120 signals was used to test the proposed approach. A high success rate of 98.42 % was achieved with the proposed MP + LSTM method. As a result, it has been seen that the proposed approach has been obtained with a high success rate for HAR using sensor signals.

Ergonomics evaluation of lawn mower operator's working posture using JACK software and kinect interface.

Nowadays, real-time motion tracking devices are widely used for ergonomic assessment of several manual quotidian activities. The real-time tracking of human activities makes it easier to observe the exposure of work-related musculoskeletal disorders (WMSDs) in the human body. This study aims to determine the suitability of a real-time motion tracking device (Kinect v1 interfaced with a commercial ergonomic assessment software, JACK) for real-time ergonomic evaluation of the strenuous operation of the manual lawn mower. The lawn mower operators perform various strenuous activities while operating the manual lawn mower for long intervals of time, which causes WMSDs in the entire body of the operators. These working operators' activities have been captured using Kinect v1 interfaced with JACK, to address the ergonomic issues responsible for the whole-body WMSDs. The forces acting on the lower back, Rapid Upper Limb Assessment score and static strength have been predicted using JACK. This study proves the exposure of the operators towards the whole-body WMSDs while operating the manual lawn mower. The findings provide a quick and straightforward approach for performing the real-time ergonomic evaluation of any operation, which can help the industrial staff estimate the risk of level WMSDs.

Modeling Two-Person Segmentation and Locomotion for Stereoscopic Action Identification: A Sustainable Video Surveillance System

Due to the constantly increasing demand for automatic tracking and recognition systems, there is a need for more proficient, intelligent and sustainable human activity tracking. The main purpose of this study is to develop an accurate and sustainable human action tracking system that is capable of error-free identification of human movements irrespective of the environment in which those actions are performed. Therefore, in this paper we propose a stereoscopic Human Action Recognition (HAR) system based on the fusion of RGB (red, green, blue) and depth sensors. These sensors give an extra depth of information which enables the three-dimensional (3D) tracking of each and every movement performed by humans. Human actions are tracked according to four features, namely, (1) geodesic distance; (2) 3D Cartesian-plane features; (3) joints Motion Capture (MOCAP) features and (4) way-points trajectory generation. In order to represent these features in an optimized form, Particle Swarm Optimization (PSO) is applied. After optimization, a neuro-fuzzy classifier is used for classification and recognition. Extensive experimentation is performed on three challenging datasets: A Nanyang Technological University (NTU) RGB+D dataset; a UoL (University of Lincoln) 3D social activity dataset and a Collective Activity Dataset (CAD). Evaluation experiments on the proposed system proved that a fusion of vision sensors along with our unique features is an efficient approach towards developing a robust HAR system, having achieved a mean accuracy of 93.5% with the NTU RGB+D dataset, 92.2% with the UoL dataset and 89.6% with the Collective Activity dataset. The developed system can play a significant role in many computer vision-based applications, such as intelligent homes, offices and hospitals, and surveillance systems.

Urban Space Insights Extraction Using Acoustic Histogram Information

Urban data mining can be identified as a highly potential area that can enhance the smart city services toward better sustainable development especially in the urban residential activity tracking. While existing human activity tracking systems have demonstrated the capability to unveil the hidden aspects of citizens' behavior, they often come with a high implementation cost and require a large communication bandwidth. In this article, we study the implementation of low-cost analog sound sensors to detect outdoor activities and estimate the raining period in an urban residential area. The analog sound sensors are transmitted to the cloud every 5 min in histogram format, which consists of sound data sampled every 100 ms (10 Hz). We then use wavelet transformation and principal component analysis to generate a more robust and consistent feature set from the histogram. After that, we performed unsupervised clustering and attempt to understand the individual characteristics of each cluster to identify outdoor residential activities. In addition, on-site validation has been conducted to show the effectiveness of our approach.

On the Challenges and Potential of Using Barometric Sensors to Track Human Activity.

Barometers are among the oldest engineered sensors. Historically, they have been primarily used either as environmental sensors to measure the atmospheric pressure for weather forecasts or as altimeters for aircrafts. With the advent of microelectromechanical system (MEMS)-based barometers and their systematic embedding in smartphones and wearable devices, a vast breadth of new applications for the use of barometers has emerged. For instance, it is now possible to use barometers in conjunction with other sensors to track and identify a wide range of human activity classes. However, the effectiveness of barometers in the growing field of human activity recognition critically hinges on our understanding of the numerous factors affecting the atmospheric pressure, as well as on the properties of the sensor itself—sensitivity, accuracy, variability, etc. This review article thoroughly details all these factors and presents a comprehensive report of the numerous studies dealing with one or more of these factors in the particular framework of human activity tracking and recognition. In addition, we specifically collected some experimental data to illustrate the effects of these factors, which we observed to be in good agreement with the findings in the literature. We conclude this review with some suggestions on some possible future uses of barometric sensors for the specific purpose of tracking human activities.

Acoustic-based sensing and applications: A survey

With advancements of wireless and sensing technologies, recent studies have demonstrated technical feasibility and effectiveness of using acoustic signals for sensing. In the past decades, low-cost audio infrastructures are widely-deployed and integrated into mobile and Internet of Things (IoT) devices to facilitate a broad array of applications including human activity recognition, tracking, localization, and security monitoring. The technology underpinning these applications lies in the analysis of propagation properties of acoustic signals (e.g., reflection, diffraction, and scattering) when they encounter human bodies. As a result, these applications serve as the foundation to support various daily functionalities such as safety protection, smart healthcare, and smart appliance interaction. The already-existing acoustic infrastructure could also complement RF-based localization and other approaches based on short-range communications such as Near-Field Communication (NFC) and Quick Response (QR) code. In this paper, we provide a comprehensive review on acoustic-based sensing in terms of hardware infrastructure, technical approaches, and its broad applications. First we describe different methodologies and techniques of using acoustic signals for sensing including Time-of-Arrival (ToA), Frequency Modulated Continuous Wave (FMCW), Time-Difference-of-Arrival (TDoA), and Channel Impulse Response (CIR). Then we classify various applications and compare different acoustic-based sensing approaches: in recognition and tracking, we review daily activity recognition, human health and behavioral monitoring hand gesture recognition, hand movement tracking, and speech recognition; in localization and navigation, we discuss ranging and direction finding, indoor and outdoor localization, and floor map construction; in security and privacy, we survey user authentication, keystroke snooping attacks, audio adversarial attacks, acoustic vibration attacks, and privacy protection schemes. Lastly we discuss future research directions and limitations of the acoustic-based sensing.

A comparison on visual prediction models for MAMO (multi activity-multi object) recognition using deep learning

Multi activity-multi object recognition (MAMO) is a challenging task in visual systems for monitoring, recognizing and alerting in various public places, such as universities, hospitals and airports. While both academic and commercial researchers are aiming towards automatic tracking of human activities in intelligent video surveillance using deep learning frameworks. This is required for many real time applications to detect unusual/suspicious activities like tracking of suspicious behaviour in crime events etc. The primary purpose of this paper is to render a multi class activity prediction in individuals as well as groups from video sequences by using the state-of-the-art object detector You Look only Once (YOLOv3). By optimum utilization of the geographical information of cameras and YOLO object detection framework, a Deep Landmark model recognize a simple to complex human actions on gray scale to RGB image frames of video sequences. This model is tested and compared with various benchmark datasets and found to be the most precise model for detecting human activities in video streams. Upon analysing the experimental results, it has been observed that the proposed method shows superior performance as well as high accuracy.

Energy per Operation Optimization for Energy-Harvesting Wearable IoT Devices.

Wearable internet of things (IoT) devices can enable a variety of biomedical applications, such as gesture recognition, health monitoring, and human activity tracking. Size and weight constraints limit the battery capacity, which leads to frequent charging requirements and user dissatisfaction. Minimizing the energy consumption not only alleviates this problem, but also paves the way for self-powered devices that operate on harvested energy. This paper considers an energy-optimal gesture recognition application that runs on energy-harvesting devices. We first formulate an optimization problem for maximizing the number of recognized gestures when energy budget and accuracy constraints are given. Next, we derive an analytical energy model from the power consumption measurements using a wearable IoT device prototype. Then, we prove that maximizing the number of recognized gestures is equivalent to minimizing the duration of gesture recognition. Finally, we utilize this result to construct an optimization technique that maximizes the number of gestures recognized under the energy budget constraints while satisfying the recognition accuracy requirements. Our extensive evaluations demonstrate that the proposed analytical model is valid for wearable IoT applications, and the optimization approach increases the number of recognized gestures by up to 2.4× compared to a manual optimization.

Segmentation of Moving Objects using Numerous Background Subtraction Methods for Surveillance Applications

Background subtraction is a key part to detect moving objects from the video in computer vision field. It is used to subtract reference frame to every new frame of video scenes. There are wide varieties of background subtraction techniques available in literature to solve real life applications like crowd analysis, human activity tracking system, traffic analysis and many more. Moreover, there were not enough benchmark datasets available which can solve all the challenges of subtraction techniques for object detection. Thus challenges were found in terms of dynamic background, illumination changes, shadow appearance, occlusion and object speed. In this perspective, we have tried to provide exhaustive literature survey on background subtraction techniques for video surveillance applications to solve these challenges in real situations. Additionally, we have surveyed eight benchmark video datasets here namely Wallflower, BMC, PET, IBM, CAVIAR, CD.Net, SABS and RGB-D along with their available ground truth. This study evaluates the performance of five background subtraction methods using performance parameters such as specificity, sensitivity, FNR, PWC and F-Score in order to identify an accurate and efficient method for detecting moving objects in less computational time.

Human Activity Tracking by Mobile Phones Through Hebbian Learning

A method for human activity recognition using mobile phones is introduced. Using the accelerometer and gyroscope typically found in modern smartphones, a system that uses the proposed method is able to recognize low level activities, including athletic exercises, with high accuracy. A Hebbian learning preprocessing stage is used to render accelerometer and gyroscope signals independent to the orientation of the smartphone inside the user’s pocket. After preprocessing, a selected set of features are obtained and used for classification by a k-nearest neighbor or a multilayer perceptron. The trained algorithm achieves an accuracy of 95.3 percent when using the multilayer perceptron and tested on unknown users who are asked to perform the exercises after placing the mobile device in their pocket without any constraints on the orientation. Comparison of performance with respect to other popular methods is provided.

You Are Sensing, but Are You Biased?

Mobile devices are becoming pervasive to our daily lives: they follow us everywhere and we use them for much more than just communication. These devices are also equipped with a myriad of different sensors that have the potential to allow the tracking of human activities, user patterns, location, direction and much more. Following this direction, many movements including sports, quantified self, and mobile health ones are starting to heavily rely on this technology, making it pivotal that the sensors offer high accuracy. However, heterogeneity in hardware manufacturing, slight substrate differences, electronic interference as well as external disturbances are just few of the reasons that limit sensor output accuracy which in turn hinders sensor usage in applications which need very high granularity and precision, such as quantified-self applications. Although, calibration of sensors is a widely studied topic in literature to the best of our knowledge no publicly available research exists that specifically tackles the calibration of mobile phones and existing methods that can be adapted for use in mobile devices not only require user interaction but they are also not adaptive to changes. Additionally, alternative approaches for performing more granular and accurate sensing exploit body-wide sensor networks using mobile phones and additional sensors; as one can imagine these techniques can be bulky, tedious, and not particularly user friendly. Moreover, existing techniques for performing data corrections post-acquisition can produce inconsistent results as they miss important context information provided from the device itself; which when used, has been shown to produce better results without a imposing a significant power-penalty. In this paper we introduce a novel multiposition calibration scheme that is specifically targeted at mobile devices Our scheme exploits machine learning techniques to perform an adaptive, power-efficient auto-calibration procedure with which achieves high output sensor accuracy when compared to state of the art techniques without requiring any user interaction or special equipment beyond device itself Moreover, the energy costs associated with our approach are lower than the alternatives (such as Kalman filter based solutions) and the overall power penalty is < 5% when compared against power usage that is exhibited when using uncalibrated traces, thus, enabling our technique to be used efficiently on a wide variety of devices Finally, our evaluation illustrates that calibrated signals offer a tangible benefit in classification accuracy, ranging from 3 to 10%, over uncalibrated ones when using state of the art classifiers, on the other hand when using simpler SVM classifiers the classification improvement is boosted ranging from 8% to 12% making lower performing classifiers much more reliable Additionally, we show that for similar activities which are hard to distinguish otherwise, we reach an accuracy of > 95% when using neural network classifiers and > 88% when using SVM classifiers where uncalibrated data classification only reaches ~ 85% and ~ 80% respectively This can be a make or break factor in the use of accelerometer and gyroscope data in applications requiring high accuracy e g sports, health, games and others

Wearable Internet of Things - from human activity tracking to clinical integration.

Wearable devices for human activity tracking have been emerging rapidly. Most of them are capable of sending health statistics to smartphones, smartwatches or smart bands. However, they only provide the data for individual analysis and their data is not integrated into clinical practice. Leveraging on the Internet of Things (IoT), edge and cloud computing technologies, we propose an architecture which is capable of providing cloud based clinical services using human activity data. Such services could supplement the shortage of staff in primary healthcare centers thereby reducing the burden on healthcare service providers. The enormous amount of data created from such services could also be utilized for planning future therapies by studying recovery cycles of existing patients. We provide a prototype based on our architecture and discuss its salient features. We also provide use cases of our system in personalized and home based healthcare services. We propose an International Telecommunication Union based standardization (ITU-T) for our design and discuss future directions in wearable IoT.

Review on Advanced Short-Range Multimode Continuous-Wave Radar Architectures for Healthcare Applications

Short-range radar systems are noncontact devices that are attracting much attention from the RF/microwave community. In particular, they are being applied in healthcare contexts, ranging from hospital to indoor and outdoor environments, e.g., for monitoring of vital signs and/or tracking of human activities. This article reviews the standard architectures for short-range biomedical radars and presents recently proposed hybrid schemes, which combine different operation modes, leading to performance improvement. Specifically, hybridizations associated with Doppler, frequency-modulated continuous-wave, and tone-ranging-based schemes are provided from theoretical and simulation viewpoints. Some experiments on vital-sign monitoring and human-aware localization are also reported.

A Robust and Device-Free System for the Recognition and Classification of Elderly Activities.

Human activity recognition, tracking and classification is an essential trend in assisted living systems that can help support elderly people with their daily activities. Traditional activity recognition approaches depend on vision-based or sensor-based techniques. Nowadays, a novel promising technique has obtained more attention, namely device-free human activity recognition that neither requires the target object to wear or carry a device nor install cameras in a perceived area. The device-free technique for activity recognition uses only the signals of common wireless local area network (WLAN) devices available everywhere. In this paper, we present a novel elderly activities recognition system by leveraging the fluctuation of the wireless signals caused by human motion. We present an efficient method to select the correct data from the Channel State Information (CSI) streams that were neglected in previous approaches. We apply a Principle Component Analysis method that exposes the useful information from raw CSI. Thereafter, Forest Decision (FD) is adopted to classify the proposed activities and has gained a high accuracy rate. Extensive experiments have been conducted in an indoor environment to test the feasibility of the proposed system with a total of five volunteer users. The evaluation shows that the proposed system is applicable and robust to electromagnetic noise.

Human Activity Tracking by Mobile Phones Through Hebbian Learning

Human Activity Tracking by Mobile Phones Through Hebbian Learning

A survey on Flickr multimedia research challenges

Multimedia content sharing within social networks has become one of the most interesting and trending research fields over the last few years. This undoubted emerge of related research works is rather twofold, namely it includes both the analysis and management techniques of the content itself, as well as new ways for its accompanied meaningful interpretation and exploitation. In this paper, we review the recent advances in the above fields in the humanistic framework of the popular Flickr social network. In addition, the major research challenges in the area are demonstrated and discussed, which include current state-of-the-art approaches with respect to interesting humanistic data collection and interpretation research fields, such as multimedia information retrieval, (semi-) automatic tag manipulation, travel applications, semantic knowledge extraction, human activity tracking, as well as related benchmarking efforts. At the end of this survey, we also discuss the main challenges and propose a number of future research directions for interested fellow researchers to continue investigation in the field.

Dense RGB-D Map-Based Human Tracking and Activity Recognition using Skin Joints Features and Self-Organizing Map

This paper addresses the issues of 3D human activity detection, tracking and recognition from RGB-D video sequences using a feature structured framework. During human tracking and activity recognition, initially, dense depth images are captured using depth camera. In order to track human silhouettes, we considered spatial/temporal continuity, constraints of human motion information and compute centroids of each activity based on chain coding mechanism and centroids point extraction. In body skin joints features, we estimate human body skin color to identify human body parts (i.e., head, hands, and feet) likely to extract joint points information. These joints points are further processed as feature extraction process including distance position features and centroid distance features. Lastly, self-organized maps are used to recognize different activities. Experimental results demonstrate that the proposed method is reliable and efficient in recognizing human poses at different realistic scenes. The proposed system should be applicable to different consumer application systems such as healthcare system, video surveillance system and indoor monitoring systems which track and recognize different activities of multiple users.