Activity Recognition In Smart Homes Research Articles

A systematic review of non-intrusive human activity recognition in smart homes using deep learning

<p>Smart homes are a viable solution for improving the independence and privacy of elderly and dependent people thanks to IoT sensors. Reliable human activity recognition (HAR) devices are required to enable precise monitoring inside smart homes. Despite various reviews on HAR, there is a lack of comprehensive studies that include a diverse range of approaches, including sensor-based, wearable, ambient, and device-free methods. Considering this research gap, this study aims to systematically review the HAR studies that apply deep learning as their main solution and utilize a non-intrusive approach for activity monitoring. Out of the 2,171 studies in the IEEE Explore database, we carefully selected and thoroughly analyzed 37 studies for our research, following the guidelines provided by the preferred reporting items for systematic reviews and meta-analyses (PRISMA) methodology. In this paper, we explore various modalities, deep learning approaches, and datasets employed in the context of non-intrusive HAR. This study presents essential data for researchers to employ deep learning techniques for HAR in smart home environments. Additionally, it identifies and highlights the main trends, challenges, and future directions.</p>

AtomGID: An Atomic Gesture Identifier for Qualitative Spatial Reasoning

In this paper, we present a novel non-deep-learning-based approach for real-time object tracking and activity recognition within smart homes, aiming to minimize human intervention and dataset requirements. Our method utilizes discreet, easily concealable sensors and passive RFID technology to track objects in real-time, enabling precise activity recognition without the need for extensive datasets typically associated with deep learning techniques. Central to our approach is AtomGID, an algorithm tailored to extract highly generalizable spatial features from RFID data. Notably, AtomGID’s adaptability extends beyond RFID to other imprecise tracking technologies like Bluetooth beacons and radars. We validate AtomGID through simulation and real-world RFID data collection within a functioning smart home environment. To enhance recognition accuracy, we employ a clustering adaptation of the flocking algorithm, leveraging previously published Activities of Daily Living (ADLs) data. Our classifier achieves a robust classification rate ranging from 85% to 93%, underscoring the efficacy of our approach in accurately identifying activities. By prioritizing non-deep-learning techniques and harnessing the strengths of passive RFID technology, our method offers a pragmatic and scalable solution for activity recognition in smart homes, significantly reducing dataset dependencies and human intervention requirements.

Using Graphs to Perform Effective Sensor-Based Human Activity Recognition in Smart Homes.

There has been a resurgence of applications focused on human activity recognition (HAR) in smart homes, especially in the field of ambient intelligence and assisted-living technologies. However, such applications present numerous significant challenges to any automated analysis system operating in the real world, such as variability, sparsity, and noise in sensor measurements. Although state-of-the-art HAR systems have made considerable strides in addressing some of these challenges, they suffer from a practical limitation: they require successful pre-segmentation of continuous sensor data streams prior to automated recognition, i.e., they assume that an oracle is present during deployment, and that it is capable of identifying time windows of interest across discrete sensor events. To overcome this limitation, we propose a novel graph-guided neural network approach that performs activity recognition by learning explicit co-firing relationships between sensors. We accomplish this by learning a more expressive graph structure representing the sensor network in a smart home in a data-driven manner. Our approach maps discrete input sensor measurements to a feature space through the application of attention mechanisms and hierarchical pooling of node embeddings. We demonstrate the effectiveness of our proposed approach by conducting several experiments on CASAS datasets, showing that the resulting graph-guided neural network outperforms the state-of-the-art method for HAR in smart homes across multiple datasets and by large margins. These results are promising because they push HAR for smart homes closer to real-world applications.

Optimizing Deep LSTM Model through Hyperparameter Tuning for Sensor-Based Human Activity Recognition in Smart Home

Optimizing Deep LSTM Model through Hyperparameter Tuning for Sensor-Based Human Activity Recognition in Smart Home

Activity Recognition in Smart Homes via Feature-Rich Visual Extraction of Locomotion Traces

The proliferation of sensors in smart homes makes it possible to monitor human activities, routines, and complex behaviors in an unprecedented way. Hence, human activity recognition has gained increasing attention over the last few years as a tool to improve healthcare and well-being in several applications. However, most existing activity recognition systems rely on cameras or wearable sensors, which may be obtrusive and may invade the user’s privacy, especially at home. Moreover, extracting expressive features from a stream of data provided by heterogeneous smart-home sensors is still an open challenge. In this paper, we investigate a novel method to detect activities of daily living by exploiting unobtrusive smart-home sensors (i.e., passive infrared position sensors and sensors attached to everyday objects) and vision-based deep learning algorithms, without the use of cameras or wearable sensors. Our method relies on depicting the locomotion traces of the user and visual clues about their interaction with objects on a floor plan map of the home, and utilizes pre-trained deep convolutional neural networks to extract features for recognizing ongoing activity. One additional advantage of our method is its seamless extendibility with additional features based on the available sensor data. Extensive experiments with a real-world dataset and a comparison with state-of-the-art approaches demonstrate the effectiveness of our method.

A multimodal fusion enabled ensemble approach for human activity recognition in smart homes.

How to deal with multi-modality data from different types of devices is a challenging issue for accurate recognition of human activities in a smart environment. In this paper, we propose a multimodal fusion enabled ensemble approach. Firstly, useful features collected from Bluetooth beacons, binary sensors, and smart floor are extracted and presented by fuzzy logic based-method with variable-size temporal windows. Secondly, a group of support vector machine classifiers are used to perform the classification task. Finally, a weighted ensemble method is used to obtain the final prediction. Especially, by applying the geometric framework, we are able to obtain the optimal weights for the ensemble. The proposed approach is evaluated on the UJAmI dataset. The experimental results demonstrate the efficacy and robustness of the proposed method.

Real-time Monitoring of Activity Recognition in Smart Homes: An Intelligent IoT Framework

The rapid proliferation of the Internet of Things (IoT) has paved the way for transformative innovations, and this paper explores its profound impact on the realm of elderly care within smart homes. We present a pioneering IoT-based approach for human activity recognition, addressing the critical need for accurate and non-intrusive monitoring of elderly individuals. Our IoT-based approach begins with data preprocessing, where raw sensor data is refined using median filtering, reducing noise and ensuring high-quality inputs for our model. We apply the series_to_supervised transformation to convert the sensor data into a supervised learning format, which is critical for training the GRU-based activity recognition model. The heart of our approach lies in the federated distillation-based training strategy. Edge devices within the IoT network locally train their GRU models using their datasets while sharing knowledge with a central server and other edge devices. Knowledge distillation further enhances the model's performance by transferring knowledge from the global model to the edge devices. Experimental analysis demonstrated an impressive accuracy of 95% and an F1-score of 0.94, Our system excels in recognizing and classifying a wide range of human activities, from daily routines to emergencies.

Use of Clustering Algorithms for Sensor Placement and Activity Recognition in Smart Homes

This work presents a novel method for motion sensor placement within smart homes. Using recordings from 3D depth cameras within six real homes, clusters are created with the resident’s tracked location. The resulting clusters identify the possible position of a sensor and its field of view. By using a sequence of clusters as input to a Recurrent Neural Network, we evaluate our method on the task of activity recognition and prediction. These results are compared to using sensor events as input sequence, from motion sensors that were installed empirically in the same homes. Different clustering methods are investigated and all outperform the installed motion sensors, achieving a significant increase of prediction accuracy and F1-score.

MLMO-HSM: Multi-label Multi-output Hybrid Sequential Model for multi-resident smart home activity recognition

MLMO-HSM: Multi-label Multi-output Hybrid Sequential Model for multi-resident smart home activity recognition

Research on Hyper-Parameter Optimization of Activity Recognition Algorithm Based on Improved Cuckoo Search

Activity recognition methods often include some hyper-parameters based on experience, which greatly affects their effectiveness in activity recognition. However, the existing hyper-parameter optimization algorithms are mostly for continuous hyper-parameters, and rarely for the optimization of integer hyper-parameters and mixed hyper-parameters. To solve the problem, this paper improved the traditional cuckoo algorithm. The improved algorithm can optimize not only continuous hyper-parameters, but also integer hyper-parameters and mixed hyper-parameters. This paper validated the proposed method with the hyper-parameters in Least Squares Support Vector Machine (LS-SVM) and Long-Short-Term Memory (LSTM), and compared the activity recognition effects before and after optimization on the smart home activity recognition data set. The results show that the improved cuckoo algorithm can effectively improve the performance of the model in activity recognition.

Online conflict resolution strategies for human activity recognition in smart homes

ABSTRACT DCR-OL is a Distributed Collaborative Reasoning multi-agent model with an Online Learning that aims to identify human activities in smart homes from distributed, heterogeneous and dynamic sensor data. In this model, distributed learning agents with diverse classifiers, detect sensor stream data, make local predictions, communicate and collaborate to identify current activities. Then, they learn from their collaborations to improve their own performance in activity recognition. Conflict resolution strategies are applied to generate one final predicted activity when the local predicted activity of an agent is different from received predicted activities of other agents. In this paper, two conflict resolution strategies using online learning, w-max-trust and w-max-freq, are proposed. We experimentally test these strategies by performing an evaluation study on the Aruba dataset. The obtained results indicate an enhancement in terms of accuracy and F-measure metrics compared to the offline strategies max-trust and max-freq and also to the online existing one max-wPerf .

Are Smart Homes Adequate for Older Adults with Dementia?

Smart home technologies can enable older adults, including those with dementia, to live more independently in their homes for a longer time. Activity recognition, in combination with anomaly detection, has shown the potential to recognise users’ daily activities and detect deviations. However, activity recognition and anomaly detection are not sufficient, as they lack the capacity to capture the progression of patients’ habits across the different stages of dementia. To achieve this, smart homes should be enabled to recognise patients’ habits and changes in habits, including the loss of some habits. In this study, we first present an overview of the stages that characterise dementia, alongside real-world personas that depict users’ behaviours at each stage. Then, we survey the state of the art on activity recognition in smart homes for older adults with dementia, including the literature that combines activity recognition and anomaly detection. We categorise the literature based on goals, stages of dementia, and targeted users. Finally, we justify the necessity for habit recognition in smart homes for older adults with dementia, and we discuss the research challenges related to its implementation.

Smart Home Activity Recognition for Ambient Assisted Living (AAL)

With the increasing age of an individual, the chances of being prone to chronic diseases like diabetes or non-curable diseases like Alzheimer’s Syndrome or Parkinson's Syndrome increases. Due to the health issues, elderly must be accompanied by caretakers to monitor their well-being at all times. With growing responsibilities and work pressure, the family members may find it challenging to find a trustworthy caretaker. In such scenarios, an assisted living environment acts as a boon. A normal home embedded with different sensors to monitor an individual’s well-being is called as Ambient Assisted Living(AAL). This living environment detects anomalous behaviour and recognizes human activities. In this research paper, a smart home activity recognition model is proposed and implemented using four machine learning algorithms using six different publicly available datasets. It has been observed that Random Forest machine learning algorithm shows the best accuracy on most of the dataset.

New incremental SVM algorithms for human activity recognition in smart homes

Smart homes are equipped with several sensor networks to keep an eye on both residents and their environment, to interpret the current situation and to react immediately. Handling large scale dataset of sensory events on real time to enable efficient interventions is challenging and very difficult. To deal with these data flows and challenges, traditional streaming data classification approaches can be boosted by use of incremental learning. In this paper, we presented two new Incremental SVM methods to improve the performance of SVM classification in the context of human activity recognition tasks. Two feature extraction methods elaborated by refining dependency sensor extraction feature and focusing on the last sensor event only have been suggested. On the other hand, a clustering based approach and a similarity based approach have been suggested to boost learning performance of the incremental SVM algorithms capitalizing on the relationship between data chunk and support vectors of previous chunk. We demonstrate through several simulations on two major publicly available data sets (Aruba and Tulum), the feasibility and improvements in learning and classification performances in real time achieved by our proposed methods over the state-of-the-art. For instance, we have shown that the introduced similarity-based incremental learning is 5 to 9 times faster than other methods in terms of training performances. Similarly, the introduced Last-state sensor feature method induces at least 5% improvement in terms of F1-score when using baseline SVM classifier.

A Deep Learning-Based Framework for Human Activity Recognition in Smart Homes

Human behavior modeling in smart environments is a growing research area treating several challenges related to ubiquitous computing, pattern recognition, and ambient assisted living. Thanks to recent progress in sensing devices, it is now possible to design computational models able of accurate detection of residents’ activities and daily routines. For this goal, we introduce in this paper a deep learning-based framework for activity recognition in smart homes. This framework proposes a detailed methodology for data preprocessing, feature mining, and deep learning techniques application. The novel framework was designed to ensure a deep exploration of the feature space since three main approaches are tested, namely, the all-features approach, the selection approach, and the reduction approach. Besides, the framework proposes the evaluation and the comparison of several well-chosen deep learning techniques such as autoencoder, recurrent neural networks (RNN), and some of their derivatives models. Concretely, the framework was applied on the “Orange4Home” dataset which represents a recent dataset specially designed for smart homes research. Our main findings show that the best approach for efficient classification is the selection approach. Furthermore, our overall results outperformed baseline models based on random forest classifiers and the principal component analysis technique, especially the results of our RNN-based model for the all-features approach and the results of our autoencoder-based model for the feature reduction approach.

A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning.

Recent advances in Internet of Things (IoT) technologies and the reduction in the cost of sensors have encouraged the development of smart environments, such as smart homes. Smart homes can offer home assistance services to improve the quality of life, autonomy, and health of their residents, especially for the elderly and dependent. To provide such services, a smart home must be able to understand the daily activities of its residents. Techniques for recognizing human activity in smart homes are advancing daily. However, new challenges are emerging every day. In this paper, we present recent algorithms, works, challenges, and taxonomy of the field of human activity recognition in a smart home through ambient sensors. Moreover, since activity recognition in smart homes is a young field, we raise specific problems, as well as missing and needed contributions. However, we also propose directions, research opportunities, and solutions to accelerate advances in this field.

Novel evolutionary-EAC instance-learning-based algorithm for fast data stream mining in assisted living with extreme connectivity

In modern healthcare, sensing technologies such as IoT empower the quality of assisted living service by knowing what a resident is doing in real-time. Using extreme connectivity and cloud computing in a smart home, where a collection of sensors is installed, the sensors sample continuously from the movements of the resident as well as ambient data from the surrounding inside the house. Automatic human activity recognition of the resident's activities is one of the key components of assisted living in smart home. For monitoring in-home safety, the ability in recognizing abnormal activities such as accident, falling, acute disease attack (e.g. asthma, stroke, etc.), fainting, wobbling, is particularly important. The detection and machine learning process must be both accurate and fast, to cope with the real-time activity recognition. To this end, a novel streamlined sensor data processing method is proposed called Evolutionary Expand-and-Contract Instance-based Learning algorithm (EEAC-IBL). The multivariate data stream is first expanded into many subspaces, then the subspaces which are corresponding to the characteristics of the features are selected and condensed into a significant feature subset. The selection operates scholastically instead of deterministically by evolutionary optimization which approximates the best subgroup. Followed by data stream mining, the machine learning for activity recognition is done on the fly. This approach is unique and suitable for such extreme connectivity scenario where precise feature selection is not required, and the relative importance of each feature among the sensor data changes over time. This stochastic approximation method is fast and accurate, offering an alternative to traditional machine learning method for smart home activity recognition application. Our experimental results show computing advantages over other classical approaches.

A distributable event-oriented architecture for activity recognition in smart homes

In this paper, a new architecture is proposed for continuously generating, propagating, and delivering information by using event-based communication between independent agents. The resulting system can both handle heterogeneous smart environments and compute information in multiple places. With a communication method working as an abstraction layer, the proposed solution enables the use of multiple technologies at once. Additionally, different options for delivering the resulting data to client applications are explored. The implementation of this design as a platform written in Java with the Spring Framework is also presented, along with its handling of ten housing facilities equipped with various sensors (electromagnetic contacts, smart plugs, motion detectors, humidity, temperature, and light sensors). This paper is then concluded by an analysis of the platform workloads incurred by the tracking of a set of low-level activities. Finally, the code is distributed online for the benefit of the community.

IoT Based Approach for Load Monitoring and Activity Recognition in Smart Homes

Appliance load monitoring in smart homes has been gaining importance due to its significant advantages in achieving an energy efficient smart grid. The methods to manage such processes can be classified into hardware-based methods, including intrusive load monitoring (ILM) and software-based methods referring to non-intrusive load monitoring (NILM). ILM is based on low-end meter devices attached to home appliances in opposition to NILM techniques, where only a single point of sensing is needed. Although ILM solutions can be relatively expensive, they provide higher efficiency and reliability than NILMs. Moreover, future solutions are expected to be hybrid, combining the benefits of NILM along with individual power measurement by smart plugs and smart appliances. This paper proposes a novel ILM approach for load monitoring that aims to develop an activity recognition system based on IoT architecture. The proposed IoT architecture consists of the appliances layer, perception layer, communication network layer, middleware layer, and application layer. The main function of the appliance recognition module is to label sensor data and allow the implementation of different home applications. Three different classifier models are tested using real data from the UK-DALE dataset: feed-forward neural network (FFNN), long short-term memory (LSTM), and support vector machine (SVM). The developed activities of daily living (ADL) algorithm maps each ADL to a set of criteria depending on the appliance used. The features are extracted according to the consumption in Watt-hours and the times where they are switched on. In the FFNN and the LSTM networks, the accuracy is above 0.9 while around 0.8 for the SVM network. Other experiments are performed to evaluate the classifier model using a new test set. A sensitivity analysis is also carried out to study the impact of the group size on the classifier accuracy.

Predicting Activities of Daily Living with Spatio-Temporal Information

The smart home has begun playing an important role in supporting independent living by monitoring the activities of daily living, typically for the elderly who live alone. Activity recognition in smart homes has been studied by many researchers with much effort spent on modeling user activities to predict behaviors. Most people, when performing their daily activities, interact with multiple objects both in space and through time. The interactions between user and objects in the home can provide rich contextual information in interpreting human activity. This paper shows the importance of spatial and temporal information for reasoning in smart homes and demonstrates how such information is represented for activity recognition. Evaluation was conducted on three publicly available smart-home datasets. Our method achieved an average recognition accuracy of more than 81% when predicting user activities given the spatial and temporal information.