Multi-resident Activity Recognition Research Articles

Class imbalance in multi-resident activity recognition: an evaluative study on explainability of deep learning approaches

Class imbalance in multi-resident activity recognition: an evaluative study on explainability of deep learning approaches

Fusion of Deep Learning Models with Attention for Multi-Resident Activity Recognition in Ambient Assisted Living

Fusion of Deep Learning Models with Attention for Multi-Resident Activity Recognition in Ambient Assisted Living

Multilabel Classification Methods for Human Activity Recognition: A Comparison of Algorithms.

As the world’s population is aging, and since access to ambient sensors has become easier over the past years, activity recognition in smart home installations has gained increased scientific interest. The majority of published papers in the literature focus on single-resident activity recognition. While this is an important area, especially when focusing on elderly people living alone, multi-resident activity recognition has potentially more applications in smart homes. Activity recognition for multiple residents acting concurrently can be treated as a multilabel classification problem (MLC). In this study, an experimental comparison between different MLC algorithms is attempted. Three different techniques were implemented: RAkEL, classifier chains, and binary relevance. These methods are evaluated using the ARAS and CASAS public datasets. Results obtained from experiments have shown that using MLC can recognize activities performed by multiple people with high accuracy. While RAkEL had the best performance, the rest of the methods had on-par results.

Applying Machine Learning Technologies Based on Historical Activity Features for Multi-Resident Activity Recognition.

Due to the aging population, home care for the elderly has become very important. Currently, there are many studies focusing on the deployment of various sensors in the house to recognize the home activities of the elderly, especially for the elderly living alone. Through these, we can detect the home situation of the single person and ensure his/her living safety. However, the living environment of the elderly includes, not only the person living alone, but also multiple people living together. By applying the traditional methods for a multi-resident environment, the “individual” activities of each person could not be accurately identified. This resulted in an inability to distinguish which person was involved in what activities, and thus, failed to provide personal care. Therefore, this research tries to investigate how to recognize home activities in multi-resident living environments, in order to accurately distinguish the association between residents and home activities. Specifically, we propose to use the special characteristics of historical activity of residents in a multi-person environment, including activity interaction, activity frequency, activity period length, and residential behaviors, and then apply a suite of machine learning methods to train and test. Five traditional models of supervised learning and two deep learning methods are explored to tackle this problem. Through the experiments with real datasets, the proposed methods were found to achieve higher precision, recall and accuracy with less training time. The best accuracy can reach up to 91% and 95%, by J48DT, and LSTM, respectively, in different living environments.

Sequential neural networks for multi-resident activity recognition in ambient sensing smart homes

Advances in smart home technology and IoT devices had made us capable of monitoring human activities in a non-intrusive way. This data, in turn, enables us to predict the health status and energy consumption patterns of residents of these smart homes. Machine learning has been an excellent tool for the prediction of human activities from raw sensor data of a single resident. However, Multi Resident activity recognition is still a challenging task, as there is no correlation between sensor values and resident activities. In this paper, we have applied deep learning algorithms on the real world ARAS Multi Resident data set, which consists of data from two houses, each with two residents. We have used different variations of Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and their combination on the data set and kept the labels separate for both residents. We have evaluated the performance of models based on several metrics.

Hybrid Fuzzy C-Means CPD-Based Segmentation for Improving Sensor-Based Multiresident Activity Recognition

Multiresident activity recognition (AR), which has become a popular research field in smart environments, aims to recognize the activities of multiple residents based on data collected from various types of sensors, and sensor events segmentation is an important technique for enhancing the performance of AR. While quite some segmentation methods have been proposed for the single person setting, few studies have been done for the multiresident setting. In this article, we first evaluate the baseline and the state-of-the-art segmentation methods using the popular multiresident data set CASAS, to confirm that the performance of multiresident AR can be improved by applying segmentation techniques; we then propose a novel Hybrid fuzzy c-means (FCM) change point detection (CPD)-based segmentation method that can further enhance the performance of multiresident AR. We combine a FCM method with a CPD-based method for sensor event segmentation. The FCM method is used to classify the sensor events in terms of sensor locations, and then the CPD technique is used to probe the transition actions to determine the segmentation sequence. Our experimental results show that the proposed method significantly improves the performance of multiresident AR in comparison with the baseline and state-of-the-art classification methods.

Multi-label classification based ensemble learning for human activity recognition in smart home

In recent times with advancements in wireless sensor technologies, human activity recognition in smart home environments have gained significant interest amongst the researchers. Numerous literatures have been focusing only on single occupant-based activity recognition because of escalating difficulties with multiple occupants within the same conditions. But having multiple occupants is more general. Therefore, in this study, the application of the Classifier Chain method of Multi-Label Classification is described to address the complicated problem of multi-resident activity recognition. Four different classifiers namely Bernoulli Naïve Bayes, Decision Tree, Logistic Regression and K-Nearest Neighbor as base classifiers for this multi-label classification approach are implemented and a comparative study of these model's performance is presented. Furthermore, the Majority Voting Ensemble Classifier method based on the ensemble learning is developed for the activity recognition problem. All these models are evaluated using several evaluation metrics on publicly available ARAS datasets. Through the results obtained from experiments, it can be deduced that the Classifier Chain method not only handles the challenges of this complex problem competently but also exhibits the importance of multi-label classification approach towards the field of activity recognition.

Sequence Classification Restricted Boltzmann Machines With Gated Units

For the classification of sequential data, dynamic Bayesian networks and recurrent neural networks (RNNs) are the preferred models. While the former can explicitly model the temporal dependences between the variables, and the latter have the capability of learning representations. The recurrent temporal restricted Boltzmann machine (RTRBM) is a model that combines these two features. However, learning and inference in RTRBMs can be difficult because of the exponential nature of its gradient computations when maximizing log likelihoods. In this article, first, we address this intractability by optimizing a conditional rather than a joint probability distribution when performing sequence classification. This results in the "sequence classification restricted Boltzmann machine" (SCRBM). Second, we introduce gated SCRBMs (gSCRBMs), which use an information processing gate, as an integration of SCRBMs with long short-term memory (LSTM) models. In the experiments reported in this article, we evaluate the proposed models on optical character recognition, chunking, and multiresident activity recognition in smart homes. The experimental results show that gSCRBMs achieve the performance comparable to that of the state of the art in all three tasks. gSCRBMs require far fewer parameters in comparison with other recurrent networks with memory gates, in particular, LSTMs and gated recurrent units (GRUs).

Mixed-dependency models for multi-resident activity recognition in smart homes

Recent growing interest in ambient intelligent environments has driven a desire for effective models to reason about activities of multiple residents. Such models are the keystone for the future of smart homes where occupants can be assisted with non-intrusive technologies. Much attention has been put on this research, however current works tend to focus on developing statistical algorithms for prediction, whilst there still lacks a study to fully understand the relations of residents’ behaviours and how they are reflected through the sensors’ states. In this paper we investigate the dependencies of the activities from residents and their interaction with the environments. We represent such dependencies in Bayesian networks that leads to construction of six variants of Hidden Markov Models (HMMs). Furthermore, we argue that a complete model should embody more than one type of dependency. Therefore, we propose an ensemble of HMMs, and then generalize it to a novel mixed-dependency model. In the experiments we perform intensive evaluation of our study on multi-resident activity recognition task. The results show that the proposed models outperform other models in three smart home environments, thus asserting our hypothesis.

Recognition of Daily Activities of Two Residents in a Smart Home Based on Time Clustering.

With the development of population aging, the recognition of elderly activity in smart homes has received increasing attention. In recent years, single-resident activity recognition based on smart homes has made great progress. However, few researchers have focused on multi-resident activity recognition. In this paper, we propose a method to recognize two-resident activities based on time clustering. First, to use a de-noising method to extract the feature of the dataset. Second, to cluster the dataset based on the begin time and end time. Finally, to complete activity recognition using a similarity matching method. To test the performance of the method, we used two two-resident datasets provided by Center for Advanced Studies in Adaptive Systems (CASAS). We evaluated our method by comparing it with some common classifiers. The results show that our method has certain improvements in the accuracy, recall, precision, and F-Measure. At the end of the paper, we explain the parameter selection and summarize our method.

Unsupervised Recognition of Multi-Resident Activities in Smart-Homes

Several methods have been proposed in the last two decades to recognize human activities based on sensor data acquired in smart-homes. While most existing methods assume the presence of a single inhabitant, a few techniques tackle the challenging issue of multi-resident activity recognition. To the best of our knowledge, all existing methods for multi-inhabitant activity recognition require the acquisition of a labeled training set of activities and sensor events. Unfortunately, activity labeling is costly and may disrupt the users' privacy. In this article, we introduce a novel technique to recognize multi-inhabitant activities without the need of labeled datasets. Our technique relies on an unlabeled sensor data stream acquired from a single resident, and on ontological reasoning to extract probabilistic associations among sensor events and activities. Extensive experiments with a large dataset of multi-inhabitant activities show that our technique achieves an average accuracy very close to the one of state-of-the-art supervised methods, without requiring the acquisition of labeled data.

On multi-resident activity recognition in ambient smart-homes

Increasing attention to the research on activity monitoring in smart homes has motivated the employment of ambient intelligence to reduce the deployment cost and solve the privacy issue. Several approaches have been proposed for multi-resident activity recognition, however, there still lacks a comprehensive benchmark for future research and practical selection of models. In this paper, we study different methods for multi-resident activity recognition and evaluate them on the same sets of data. In particular, we explore the effectiveness and efficiency of temporal learning algorithms using sequential data and non-temporal learning algorithms using temporally-manipulated features. In the experiments we compare and analyse the results of the studied methods using datasets from three smart homes.

Multi-Resident Activity Recognition in a Smart Home Using RGB Activity Image and DCNN

In the last decade, unobtrusive (device-free and non-privacy invasive) recognition of activities of daily living for an individual in a smart home has been studied by many researchers. However, the unobtrusive recognition of multi-resident activities in a smart home is hardly studied. We propose a novel RGB activity image-based DCNN classifier for the unobtrusive recognition of the multi-resident activities (Bed_to_Toilet, Bed, Breakfast, Lunch, Leave_home, Laundry, Dinner, Night_wandering, R2_work, and R1_medicine) using Cairo open data set provided by the CASAS Project. The open data set is collected by environmental sensors (PIR and temperature sensors) in Cairo testbed, while an adult couple with a dog was living for 55 days. The data set is preprocessed with activity segmentation, sliding window, and RGB activity image conversion steps. The experimental results demonstrate that our classifier has the highest total accuracy of 95.2% among the previously developed machine learning classifiers that employed the same data set. Moreover, the proposed RGB activity image was proven to be helpful for increasing the recognition rate. Therefore, we conclude that the proposed DCNN classifier is a useful tool for the unobtrusive recognition of the multi-resident activity in a home.

Recognizing multi-resident activities in non-intrusive sensor-based smart homes by formal concept analysis

Abstract Activity recognition is one of the most important prerequisites for smart home applications. It is a challenging topic due to the high requirements for reliable data acquisition and efficient data analysis. Besides, the heterogeneous layouts of smart homes, the number of residents and varied human behavioral patterns also aggravate the complexity of recognition. Therefore, most human activity recognition systems are based on an unrealistic assumption that there is only one resident performing activities. In this paper, we investigate the issue of multi-resident activity recognition and propose a knowledge-driven solution on the basis of formal concept analysis (FCA) to identify human activities from non-intrusive sensor data. We extract the ontological correlations among sequential behavioral patterns. At the same time, these correlations are well organized in a graphical knowledge base, without intervention from domain experts. We propose an incremental lattice search strategy in order to retrieve the best inference given a few sensor events. Compared with other conventional probabilistic methods, our solution outperforms on the CASAS multi-resident benchmark dataset. Furthermore, we open up a promising solution of sequential pattern mining to discover the ontological features of temporal and sequential sensor data.

Multi-resident activity tracking and recognition in smart environments

During last decade, smart homes in which the activities of the residents are monitored automatically have been developed and demonstrated. However, smart homes with multiple residents still remains an open challenge. In order to tackle the multiple resident concurrent activity recognition problem in smart homes equipped with interaction-based sensors and with multiple residents, we propose two different approaches. In the first approach, we use a factorial hidden Markov model for modeling two separate chains corresponding to two residents. Secondly, we use nonlinear Bayesian tracking for decomposing the observation space into the number of residents. As opposed to the previous studies, we handle multiple residents at the same time without assuming any explicit identification mechanisms. We perform two experiments on real-world multi-resident Activity Recognition with Ambient Sensing data sets. In each experiment, we compare the proposed approach with a counterpart method. We also compare each approach with the manually separated observation performances. We show that both of the proposed methods consistently outperform their counterparts in both houses of the data sets and for both residents. We also discuss the advantages and disadvantages of each approach in terms of run time complexity, flexibility and generalizability.

A Two-stage Method for Solving Multi-resident Activity Recognition in Smart Environments

To recognize individual activities in multi-resident environments with pervasive sensors, some researchers have pointed out that finding data associations can contribute to activity recognition and previous methods either need or infer data association when recognizing new multi-resident activities based on new observations from sensors. However, it is often difficult to find out data associations, and available approaches to multi-resident activity recognition degrade when the data association is not given or induced with low accuracy. This paper exploits some simple knowledge of multi-resident activities through defining Combined label and the state set, and proposes a two-stage activity recognition method for multi-resident activity recognition. We define Combined label states at the model building phase with the help of data association, and learn Combined label states at the new activity recognition phase without the help of data association. Our two stages method is embodied in the new activity recognition phase, where we figure out multi-resident activities in the second stage after learning Combined label states at first stage. The experiments using the multi-resident CASAS data demonstrate that our method can increase the recognition accuracy by approximately 10%.