Ambient Intelligence Systems Research Articles

A Digital Companion Architecture for Ambient Intelligence

Ambient Intelligence (AmI) focuses on creating environments capable of proactively and transparently adapting to users and their activities. Traditionally, AmI focused on the availability of computational devices, the pervasiveness of networked environments, and means to interact with users. In this paper, we propose a renewed AmI architecture that takes into account current technological advancements while focusing on proactive adaptation for assisting and protecting users. This architecture consist of four phases: Perceive, Interpret, Decide, and Interact. The AmI systems we propose, called Digital Companions (DC), can be embodied in a variety of ways (e.g., through physical robots or virtual agents) and are structured according to these phases to assist and protect their users. We further categorize DCs into Expert DCs and Personal DCs, and show that this induces a favorable separation of concerns in AmI systems, where user concerns (including personal user data and preferences) are handled by Personal DCs and environment concerns (including interfacing with environmental artifacts) are assigned to Expert DCs; this separation has favorable privacy implications as well. Herein, we introduce this architecture and validate it through a prototype in an industrial scenario where robots and humans collaborate to perform a task.

Computer Vision Tasks for Ambient Intelligence in Children’s Health

Computer vision is a powerful tool for healthcare applications since it can provide objective diagnosis and assessment of pathologies, not depending on clinicians’ skills and experiences. It can also help speed-up population screening, reducing health care costs and improving the quality of service. Several works summarise applications and systems in medical imaging, whereas less work is devoted to surveying approaches for healthcare goals using ambient intelligence, i.e., observing individuals in natural settings. Even more, there is a lack of papers providing a survey of works exhaustively covering computer vision applications for children’s health, which is a particularly challenging research area considering that most existing computer vision technologies have been trained and tested only on adults. The aim of this paper is then to survey, for the first time in the literature, the papers covering children’s health-related issues by ambient intelligence methods and systems relying on computer vision.

Early prediction of atherosclerosis diagnosis with medical ambient intelligence.

Atherosclerosis is a chronic vascular disease that poses a significant threat to human health. Common diagnostic methods mainly rely on active screening, which often misses the opportunity for early detection. To overcome this problem, this paper presents a novel medical ambient intelligence system for the early detection of atherosclerosis by leveraging clinical data from medical records. The system architecture includes clinical data extraction, transformation, normalization, feature selection, medical ambient computation, and predictive generation. However, the heterogeneity of examination items from different patients can degrade prediction performance. To enhance prediction performance, the "SEcond-order Classifier (SEC)" is proposed to undertake the medical ambient computation task. The first-order component and second-order cross-feature component are then consolidated and applied to the chosen feature matrix to learn the associations between the physical examination data, respectively. The prediction is lastly produced by aggregating the representations. Extensive experimental results reveal that the proposed method's diagnostic prediction performance is superior to other state-of-the-art methods. Specifically, the Vitamin B12 indicator exhibits the strongest correlation with the early stage of atherosclerosis, while several known relevant biomarkers also demonstrate significant correlation in experimental data. The method proposed in this paper is a standalone tool, and its source code will be released in the future.

A blockchain ledger for securing isolated ambient intelligence deployments using reputation and information theory metrics

Ambient Intelligence deployments are very vulnerable to Cyber-Physical attacks. In these attacking strategies, intruders try to manipulate the behavior of the global system by affecting some key elements within the deployment. Typically, attackers inject false information, integrate malicious devices within the deployment, or infect communications among sensor nodes, among other possibilities. To protect Ambient Intelligence deployments against these attacks, complex data analysis algorithms are usually employed in the cloud to remove anomalous information from historical series. However, this approach presents two main problems. First, it requires all Ambient Intelligence systems to be networked and connected to the cloud. But most new applications for Ambient Intelligence are supported by isolated systems. And second, they are computationally heavy and not compatible with new decentralized architectures. Therefore, in this paper we propose a new decentralized security solution, based on a Blockchain ledger, to protect isolated Ambient Intelligence deployments. In this ledger, new sensing data are considered transactions that must be validated by edge managers, which operate a Blockchain network. This validation is based on reputation metrics evaluated by sensor nodes using historical network data and identity parameters. Through information theory, the coherence of all transactions with the behavior of the historical deployment is also analyzed and considered in the validation algorithm. The relevance of edge managers in the Blockchain network is also weighted considering the knowledge they have about the deployment. An experimental validation, supported by simulation tools and scenarios, is also described. Results show that up to 93% of Cyber-Physical attacks are correctly detected and stopped, with a maximum delay of 37 s.

Ontology-based hybrid commonsense reasoning framework for handling context abnormalities in uncertain and partially observable environments

Ambient intelligence (AmI) systems aim to provide users with context-aware assistance services intended to improve the quality of their lives in terms of autonomy, safety, and well-being. Taking the uncertainty and partial observability of these environments into account is of major importance for context recognition and, more specifically, to detect and solve context abnormalities such as those related to the user's behavior or those related to context attribute prediction. In this paper, an ontology-based framework integrating machine learning and probabilistic planning within commonsense reasoning is proposed to recognize the user's context and abnormalities associated with it. The reasoning is performed using event calculus in answer set programming (ECASP); ECASP allows for abductive and temporal reasoning, which results in an eXplainable AI (XAI) approach. A context ontology is proposed to axiomatize the reasoning and introduce the notion of probabilistic fluents into the EC formalism in order to perform probabilistic reasoning. The reasoning incorporates probabilistic planning based on a partially observable Markov decision process (POMDP) to solve knowledge incompleteness. To evaluate the proposed framework, real-life scenarios, based on the Orange4Home and SIMADL public datasets are implemented and discussed.

XACML-based semantic rules language and ontological model for reconciling semantic differences of access control rules

Internet of things interconnects increasing numbers of artefacts and individuals, allowing the setup of ambient intelligence systems in multi-domains (e.g., hospitals). Designing a suitable architectural framework is required to ensure the efficiency and sustainability of the implementation of such systems. Consolidating public services about citizens' safety and authorisation decisions when a resource is accessed in an open-dynamic environment are the main challenges that can be highlighted. Therefore, the semantic heterogeneity between the local policies of the different domains is a crucial lock for implementing this process. Our approach goes beyond the semantic web languages' weaknesses by combining the XACML-based security policy model with a semantic rules language developed during the European SembySem Project. Built on top of RDF(S), the proposed model aims to abstract the security implementation, reconcile semantic differences across multi-domain, and maintain the local security policy. Moreover, this model addresses the semantic heterogeneity of sensors' data during knowledge-sharing.

Fast Temporal Graph Convolutional Model for Skeleton-Based Action Recognition.

Human action recognition has a wide range of applications, including Ambient Intelligence systems and user assistance. Starting from the recognized actions performed by the user, a better human–computer interaction can be achieved, and improved assistance can be provided by social robots in real-time scenarios. In this context, the performance of the prediction system is a key aspect. The purpose of this paper is to introduce a neural network approach based on various types of convolutional layers that can achieve a good performance in recognizing actions but with a high inference speed. The experimental results show that our solution, based on a combination of graph convolutional networks (GCN) and temporal convolutional networks (TCN), is a suitable approach that reaches the proposed goal. In addition to the neural network model, we design a pipeline that contains two stages for obtaining relevant geometric features, data augmentation and data preprocessing, also contributing to an increased performance.

SeSAME: Re-identification-based ambient intelligence system for museum environment

• SeSAME system for collecting and analysing the behaviours of visitors inside a museum environment. • SeSAME is based on a multi-modal deep neural network architecture. • SeSAME uses re-identification techniques to perform visual profiling of visitor interest. • SeSAME aims at developing a tailored user experience in museum environment. Nowadays, understanding and analysing visitors activities and behaviours is becoming imperative for personalising and improving the user experience in a museum environment. Users’ behaviour can provide important statistics, insights and objective information about their interactions, such as attraction, attention and action. These data represent a precious value for the museum curators, and they are one of the parameters that need to be assessed. These information are collected through manual approaches based on questionnaires or visual observations. This procedure is time consuming and can be affected by the subjective interpretation of the evaluator . From such premises, SeSAME (Senseable Self Adapting Museum Environment) a novel system for collecting and analysing the behaviours of visitors inside a museum environment is presented in this paper. SeSAME is based on a multi-modal deep neural network architecture able to extract anthropometric and appearance features from RGB-D videos acquired in crowded environments. Our approach has been tested on four different temporal modelling methods to aggregate a sequence of image-level features into clip-level features. This paper uses as a benchmark TVPR2, a public dataset of acquired videos with an RGB-D camera in a top-view configuration, in the presence of persistent and temporarily heavy occlusion. Moreover, a dataset specifically collected for this work has been acquired in a real museum environment, which is Palazzo Buonaccorsi, an important historical building in Macerata, in Marche Region in the center of Italy. During the experimental phase, the evaluation metrics show the effectiveness and the suitability of the proposed method.

Cybersecurity Issues of IoT in Ambient Intelligence (AmI) Environment

Ambient intelligence (AmI) refers to the ability of devices to interact seamlessly with their surroundings. An increased use of IoT in ambient intelligence has led to a heightened concern for cybersecurity. Vulnerabilities in the software or firmware of IoT devices have been exploited by hackers to gain control of the devices or the networks. They could also use ambient intelligence systems to collect sensitive data from IoT devices. The problem of IoT security is twofold. First, it concerns the safety of the embedded IoT system and the protection of embedded data. To protect the devices, it's essential to understand the various types of attacks that are possible and to deploy appropriate security measures. This article addresses these issues by providing an overview of potential security risks and the latest solutions and approaches that can help to tackle these problems.

Intent-Aware Interactive Internet of Things for Enhanced Collaborative Ambient Intelligence

The Internet of Things (IoT) enables the connection of a broad range of artifacts with advanced sensory technologies and produces massive amounts of data to support ambient intelligence. While the potential of IoT systems is widely recognized, little work has demonstrated a system with the ability to execute autonomously in the real world. Inspired by the success of robotics in specialized IoT environments, we propose an end-to-end solution for a generic, interactive ambient intelligence system where robots can assist humans in conducting activities in IoT-enabled smart homes. We evaluate the solution using implementations of public benchmarks on open-source platforms and use several activities to demonstrate the effectiveness of the proposed solution in real life.

Multi-Agent Interaction to Assist Visually-Impaired and Elderly People.

A voice-controlled smart home system based on conversational agents can address the specific needs of older people, proactively providing support, compensating for cognitive decline, and coping with solitude, among other features. In particular, Multi-Agent Systems (MAS) platforms provide considerable support for complex adaptive systems that are naturally distributed and situated in dynamic environments, such as Ambient intelligence (AmI) applications. Such autonomous intelligent agents are capable of independent reasoning and joint analysis of complex situations to support high-level interaction with humans, besides providing typical characteristics of MAS, such as cooperation and coordinated action. In this context, we developed an approach using a MAS previously evaluated for visually impaired users, where most of the system’s functionalities are also helpful for the elderly. Our methodology is based on the four steps of the interactive design process. As a result, we determined that our approach has elements that allow for natural interaction with users, and we identified and discussed improvements and new features for future work. We believe that our findings can point to directions for building AmI systems that are capable of more natural interaction with users.

Preserving Narrative Identity for Dementia Patients: Embodiment, Active Environments, and Distributed Memory

One goal of this paper is to argue that autobiographical memories are extended and distributed across embodied brains and environmental resources. This is important because such distributed memories play a constitutive role in our narrative identity. So, some of the building blocks of our narrative identity are not brain-bound but extended and distributed. Recognising the distributed nature of memory and narrative identity, invites us to find treatments and strategies focusing on the environment in which dementia patients are situated. A second goal of this paper is to suggest various of such strategies, including lifelogging technologies such as SenseCams, life story books, multimedia biographies, memory boxes, ambient intelligence systems, and virtual reality applications. Such technologies allow dementia patients to remember their personal past in a way that wouldn’t be possible by merely relying on their biological memory, in that way aiding in preserving their narrative identity and positively contributing to their sense of well-being.

Ambient Intelligence System Enabling People With Blindness to Develop Electrotechnical Components and Their Drivers

Along with the development of new assistive technologies, including ambient intelligence (AmI) environments, new job opportunities are created for people with blindness or visual impairment. Whereas research on software development for people with blindness has been conducted since the 1960s, the design and development of electrotechnical equipment still lacks any way to compensate for the disability which would enable people with blindness to perform even the activities pertaining to this field. This article aims to introduce these new technological procedures enabling the compensation of the disability of people with blindness when designing and developing electrotechnical components (or prototypes) and their drivers by using the AmI system RUDO modules and other tools. This includes the modules enabling measuring electrotechnical quantities, tracing of oscilloscope curves, and ensuring a unified user interface for programming and operation of the devices connected. Other approaches are introduced herein that focused on the production of mechanical components for device construction. This article also contains an illustrative video of practical use of these technological procedures by a person with blindness. To assess the usability of the designed technological procedures to the work of a designer with blindness and one without visual impairment, the cognitive walkthrough method was used. The main contribution of this article is to broaden the knowledge base by the principles of involving people with blindness in the development and construction of electrotechnical components (or prototypes). These new possibilities can be used practically, for instance, in computer education, which can offer new curricula for visually impaired students and focused on more practical issues where hardware and software approaches meet.

Improving the Design of Ambient Intelligence Systems: Guidelines Based on a Systematic Review

ABSTRACT Ambient intelligence (AmI) emerged in Europe with the idea that the computational support available in a living environment can help people’s lives. AmI introduces new challenges and critical issues because this technology differs from the traditional form of interaction that is centered on a device or system. In this work, we aim to contribute to this topic by conducting a systematic literature review in order to identify Human-Computer Interaction guidelines for the design of ambient intelligence systems. We found a total of 120 guidelines from 27 papers, and we grouped similar guidelines creating different categories. These categories of guidelines later became a unified guideline incorporating also some of our own ideas. As a result, this paper identifies ten categories and guidelines to improve user interaction with ambient intelligence systems. We believe that these guidelines significantly contribute to designing more intuitive AmI systems for users, including those with disabilities.

Auditing images collected by sensors in ambient intelligence systems with privacy and high efficiency

To determine whether sensors should transmit collected images back to the data center, an auditing protocol is desired to check these images before transmission. Since images may contain sensitive information of the environment, privacy is an essential requirement for such an auditing protocol. Moreover, since sensors are typically low-power devices and the data center has to handle a variety of sensors, this auditing protocol should be lightweight. Taking both security and efficiency into account, we propose a novel auditing protocol on images in ambient intelligence systems, called AP-AmI (i.e., Auditing Protocol in Ambient Intelligence Systems). In AP-AmI, we use the local binary pattern technique for extracting features from images and design a novel privacy-preserving Euclid distance computation algorithm for determining whether these collected images are useful. Since these two techniques are both lightweight, AP-AmI can achieve high efficiency. At the same time, since feature vectors cannot be extracted by adversaries, AP-AmI can satisfy the privacy requirement. Experimental results show AP-AmI is feasible for real-world applications.

Providing music service in Ambient Intelligence: experiments with gym users

Ambient Intelligence (AmI) is an interdisciplinary research area of ICT which has evolved since the 90s, taking great advantage from the advent of the Internet of Things (IoT). AmI creates, by using Artificial Intelligence (AI), an intelligent ecosystem in which computers, sensors, lighting, music, personal devices, and distributed services, work together to improve the user experience through the support of natural and intuitive user interfaces. Nowadays, AmI is used in various contexts, e.g., for building smart homes and smart cities, providing healthcare, and creating an adequate atmosphere in retail and public environments.In this paper, we propose a novel AmI system for gym environments, named Gym Intelligence, able to provide adequate music atmosphere, according to the users’ physical effort during the training. The music is taken from Spotify and is classified according to some music features, as provided by Spotify itself. The system is based on a multi-agent computational intelligence model built on two main components: (i) machine learning methods that forecast appropriate values for the Spotify music features, and (ii) a multi-objective dynamic genetic algorithm that selects a specific Spotify music track, according to such values. Gym Intelligence is built by sensing the ambient with a minimal, low-cost, and non-intrusive set of sensors, and it has been designed considering the outcome of a preliminary analysis in real gyms, involving real users. We have considered well-known regression methods and we have validated them using a collected data (i) about the users’ physical effort, through the sensors, and (ii) about the users’ music preferences, through an Android app that the users have used during the training. Among the regression methods considered, the one that provided the best results is the Random Forest, which predicted Spotify music features with a mean absolute error of 0.02 and a root mean squared error of 0.05. We have implemented Gym Intelligence and deployed it in five real gyms. We have evaluated it conducting several experiments. The experiments show how, with the help of Gym Intelligence, the users’ satisfaction about the provided background music, rose from 3.05 to 4.91 (on a scale from 1 to 5, where 5 is the maximum score).

Predicting interaction design patterns for designing explicit interactions in ambient intelligence systems: a case study

Ambient intelligence (AmI) focuses on supporting people by designing sensitive and responsive environments to context through implicit and explicit interactions. Explicit interactions in AmI systems have requirements specific to making interactions robust, smooth, intuitive, and reliable. Based on requirements, the designers can detect and eliminate faults from the beginning of the design process and understand the users’ needs and demands. This work presents a UIPatternM model for predicting interaction design patterns from processing text-based requirements through machine learning algorithms. We evaluate the predictions of our proposal. We also present a case study with professional designers who evaluated the UIPatternM recommender predictions according to a set of design-level requirements that emulate everyday needs. Our participants performed a set of tasks based on scenarios, and we evaluated the participants’ using effectiveness, efficiency, and satisfaction as performance metrics. Applying the UIPatternM model helped to endorse the conception and refinement of user interface design for explicit interaction in AmI systems.

An IoT-based infrastructure to enhance self-evacuations in natural hazardous events

Regardless of the extensive research conducted on large-scale evacuations, the instrumentation of these processes still represents an open issue for first response organizations. Self-evacuation of civilians that follows evacuation plans has shown to be feasible as early response to several natural disasters; however, the typical lack of interaction capability of the evacuees with first response organizations and emergency managers jeopardizes the effectiveness of these processes. This article proposes an IoT-based infrastructure that supports self-evacuation of civilians in mass processes, allowing people to participate as information providers and consumers. This infrastructure is the backbone of an ambient intelligence system used as a bridge between evacuees, first response units, and the emergency operation center managing the process. Depending on the people location and the status of the area, the system implements breadcrumbs that guide people to shelters and safe places. The proposed infrastructure includes (1) an architecture that captures the core design aspects of the solution and makes it reusable for other researchers, (2) an implementation of the system based on Raspberry Pi 3 devices with LoRa radio connectivity, (3) a mobile application that allows evacuees to interact with the evacuation system, and (4) a simplified algorithm to support the deployment of the IoT-based infrastructure into an urban area. The solution was evaluated using real measurements and simulations, and the obtained results are highly encouraging.

Design and Implementation of Tracking a user’s Behavior in a Smart Home

A smart home is automated structures with control contraptions and presented area; it comes with advanced sensing and automation systems to supply the inhabitants with monitoring and management no matter if they are in the home or outside. In light of the developments that are taking place in the smart home, machine learning, and the needs of the home, which makes the appropriate decisions based on the user’s behavior of the smart home, and the use of development in technology to facilitate the general requirements of people. Modern smart home systems tend to be standalone solutions that focus on lifestyle needs using a smart control system, such as lighting and temperature control. Ubiquitous home expects the emergence where sophisticated (Ambient intelligent) systems monitor and learn user’s behavior and lifestyles and enable the home to predict and respond to all the needs and activities of home occupants. During perception, residents perform their daily routine while the in-house embedded sensors transmit and store the readings in a database. By doing so, the intelligent agent uses these readings to generate knowledge, such as patterns and trends. Thus, it can predict based on the built knowledge to enable the smart home to identify and automate the action that meets the occupants’ aims.

AnAbEL: towards empowering people living with dementia in ambient assisted living

Ambient assisted living aims to support the well-being of people with special needs by offering assistive solutions. Systems focused on dementia increase the autonomy of people living with dementia by monitoring their activities. Topics such as activity recognition and specific solutions such as reminders and tracking users by Global Positioning System offer great advances in user safety and help them preserve a healthier lifestyle. However, these solutions are often addressed to secondary parties, providing them activity logs or alerts, but excluding the main user, the person living with dementia. Although the primary users are taken into consideration at some design stages using user-centred design frameworks, the final products tend to not fully address user needs. This paper presents an ambient intelligent system aimed at reducing this limitation by providing reminders and advice to the person living with dementia in the first instance. The system still involves caregivers if unusual or unhealthy behaviour continues. The solution is deployed in order to be validated by professionals from London city boroughs who work in housing and dementia related services, with an emphasis on enhancing healthy lifestyles by empowering the user in the early stages of dementia with autonomy. Through continued activity monitoring in real-time, the system can provide reminders and warnings to users to keep healthy routines. Continuous monitoring provides user behaviour tracking, and the context-aware logic used involves caregivers through alerts when necessary to ensure user safety. This article describes the process followed in developing the system, and covers previous concerns and practical feedback from health professionals over the deployment of the system in a real environment. Our approach also includes a novel indoor localization system to distinguish users and allows a more specific delivery of services in multi-occupancy scenarios.