Intelligent Assistance Systems Research Articles

The SmartHabits: An Intelligent Privacy-Aware Home Care Assistance System.

Many researchers and product developers are striving toward achieving ICT-enabled independence of older adults by setting up Enhanced Living Environments (ELEs). Technological solutions, which are often based on the Internet of Things (IoT), show great potential in providing support for Active Aging. To enhance the quality of life for older adults and overcome challenges in enabling individuals to achieve their full potential in terms of physical, social, and mental well-being, numerous proof-of-concept systems have been built. These systems, often labeled as Ambient Assisted Living (AAL), vary greatly in targeting different user needs. This paper presents our contribution using SmartHabits, which is an intelligent privacy-aware home care assistance system. The novel system comprising smart home-based and cloud-based parts uses machine-learning technology to provide peace of mind to informal caregivers caring for persons living alone. It does so by learning the user’s typical daily activity patterns and automatically issuing warnings if an unusual situation is detected. The system was designed and implemented from scratch, building upon existing practices from IoT reference architecture and microservices. The system was deployed in several homes of real users for six months, and we will be sharing our findings in this paper.

The study of the influence of intelligent driver assistance systems on the capacity of urban roads

The article gives theoretical and experimental studies of the time delay of cars in the stream, due to the time of the driver’s reaction to changing traffic conditions. In the present study, it was switching of allowing signal of the traffic light which was assigned as the initial moment of the measurements. The observation showed the dependence of the delay time of the car on its position in the queue at the junction with traffic light regulation. Due to the presence of the human factor, the accumulated delay along the flow has had a great influence on the number of vehicles which had the opportunity to cross the junction within one traffic light cycle. A situation was simulated in which all vehicles in a stream were equipped with an assistant driver device when moving in a traffic jam. The value of the delay of the vehicle in this case depends not on the actions of a single person, but on the electronics. It was found that the delay of the car when using electronics is 5.44 times less than the delay caused by the human factor. The study was conducted in a part of Nadezhda Suslova street in the city of Nizhny Novgorod and showed the possibility to increase the speed and intensity of the considered actual traffic flow.

Speech enhancement for in‐vehicle voice control systems using wavelet analysis and blind source separation

Multiple sources of interference and low signal-to-interference ratio are two major challenges to speech-based intelligent driver assistant systems. They will have a serious impact on the performance of voice control commands. To solve this problem, this study proposes a speech enhancement method based on wavelet analysis and blind source separation in a complicated automobile environment. Firstly, according to the characteristics of typical speech signals, an automatic selection method of optimal wavelet basis is given to optimise the signal denoising performance. Secondly, the mixed signals are separated by a complex fast-independent component analysis (ICA) algorithm, and then the inverse short-time Fourier transform is utilised to obtain the separated signals in time domain. Finally, experiments are carried out to demonstrate the effectiveness of the proposed method. Results show that its performance in terms of a correlation coefficient can be improved by about 7% compared with that of the conventional method only using fast-ICA.

Intelligent Imaging Technology in Diagnosis of Colorectal Cancer Using Deep Learning

In order to explore the application of deep learning based intelligent imaging technology in the diagnosis of colorectal cancer, Tangdu Hospital patients are selected as the research object in this study. By scanning the cancer sites, then distinguishing and extracting the features of the tumors, the collected data are input into the designed in-depth learning intelligent assistant diagnosis system for comparison. The results show that in the analysis of image prediction accuracy, the best prediction accuracy of T1-weighted image method is matrix GLCM (gray level co-occurrence matrix) algorithm, the best prediction accuracy of adding T1-weighted image method is matrix MGLSZM (multi-gray area size matrix) algorithm, and the best prediction accuracy of T2-weighted image method is ALL combination of all texture features, and the best prediction accuracy of three imaging sequences is not more than 0.8. In the AUC analysis of the area under the curve of different texture features, it is found that T2-weighted imaging method has obvious advantages in differentiating colorectal cancer from other methods. Therefore, through this study, it is found that in the use of deep learning intelligent assistant diagnosis system for the diagnosis of colorectal cancer, it can provide useful information for the clinical diagnosis of colorectal cancer to a certain extent. Although there are some deficiencies in the research process, it still provides experimental basis for the diagnosis and treatment of colorectal cancer in later clinical stage.

Intelligent Material Supply Supporting Assistive Systems for Manual Working Stations

The production of customized products in small batch sizes represents a major challenge for the industry. Especially the assembly environment is affected due to less possibilities for economical automation, thus manual assembly remains an important part of production. Assistive systems for manual assembly aim at minimizing the challenges associated with a customer driven production. These systems support human workers by providing manufacturing information, highlighting material boxes or identifying assembly errors. To do so assistive systems need to be user-centered and allow a flexible adaption to new products or assembly processes. With our approach, we present a concept that uses RFID technology and sensory input to enable an intelligent material supply system. This enables an automatic identification and data capture (AIDC) for assistive systems. Existing materials at the working station, including position and numerical quantity can be recognized and matched with the production order. Advantages and potential outcomes for assistive systems, like autonomous pull request for material (Kanban), are discussed in this paper. Furthermore, we describe the structure and implementation of our concept using the example of the manual working station of SmartFactoryKL. The development of the system, the achieved improvements as well as identified challenges for assistive systems will be presented. The prospects for further developments and their potential will be discussed.

Detection of Control Structures in Spoken Utterances

State-of-the-art intelligent assistant systems such as Siri and Cortana do not consider control structures in the user input. They reliably react to ordinary commands. However, their architectures are not designed to cope with queries that require complex control flow structuring. We propose a system to overcome these limitations. Our approach models if-then-else, loop, and concurrency constructs in spoken utterances explicitly. The model bridges the gap between linguistic and programmatic semantics.To demonstrate our concept, we apply a rule-based approach. We have implemented three prototypes that use keyphrases to discover potential control structures depending on the type of control structure. However, the full structures are determined differently. For conditionals we use chunk and part-of-speech (POS) tags provided by natural language processing tools; for loops and concurrency we make use of an action extraction approach based on semantic role labeling (SRL). Additionally, we use coreference information to determine the extent of the respective structure.The explicit modeling of conditionals, loops, and concurrent sections allows us to evaluate the accuracy of our approaches independently from each other and from other language understanding tasks. We have conducted two user studies in the domain of humanoid robotics. The first focused on conditionals. Our prototype achieves F1scores from 0.783 (automatic speech recognition) to 0.898 (manual transcripts) on unrestricted utterances. In the second, the prototypes for loop and concurrency detection also proved useful. F1scores range from 0.588 (automatic speech recognition) to 0.814 (manual transcripts) for loops and from 0.622 (automatic speech recognition) to 0.842 (manual transcripts) for concurrent sections respectively.

Identification of driver’s braking intention based on a hybrid model of GHMM and GGAP-RBFNN

Driving intention has been widely used in intelligent driver assistance systems, automated driving systems, and electric vehicle control strategies. The accuracy, practicality, and timeliness of the driving intention identification model are its key issues. In this paper, a novel driver’s braking intention identification model based on the Gaussian mixture-hidden Markov model (GHMM) and generalized growing and pruning radial basis function neural network (GGAP-RBFNN) is proposed to improve the identification accuracy of the model. The simplest brake pedal and vehicle speed data that are easily obtained from the vehicle are used as an observation sequence to improve practicality of the model. The data of the pressing brake pedal stage are used to identify the braking intention to improve the timeliness of the model. The experimental data collected from real vehicle tests are used for off-line training and online identification. The research results show that the accuracy of driver’s braking intention identification model based on the GHMM/GGAP-RBFNN hybrid model is 94.69% for normal braking and 95.57% for slight braking, which are, respectively, 26.55% and 17.72% higher than achieved by the GHMM. In addition, the data of the pressing brake pedal stage are used for intention identification, which is 1.2 s faster than that of the existing identification model based on the GHMM.

Design of music training assistant system based on artificial intelligence

In order to improve the input accuracy and response speed of music training, this paper designs an intelligent assistant system. The architecture is divided into infrastructure layer, data layer, application layer and presentation layer. In the hardware design, the combination of ARM and digital s

Problems of the application of intelligent driver assistance systems on a single-track vehicles

This article considers some problems and features of the application of modern systems of driver assistance on single-track vehicles (“STV”). It is also mentioned the classification of intelligent driver assistance systems and design features of the “STV” that require an improvement for their successful application on such vehicle. There is also described perspective of systems’ development that helps the “STV” driver. Also perspective the “STV” types with high safety construction are considered in this article.

An intelligent emergency rescue assistance system for mountaineers

An intelligent emergency rescue assistance system for mountaineers

An intelligent emergency rescue assistance system for mountaineers

Mountaineering is one of main recreational activities of modern people. The mountain accidents include mountain sickness, hypothermia, missing, accidental fall and so on. This paper proposes an intelligent emergency rescue assisted mountaineering system, including a wearable device and a management platform. The physiological signals are monitored for mountain sickness and hypothermia, asking the user to take precautionary measures in advance. In the missing and accidental fall scenarios, the search and rescue team is provided with timely information, and the wearable device is switched to emergency beacon signal automatically, helpful to increase the efficiency of search and rescue. Added to this, the mobile device is integrated with Bluetooth, the mobile device can send the geo information and the information on the wearable device to the management system by means of its communication capability. When an accident happens, the mobile device and wearable device send the beacon signal crosswise, the power usage rate is maximised effectively, the search and rescue time is prolonged. This system architecture effectively prevents mountain accidents, indicates the location and informs related units to provide emergency rescue at the soonest.

Intelligent assistant carer for active aging

We present the concept of an Intelligent Assistant Carer system for the elderly, designed to help with active aging and to facilitate the interactions with carers. The system is modular, allowing the users to choose the appropriate functions according to their needs, and is built on an open platform in order to make it compatible with third-party products and services. Currently, the system consists of a wearable device (a smartwatch) and an internet portal that manages the data and takes care of the interactions between the user, the carers, and the support services. We present in detail one of the modules, i.e., fall detection, and the results of a pilot study for the system on 150 users over the course of 3 months.

OR.NET RT: how service-oriented medical device architecture meets real-time communication.

Today's landscape of medical devices is dominated by stand-alone systems and proprietary interfaces lacking cross-vendor interoperability. This complicates or even impedes the innovation of novel, intelligent assistance systems relying on the collaboration of medical devices. Emerging approaches use the service-oriented architecture (SOA) paradigm based on Internet protocol (IP) to enable communication between medical devices. While this works well for scenarios with no or only soft timing constraints, the underlying best-effort communication scheme is insufficient for time critical data. Real-time (RT) networks are able to reliably guarantee fixed latency boundaries, for example, by using time division multiple access (TDMA) communication patterns. However, deterministic RT networks come with their own limitations such as tedious, inflexible configuration and a more restricted bandwidth allocation. In this contribution we overcome the drawbacks of both approaches by describing and implementing mechanisms that allow the two networks to interact. We introduce the first implementation of a medical device network that offers hard RT guarantees for control and sensor data and integrates into SOA networks. Based on two application examples we show how the flexibility of SOA networks and the reliability of RT networks can be combined to achieve an open network infrastructure for medical devices in the operating room (OR).

Work process and task-based design of intelligent assistance systems in German textile industry

The mid-sized embossed German textile industry must face social challenges e.g. demographic change or technical changing processes. Interaction with intelligent systems (on machines) and increasing automation changes processes, working structures and employees’ tasks on all levels. Work contents are getting more complex, resulting in the necessity for diversified and enhanced competencies. Mobile devices like tablets or smartphones are increasingly finding their way into the workplace. Employees who grew up with new forms of media have certain advantages regarding the usage of modern technologies compared to older employees. Therefore, it is necessary to design new systems which help to adapt the competencies of both younger and older employees to new automated production processes in the digital work environment. The key to successful integration of technical assistance systems is user-orientated design and development that includes concepts for competency development under consideration of, e.g., ethical and legal aspects.

Fast care – real-time sensor data analysis framework for intelligent assistance systems

Abstract Main target of fast care is the development of a real-time capable sensor data analysis framework for intelligent assistive systems in the field of Ambient Assisted Living, eHealth, Tele Rehabilitation, and Tele Care. The aim is to provide a medically valid integrated situation model based on a distributed, ad-hoc connected, energy-efficient sensor infrastructure suitable for daily use. The integrated situation model combining physiological, cognitive, and kinematic information about the patient is grounded on the intelligent fusion of heterogeneous sensor data on different levels. The model can serve as a tool for quickly identifying risk and hazards as well as enable medical assistance systems to autonomously intervene in real-time and actively give telemedical feedback.

An Intelligent Computer Assistance System for Artifact Restoration Based on Genetic Algorithms with Plane Image Features

Restoration work of archaeological artifacts broken into pieces is similar to putting together a jigsaw puzzle. The purpose of this study is to construct an intelligent computer assistance system to conveniently restore archaeological discoveries from some fragments. AReal-Coded Genetic Algorithm (RCGA) was applicable for solving the positioning problem of a three-dimensional (3D) restoration. The fitness function value for RCGA was calculated from image similarity between the target and correct patterns in plane images at multiple camera angles. Image features of a 3D object were obtained by the ORB (Oriented FAST and Rotated BRIEF), BRISK (Binary Robust Invariant Scalable Keypoints), and Accelerated KAZE (AKAZE) techniques; they were considered as a part of the fitness function value. Simulation study revealed that the RCGA approach was capable of automatically and efficiently adjusting the positions of 3D fragments, especially in the AKAZE technique. A user interface with the functions of design drawing was also created to assist in repair work. The interactive assistance interface for 3D restoration based on RCGA and followed by the hill-climbing algorithm would be applied to practical applications for digital archives of artifacts.

Robotic learning from demonstration of therapist's time-varying assistance to a patient in trajectory-following tasks.

The number of people with physical disabilities and impaired motion control is increasing. Consequently, there is a growing demand for intelligent assistive robotic systems to cooperate with people with disability and help them carry out different tasks. To this end, our group has pioneered the use of robot learning from demonstration (RLfD) techniques, which eliminate the need for task-specific robot programming, in robotic rehabilitation and assistive technologies settings. First, in the demonstration phase, the therapist (or in general, a helper) provides an intervention (typically assistance) and cooperatively performs a task with a patient several times. The demonstrated motion is modelled by a statistical RLfD algorithm, which will later be used in the robot controllers to reproduce a similar intervention robotically. In this paper, by proposing a Tangential-Normal Varying-Impedance Controller (TNVIC), the robotic manipulator not only follows the therapist's demonstrated motion, but also mimics his/her interaction impedance during the therapeutic/assistive intervention. The feasibility and efficacy of the proposed framework are evaluated by conducting an experiment involving a healthy adult with cerebral palsy symptoms being induced using transcutaneous electrical nerve stimulation.

A smartphone based technique to monitor driving behavior using DTW and crowdsensing

Safety issues while driving in smart cities are considered to be top-notch priority in contrast to traveling. Today’s fast paced society, often leads to accidents. In order to reduce the road accidents, one key area of research is monitoring the driving behavior of drivers. Understanding the driver behavior is an essential component in Intelligent Driver Assistance Systems. One of potential cause of traffic fatalities is aggressive driving behavior. However, drivers are not fully aware of their aggressive actions. So, in order to increase awareness and to promote driver safety, a novel system has been proposed. In this work, we focus on DTW based event detection technique, which have not been researched in motion sensors based time series data to a great extent. Our motivation is to improve the classification accuracy to detect sudden braking and aggressive driving behaviors using sensory data collected from smartphone. A very significant feature of DTW is to be able to automatically cope with time deformations and different speeds associated with time-dependent data which makes it suitable for our chosen application where data might get affected due to factors such as: high variability in road and vehicle conditions, heterogeneous smartphone sensors, etc. Our technique is novel as it uses fusion of sensors to enhance detection accuracy. The experimental results show that proposed algorithm outperforms the existing machine learning and threshold-based techniques with 100% detection rate of braking events and 97% & 86.67% detection rate of normal left & right turns and aggressive left & right turns respectively.

Trust in haptic assistance: weighting visual and haptic cues based on error history

To effectively interpret and interact with the world, humans weight redundant estimates from different sensory cues to form one coherent, integrated estimate. Recent advancements in physical assistance systems, where guiding forces are computed by an intelligent agent, enable the presentation of augmented cues. It is unknown, however, if cue weighting can be extended to augmented cues. Previous research has shown that cue weighting is determined by the reliability (inversely related to uncertainty) of cues within a trial, yet augmented cues may also be affected by errors that vary over trials. In this study, we investigate whether people can learn to appropriately weight a haptic cue from an intelligent assistance system based on its error history. Subjects held a haptic device and reached to a hidden target using a visual (Gaussian distributed dots) and haptic (force channel) cue. The error of the augmented haptic cue varied from trial to trial based on a Gaussian distribution. Subjects learned to estimate the target location by weighting the visual and augmented haptic cues based on their perceptual uncertainty and experienced errors. With both cues available, subjects were able to find the target with an improved or equal performance compared to what was possible with one cue alone. Our results show that the brain can learn to reweight augmented cues from intelligent agents, akin to previous observations of the reweighting of naturally occurring cues. In addition, these results suggest that the weighting of a cue is not only affected by its within-trial reliability but also the history of errors.

Multidimensional assessment of challenging behaviors in advanced stages of dementia in nursing homes—The insideDEM framework

IntroductionAssessment of challenging behaviors in dementia is important for intervention selection. Here, we describe the technical and experimental setup and the feasibility of long-term multidimensional behavior assessment of people with dementia living in nursing homes. MethodsWe conducted 4 weeks of multimodal sensor assessment together with real-time observation of 17 residents with moderate to very severe dementia in two nursing care units. Nursing staff received extensive training on device handling and measurement procedures. Behavior of a subsample of eight participants was further recorded by videotaping during 4 weeks during day hours. Sensors were mounted on the participants' wrist and ankle and measured motion, rotation, as well as surrounding loudness level, light level, and air pressure. ResultsParticipants were in moderate to severe stages of dementia. Almost 100% of participants exhibited relevant levels of challenging behaviors. Automated quality control detected 155 potential issues. But only 11% of the recordings have been influenced by noncompliance of the participants. Qualitative debriefing of staff members suggested that implementation of the technology and observation platform in the routine procedures of the nursing home units was feasible and identified a range of user- and hardware-related implementation and handling challenges. DiscussionOur results indicate that high-quality behavior data from real-world environments can be made available for the development of intelligent assistive systems and that the problem of noncompliance seems to be manageable. Currently, we train machine-learning algorithms to detect episodes of challenging behaviors in the recorded sensor data.