Intelligent Assistance Systems Research Articles

Themes identified and lessons learned through the development of intelligent environments that support healthy well-being

The dynamic, adaptive, and autonomous nature of intelligent environments makes them ideal for supporting people's health and well-being. The Intelligent Assistive Technology and Systems Lab IATSL and their collaborators are a team of engineers, computer scientists, and rehabilitation specialists who have been working with end users to develop devices and systems for supporting people in home and institutional environments for over a decade. This paper presents key themes for supporting the development of health-centric intelligent environments and systems. Examples of the themes are illustrated through several case studies of real-world applications developed through IATSL's research. The paper concludes by high-lighting some areas that need to be considered by the field as intelligent environments move out of the research lab and into people's homes.

Mobility assistive devices and self-transfer robotic systems for elderly, a review

When mobility degrades with age, it is of great significance to develop devices which can support the elderly in their day-to-day life. With the usage of intelligent assistive robotic systems, elderly population can lead a better quality of life independently. This article is a review of various assistive devices for elderly focussing on mobility and self-transfer systems. The practical difficulties in walking and moving from bed to wheel chair or wheel chair to toilet seat affect the daily activities of aged people. Depending on caregivers to access toilets affects one's dignity. The review covers various advances that have been evolved in this area of research and addresses the limitations to be overcome.

An intelligent integrative assistive system for dyslexic learners

Purpose – The objective of this paper is to present a comprehensive literature survey on dyslexic learners and to propose a model for integrated assistive technology of dyslexic learners. Design/methodology/approach – The use of the proposed model through real-life scenarios categorized as “phonological, reading and writing scenarios”. We have also surveyed some systems for use with dyslexic learners currently in use and have compared them on the basis of number of barriers, technological innovation, age group and fostering. Findings – Dyslexic learners are characterized by slow learning, poor handwriting, poor spelling skills and difficulties in planning, organizing, revising and editing texts; technology plays a major role in the educational environment; it has become crucial in impacting knowledge across the globe; and open research issues and challenges that have to be addressed in the design of the current dyslexic system have been presented in detail. Research limitations/implications – Full implementation of the proposed model and its application in developing countries. Practical implications – The system improves the phonological awareness, reading and writing skills of dyslexic learners and it provides a solution for children as well as adults with dyslexia. Social implications – This survey can be used as a reference guide to understand learning barriers and intelligent systems for dyslexic learners and to promote the use of these in schools for those with learning disabilities who need assistive technologies. Originality/value – Development of a newly proposed integrated intelligent assistive system for dyslexic learners; knowledge generation as a reference guide to understand dyslexic learning ability in general and an intelligent assistive dyslexic system in particular; and survey of open research issues and challenges to further research in intelligent dyslexic systems.

Application of global positioning system and questionnaires data for the study of driver behaviour on two‐lane rural roads

Methodologies based on naturalistic observation provide the most accurate data for studying drivers' behaviour. This study presents a new methodology to obtain naturalistic data related to drivers' behaviour in a road segment. It is based on the combination of using global positioning system data and drivers' questionnaires. The continuous speed profiles along a road segment and the characteristics of drivers, of their trips and the type of their vehicles can be obtained for a great amount of drivers. It has already been successfully used for several studies, such as the development of models to estimate operating speed profile in two-lane rural road segments; or the characterisation of driving styles. These operating speed models have been the key for the development of a new geometric design consistency model, allowing an easier road safety evaluation. Besides, knowledge on the human factors that influence speed choice may be useful for road safety media campaigns and education programs designers, and also for the improvement of intelligent driver assistance systems.

Drivable Road Boundary Detection for Intelligent Vehicles based on Stereovision with Plane-induced Homography

Road detection is one of the key issues for intelligent vehicles and advanced driver assistance systems (ADAS). In this paper, we present a stereovision-based approach for estimating the boundary between the drivable road region and the non-road region. It is based on a formulation of stereo with the ground-plane-induced homography in a hidden Markov model (HMM). Under this formulation, we employ the Viterbi algorithm and propose a sophisticated measure of the probability of the state sequence to find the most likely road/non-road boundary. Experimental results on a wide variety of typical but challenging real road scenes have substantiated the effectiveness as well as robustness of the proposed approach.

Intelligent Tutoring and Training Tools for the Electric Power Sector Developed at IIE

The electric power industry requires qualied personnel to support an optimal and safe operation. Since its beginnings, the IIE has been developing dierent training technologies and systems for CFE, the main utility for generation, transmission and distribution of electric power in Mexico. Some of these endeavors provide tailored instruction considering trainees traits such as learning styles, aective states and current knowledge. Other developments are focused on enabling multi- functionality. The IIE has also developed intelligent assistant systems, virtual reality systems, and power plant simulators. Besides this, the IIE is interested in developing e-learning platforms to support CFE's per- sonnel training. This paper presents a summary of these developments.

Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey

In this paper, we provide a survey of the traffic sign detection literature, detailing detection systems for traffic sign recognition (TSR) for driver assistance. We separately describe the contributions of recent works to the various stages inherent in traffic sign detection: segmentation, feature extraction, and final sign detection. While TSR is a well-established research area, we highlight open research issues in the literature, including a dearth of use of publicly available image databases and the over-representation of European traffic signs. Furthermore, we discuss future directions of TSR research, including the integration of context and localization. We also introduce a new public database containing U.S. traffic signs.

Minimum discrimination information-based language model adaptation using tiny domain corpora for intelligent personal assistants

This paper proposes a novel Language Model (LM) adaptation method based on Minimum Discrimination Information (MDI). In the proposed method, a background LM is viewed as a discrete distribution and an adapted LM is built to be as close as possible to the background LM, while satisfying unigram constraint. This is due to the fact that there is a limited amount of domain corpus available for the adaptation of a natural language-based intelligent personal assistant system. Two unigram constraint estimation methods are proposed: one based on word frequency in the domain corpus, and one based on word similarity estimated from WordNet. In terms of the adapted LM's perplexity using word frequency in tiny domain corpora (ranging from 30~120 seconds in length) the relative performance improvements are measured at 13.9%~16.6%. Further relative performance improvements (1.5%~2.4%) are observed when WordNet is used to generate word similarities. These successes express an efficient ways for re-scaling and normalizing the conditional distribution, which uses an interpolation-based LM.

Human Pose Estimation and Activity Recognition From Multi-View Videos: Comparative Explorations of Recent Developments

This paper presents a review and comparative study of recent multi-view approaches for human 3D pose estimation and activity recognition. We discuss the application domain of human pose estimation and activity recognition and the associated requirements, covering: advanced human–computer interaction (HCI), assisted living, gesture-based interactive games, intelligent driver assistance systems, movies, 3D TV and animation, physical therapy, autonomous mental development, smart environments, sport motion analysis, video surveillance, and video annotation. Next, we review and categorize recent approaches which have been proposed to comply with these requirements. We report a comparison of the most promising methods for multi-view human action recognition using two publicly available datasets: the INRIA Xmas Motion Acquisition Sequences (IXMAS) Multi-View Human Action Dataset, and the i3DPost Multi-View Human Action and Interaction Dataset. To compare the proposed methods, we give a qualitative assessment of methods which cannot be compared quantitatively, and analyze some prominent 3D pose estimation techniques for application, where not only the performed action needs to be identified but a more detailed description of the body pose and joint configuration. Finally, we discuss some of the shortcomings of multi-view camera setups and outline our thoughts on future directions of 3D body pose estimation and human action recognition.

A Framework for Hierarchical Perception–Action Learning Utilizing Fuzzy Reasoning

Perception-action (P-A) learning is an approach to cognitive system building that seeks to reduce the complexity associated with conventional environment-representation/action-planning approaches. Instead, actions are directly mapped onto the perceptual transitions that they bring about, eliminating the need for intermediate representation and significantly reducing training requirements. We here set out a very general learning framework for cognitive systems in which online learning of the P-A mapping may be conducted within a symbolic processing context, so that complex contextual reasoning can influence the P-A mapping. In utilizing a variational calculus approach to define a suitable objective function, the P-A mapping can be treated as an online learning problem via gradient descent using partial derivatives. Our central theoretical result is to demonstrate top-down modulation of low-level perceptual confidences via the Jacobian of the higher levels of a subsumptive P-A hierarchy. Thus, the separation of the Jacobian as a multiplying factor between levels within the objective function naturally enables the integration of abstract symbolic manipulation in the form of fuzzy deductive logic into the P-A mapping learning. We experimentally demonstrate that the resulting framework achieves significantly better accuracy than using P-A learning without top-down modulation. We also demonstrate that it permits novel forms of context-dependent multilevel P-A mapping, applying the mechanism in the context of an intelligent driver assistance system.

Eye Gesture Analysis with Head Movement for Advanced Driver Assistance Systems

Road traffic accidents are a major cause of death worldwide. In an attempt to reduce accidents, some research efforts have focused on creating Advanced Driver Assistance Systems (ADAS) able to detect vehicle, driver and environmental conditions and to use this information to identify cues for potential accidents. This paper presents continued work on a novel Non-intrusive Intelligent Driver Assistance and Safety System (Ni-DASS) for assessing driver point of regard within vehicles. It uses an onboard CCD camera to observe the driver’s face. A template matching approach is used to compare the driver’s eye-gaze pattern with a set of eye-gesture templates of the driver looking at different focal points within the vehicle. The windscreen is divided into cells and comparison of the driver’s eye-gaze pattern with templates of a driver’s eyes looking at each cell is used to determine the driver’s point of regard on the windscreen. Results indicate that the proposed technique could be useful in situations where low resolution estimates of driver point of regard are adequate. For instance, To allow ADAS systems to alert the driver if he/she has positively failed to observe a hazard.

Intelligent Tracking Assistance System Based on GPS and GSM

In recent years, GPS (Global Positioning System) developed rapidly, and widely used in car navigation, logistics, monitoring and other fields. However, intelligent tracking assistance for the old is relatively backward in the application. With GSM (Global System for Mobile Communications) and the QT cross-platform development tool, we design an intelligent tracking and assistance system. The system consists of the user and the terminals. The user can track the old, while the terminal can send alarm messages to the user when the old press the alarm button. We design and implement the display, alarm, help and query functions. Also, the system has good portability and scalability.

Coordinating Cognitive Assistance With Cognitive Engagement Control Approaches in Human–Machine Collaboration

In human–machine collaboration, automated machines may assist operators in a variety of ways. However, chaotic assistance may lead to negative consequences, which makes the achievement of effective coordination of the different types of assistance all the more important. This paper discusses the classification of assistance on a cognitive basis and a method of coordinating assistance. Cognitive assistance is viewed as a 2-D problem, consisting of when to provide assistance (a control problem) and what assistance to provide (an interface problem). This paper further proposes dynamically controlling cognitive engagement levels to meet the demands of maintaining performance. Cognitive engagement control determines the appropriate moment to provide the proper level of cognitive assistance. To validate the above approach, a driving assistance experiment was conducted on a driving simulator. In the experiment, an intelligent assistance system monitored the real-time driving performance of human drivers, e.g., time headway and lateral deviation. Because of the importance of visual attention in driving performance, the system monitored the cognitive engagement status of drivers by measuring their eye movements with an eye tracker. Through five sessions of car-following driving tests, the coordinated cognitive assistance (named adaptive assistance) was compared with four other types of cognitive assistance: no aid, soft aid , soft intervention, and hard intervention. The experimental results confirmed that coordinated cognitive assistance is the most effective approach to provide assistance in both primary and secondary tasks. It also appears to be more enjoyable and less intrusive when compared with other individual types of cognitive assistance.

A Vision-Based Driver Nighttime Assistance and Surveillance System Based on Intelligent Image Sensing Techniques and a Heterogamous Dual-Core Embedded System Architecture

This study proposes a vision-based intelligent nighttime driver assistance and surveillance system (VIDASS system) implemented by a set of embedded software components and modules, and integrates these modules to accomplish a component-based system framework on an embedded heterogamous dual-core platform. Therefore, this study develops and implements computer vision and sensing techniques of nighttime vehicle detection, collision warning determination, and traffic event recording. The proposed system processes the road-scene frames in front of the host car captured from CCD sensors mounted on the host vehicle. These vision-based sensing and processing technologies are integrated and implemented on an ARM-DSP heterogamous dual-core embedded platform. Peripheral devices, including image grabbing devices, communication modules, and other in-vehicle control devices, are also integrated to form an in-vehicle-embedded vision-based nighttime driver assistance and surveillance system.

A new intelligent assistant system for HAZOP analysis of complex process plant

There are several problems that should be solved in the practical application of HAZOP analysis, such as the inheritance instrument of experience knowledge, the classification of accident reasons, the recurrence of analysis process and the verification of analysis results. This paper presents a new HAZOP assistant program named HELPHAZOP, which can be used to cope with these problems. With the aid of this program, a number of influence relationship models, which can be utilized to present the relationship structure of the whole system, can be established, and a variety of spread paths, which can be employed to describe the occurrence of the accidents, can be identified. These models and paths can help analyzers to understand the analysis process of different chemical processes and to verify the analysis results. Experience knowledge or information of process parameters stored in this program, including deviations, abnormal reasons, consequences and disaster-causing ways, can contribute to the analysis of some new plants. The program has been applied in the HAZOP analysis process of residuum hydrotreating process. The probability of the human errors which would be happened because of the misunderstanding or the deficiency of plant experience in the HAZOP analysis has been reduced and the accuracy, practicability and analysis efficiency of the accident forecast have been greatly improved.

Ontology Based Intelligent Assistance System to Support Manufacturing Activities in a Distributed Manufacturing Environment

The automotive industry is experiencing several limitations due to high mass customization on product side and the consequent evolution of newer ones as well as the expansion of existing production setups. There exist various complexities in planning and control of distributed production setups. This paper addresses issues related to planning using knowledge management approaches in the decentralized manufacturing domain with two distinct cases. The first case relates to the higher level planning methodology to automate experts’ knowledge in production ramp-up whereas the second case relates to optimization of production processes using knowledgebase as an intelligent assistance system for optimization.

Modeling and prediction of driver behavior by foot gesture analysis

Understanding driver behavior is an essential component in human-centric Intelligent Driver Assistance Systems. Specifically, driver foot behavior is an important factor in controlling the vehicle, though there have been very few research studies on analyzing foot behavior. While embedded pedal sensors may reveal some information about driver foot behavior, using vision-based foot behavior analysis has additional advantages. The foot movement before and after a pedal press can provide valuable information for better semantic understanding of driver behaviors, states, and styles. They can also be used to gain a time advantage in predicting a pedal press before it actually happens, which is very important for providing proper assistance to driver in time critical (e.g. safety related) situations. In this paper, we propose and develop a new vision based framework for driver foot behavior analysis using optical flow based foot tracking and a Hidden Markov Model (HMM) based technique to characterize the temporal foot behavior. In our experiment with a real-world driving testbed, we also use our trained HMM foot behavior model for prediction of brake and acceleration pedal presses. The experimental results over different subjects provided high accuracy (∼94% on average) for both foot behavior state inference and pedal press prediction. By 133 ms before the actual press, ∼74% of the pedal presses were predicted correctly. This shows the promise of applying this approach for real-world driver assistance systems.

Simultaneous analysis of driver behaviour and road condition for driver distraction detection

The design of intelligent driver assistance systems is of increasing importance for the vehicle-producing industry and road-safety solutions. This article starts with a review of road-situation monitoring and driver's behaviour analysis. This article also discusses lane tracking using vision (or other) sensors, and the strength or weakness of different methods of driver behaviour analysis (e.g. iris or pupil status monitoring, and EEG spectrum analysis). This article focuses then on image analysis techniques and develops a multi-faceted approach in order to analyse driver's face and eye status via implementing a real-time AdaBoost cascade classifier with Haar-like features. The proposed method is tested in a research vehicle for driver distraction detection using a binocular camera. The developed algorithm is robust in detecting different types of driver distraction such as drowsiness, fatigue, drunk driving or the performance of secondary tasks.

시각장애인을 위한 M2M 기반의 지능형 보행보조시스템

시각장애인들이 주로 사용하고 있는 백색 지팡이(White Stick)는 지상의 부유장애물에 대한 감지가 어려울 뿐만 아니라, 보행중인 시각장애인에게 현재 위치정보와 응급상황 발생시 효과적인 보호 조치를 제공하는데 문제점이 있다. 본 논문에서는 이와 같은 문제점을 해결하여 시각장애인들이 보다 안전하고 편리한 보행을 할수 있는 사물지능통신(M2M : Machine-to-Machine) 기반의 지능형 보행보조시스템을 제안한다. 제안하는 시스템은 시각장애인이 사용하는 보행보조지팡이와 이동통신망을 통하여 다수의 보행보조지팡이 사용자를 원격 지원하는 서버로 구성된다. 보행보조지팡이는 초음파센서와 문자-음성 변환기술 등을 이용한 장애물 감지 및 인지 가능뿐만 아니라, 서버에서 제공하는 지리정보와 GPS(Global Positioning System) 수신 장치 등을 이용한 현재위치 알림 및 확인, 위급상황 대처 기능 등을 제공한다. 서버는 지리정보 제공뿐만 아니라, 보호자나 구호기관에 위급상황 발생위치 알림, 보행 동안의 주요 정보(장소 시간 및 횟수 등)를 제공한다. 제안 시스템의 시제품을 통한 기능 및 성능 시험 결과 만족스런 결과를 얻을수 있었다. The white stick mainly used for visually impaired person has difficulty in providing location information and effective countermeasures for emergency situations encountered during walking as well as detecting floating obstacles on the ground. In this paper, we propose a machine-to-machine based intelligent walking assistance system for safe and convenient walking of the visually impaired. The proposed system consists of a walking assistance stick used by the visually impaired and a server supporting multiple stick users in remote places through mobile communication networks. The stick equipped with ultrasonic sensors, GPS(global positioning system) receiver and vibrator not only detects floating obstacles, but also offers stick users with present location identification utilizing a text-to-voice conversion technology. Besides providing geographic information, the server notifies the emergency locations of users to guardian and aid agency, and it provides log information during walking such as the place, time and the number of accidents. Test results with a developed prototype system have shown that the system properly performs the functions and satisfies overall system performance.

From expert systems to context-based intelligent assistant systems: a testimony

AbstractThis paper presents a personal interpretation of the evolution of artificial intelligence (AI) systems during these last 25 years. This evolution is presented along five generations of AI systems, namely expert systems, joint cognitive systems, intelligent systems, intelligent assistant systems, and the coming generation of context-based intelligent assistant systems. Our testimony relies on different real-world applications in different domains, especially for the French national power company, the subway companies in Paris and in Rio de Janeiro, in medicine, a platform for e-maintenance, road safety, and open sources. Our main claim is to underline that the next generation of AI systems (context-based intelligent assistant systems) requires a radically different consideration on context and its relations with the users, the task at hand, the situation, and the environment in which the task is accomplish by the user; the observation of users through their behaviors and not a profile library; a robust conceptual framework for modeling and managing context; and a computational tool for representing in a uniform way pieces of knowledge, of reasoning, and of contexts.