Assistive Mobile Robot Research Articles

Comprehensive Review of Security Problems in Mobile Robotic Assistant Systems: Issues, Solutions, and Challenges

Nowadays, robots in the modern world are playing an important and increasingly popular role. MRA (Mobile Robotic Assistant) is a type of mobile robot designed to support humans in many different fields, helping to improve efficiency and safety in daily activities, work, or medical treatment. The number of MRAs is increasing and diverse in function, in addition to the ability to collect and process data, MRAs also have the ability to physically interact with users. Therefore, security is one of the important issues to improve the safety and effective operation of MRA. In this paper, through a comprehensive literature review and detailed analysis of the prominent MRA security attacks in recent years (based on criteria such as: attack targets, technologies used, impact level, feasibility, and contribution to addressing overall MRA security issues), a systematic classification by MRA activity fields is conducted. Security attacks, threats, and vulnerabilities are examined from various perspectives, such as hardware attacks or network/system-level attacks, operating systems/application software. Additionally, corresponding security solutions are proposed, compared, and evaluated to enhance MRA security. The paper also addresses challenges and suggests open research directions for the future.

Virtual environment for assistant mobile robot

This paper shows the development of a virtual environment for a mobile robotic system with the ability to recognize basic voice commands, which are oriented to the recognition of a valid command of bring or take an object from a specific destination in residential spaces. The recognition of the voice command and the objects with which the robot will assist the user, is performed by a machine vision system based on the capture of the scene, where the robot is located. In relation to each captured image, a convolutional network based on regions is used with transfer learning, to identify the objects of interest. For human-robot interaction through voice, a convolutional neural network (CNN) of 6 convolution layers is used, oriented to recognize the commands to carry and bring specific objects inside the residential virtual environment. The use of convolutional networks allowed the adequate recognition of words and objects, which by means of the associated robot kinematics give rise to the execution of carry/bring commands, obtaining a navigation algorithm that operates successfully, where the manipulation of the objects exceeded 90%. Allowing the robot to move in the virtual environment even with the obstruction of objects in the navigation path.<

Remote Control Device to Drive the Arm Gestures of an Assistant Humanoid Robot

This work presents a remote control device designed to drive the arm gestures of an assistant humanoid mobile robot. The remote control is a master device with two passive arms configured to replicate the four degrees of freedom of each arm of the original assistant humanoid robot and send this information to the robot. This configuration allows the mobile robot to directly replicate the position of the arms on the remote controller. The objective of this proposal is to provide the robot with enhanced non-verbal and pointing communication capabilities during human interaction or assistance. The master device registers the angular position of each joint of its passive arms and transmits this information to the mobile robot, which replicates it. The experimental evaluation of the system has shown that the humanoid robot is able to successfully replicate any gesture on the remote controller. The positions of the arms have been sampled at a frame rate of 20 ms, and the average telecontrol delay obtained in the gesture experiments has been 549 ms, without appreciable jumps or irregularities in the gestures. The conclusion is that the direct manipulation of the passive arms of the remote control device provides the APR-02 humanoid robot with enhanced non-verbal and pointing communication capabilities during human interaction or assistance.

D.O.T. PAQUITOP, an Autonomous Mobile Manipulator for Hospital Assistance

The use of robotic technologies for caregiving and assistance has become a very interesting research topic in the field of robotics. Towards this goal, the researchers at Politecnico di Torino are developing robotic solutions for indoor assistance. This paper presents the D.O.T. PAQUITOP project, which aims at developing a mobile robotic assistant for the hospital environment. The mobile robot is composed of a custom omnidirectional platform, named PAQUITOP, a commercial 6 dof robotic arm, sensors for monitoring vital signs in patients, and a tablet to interact with the patient. To prove the effectiveness of this solution, preliminary tests were conducted with success in the laboratories of Politecnico di Torino and, thanks to the collaboration with the Onlus Fondazione D.O.T. and the medical staff of Molinette Hospital in Turin (Italy), at the hematology ward of Molinette Hospital.

Continuous shared control of a mobile robot with brain–computer interface and autonomous navigation for daily assistance

Although the electroencephalography (EEG) based brain–computer interface (BCI) has been successfully developed for rehabilitation and assistance, it is still challenging to achieve continuous control of a brain-actuated mobile robot system. In this study, we propose a continuous shared control strategy combining continuous BCI and autonomous navigation for a mobile robot system. The weight of shared control is designed to dynamically adjust the fusion of continuous BCI control and autonomous navigation. During this process, the system uses the visual-based simultaneous localization and mapping (SLAM) method to construct environmental maps. After obtaining the global optimal path, the system utilizes the brain-based shared control dynamic window approach (BSC-DWA) to evaluate safe and reachable trajectories while considering shared control velocity. Eight subjects participated in two-stage training, and six of these eight subjects participated in online shared control experiments. The training results demonstrated that naïve subjects could achieve continuous control performance with an average percent valid correct rate of approximately 97 % and an average total correct rate of over 80 %. The results of online shared control experiments showed that all of the subjects could complete navigation tasks in an unknown corridor with continuous shared control. Therefore, our experiments verified the feasibility and effectiveness of the proposed system combining continuous BCI, shared control, autonomous navigation, and visual SLAM. The proposed continuous shared control framework shows great promise in BCI-driven tasks, especially navigation tasks for brain-driven assistive mobile robots and wheelchairs in daily applications.

Seeking at-home long-term autonomy of assistive mobile robots through the integration with an IoT-based monitoring system

In this paper, we propose a system that stems from the integration of an autonomous mobile robot with an IoT-based monitoring system to provide monitoring, assistance, and stimulation to older adults living alone in their own houses. The creation of an Internet of Robotics Things (IoRT) based on the interplay between pervasive smart objects and autonomous robotic systems is claimed to enable the creation of innovative services conceived for assisting the final user, especially in elderly care. The synergy between IoT and a Socially Assistive Robot (SAR) was conceived to offer robustness, reconfiguration, heterogeneity, and scalability, by bringing a strong added value to both the current SAR and IoT technologies. First, we propose a method to achieve the synergy and integration between the IoT system and the robot; then, we show how our method increases the performance and effectiveness of both to provide long-term support to the older adults. To do so, we present a case-study, where we focus on the detection of signs of the frailty syndrome, a set of vulnerabilities typically conveyed by a cognitive and physical decline in older people that concur in amplifying the risks of major diseases hindering the capabilities of independent living. Experimental evaluation is performed in both controlled settings and in a long-term real-world pilot study with 9 older adults in their own apartments, where the system was deployed autonomously for, on average, 12 weeks.

Optimal assignment method for mobile assistive robots using matching theory

Optimal assignment method for mobile assistive robots using matching theory

Automated Bioprocess Feedback Operation in a High-Throughput Facility via the Integration of a Mobile Robotic Lab Assistant

The development of biotechnological processes is challenging due to the diversity of process parameters. For efficient upstream development, parallel cultivation systems have proven to reduce costs and associated timelines successfully while offering excellent process control. However, the degree of automation of such small-scale systems is comparatively low, and necessary sample analysis requires manual steps. Although the subsequent analysis can be performed in a high-throughput manner, the integration of analytical devices remains challenging, especially when cultivation and analysis laboratories are spatially separated. Mobile robots offer a potential solution, but their implementation in research laboratories is not widely adopted. Our approach demonstrates the integration of a small-scale cultivation system into a liquid handling station for an automated cultivation and sample procedure. The samples are transported via a mobile robotic lab assistant and subsequently analyzed by a high-throughput analyzer. The process data are stored in a centralized database. The mobile robotic workflow guarantees a flexible solution for device integration and facilitates automation. Restrictions regarding spatial separation of devices are circumvented, enabling a modular platform throughout different laboratories. The presented cultivation platform is evaluated on the basis of industrially relevant E. coli BW25113 high cell density fed-batch cultivation. The necessary magnesium addition for reaching high cell densities in mineral salt medium is automated via a feedback operation loop between the analysis station located in the adjacent room and the cultivation system. The modular design demonstrates new opportunities for advanced control options and the suitability of the platform for accelerating bioprocess development. This study lays the foundation for a fully integrated facility, where the physical connection of laboratory equipment is achieved through the successful use of a mobile robotic lab assistant, and different cultivation scales can be coupled through the common data infrastructure.

Sort and Deep-SORT Based Multi-Object Tracking for Mobile Robotics: Evaluation with New Data Association Metrics

Multi-Object Tracking (MOT) techniques have been under continuous research and increasingly applied in a diverse range of tasks. One area in particular concerns its application in navigation tasks of assistive mobile robots, with the aim to increase the mobility and autonomy of people suffering from mobility decay, or severe motor impairments, due to muscular, neurological, or osteoarticular decay. Therefore, in this work, having in view navigation tasks for assistive mobile robots, an evaluation study of two MOTs by detection algorithms, SORT and Deep-SORT, is presented. To improve the data association of both methods, which are solved as a linear assignment problem with a generated cost matrix, a set of new object tracking data association cost matrices based on intersection over union, Euclidean distances, and bounding box metrics is proposed. For the evaluation of the MOT by detection in a real-time pipeline, the YOLOv3 is used to detect and classify the objects available on images. In addition, to perform the proposed evaluation aiming at assistive platforms, the ISR Tracking dataset, which represents the object conditions under which real robotic platforms may navigate, is presented. Experimental evaluations were also carried out on the MOT17 dataset. Promising results were achieved by the proposed object tracking data association cost matrices, showing an improvement in the majority of the MOT evaluation metrics compared to the default data association cost matrix. In addition, promising frame rate values were attained by the pipeline composed of the detector and the tracking module.

An end-to-end CNN with attentional mechanism applied to raw EEG in a BCI classification task

Objective. Motor-imagery (MI) classification base on electroencephalography (EEG) has been long studied in neuroscience and more recently widely used in healthcare applications such as mobile assistive robots and neurorehabilitation. In particular, EEG-based MI classification methods that rely on convolutional neural networks (CNNs) have achieved relatively high classification accuracy. However, naively training CNNs to classify raw EEG data from all channels, especially for high-density EEG, is computationally demanding and requires huge training sets. It often also introduces many irrelevant input features, making it difficult for the CNN to extract the informative ones. This problem is compounded by a dearth of training data, which is particularly acute for MI tasks, because these are cognitively demanding and thus fatigue inducing. Approach. To address these issues, we proposed an end-to-end CNN-based neural network with attentional mechanism together with different data augmentation (DA) techniques. We tested it on two benchmark MI datasets, brain–computer interface (BCI) competition IV 2a and 2b. In addition, we collected a new dataset, recorded using high-density EEG, and containing both MI and motor execution (ME) tasks, which we share with the community. Main results. Our proposed neural-network architecture outperformed all state-of-the-art methods that we found in the literature, with and without DA, reaching an average classification accuracy of 93.6% and 87.83% on BCI 2a and 2b, respectively. We also directly compare decoding of MI and ME tasks. Focusing on MI classification, we find optimal channel configurations and the best DA techniques as well as investigate combining data across participants and the role of transfer learning. Significance. Our proposed approach improves the classification accuracy for MI in the benchmark datasets. In addition, collecting our own dataset enables us to compare MI and ME and investigate various aspects of EEG decoding critical for neuroscience and BCI.

Clothoidal Local Path Template for Intention Estimation by Assistive Mobile Robots

This work proposes and evaluates various improvements to a circular local path template (LPT) that we have designed in the past to estimate driver intents and to provide navigation assistance to wheelchair drivers. This LPT may also be useful in other mobile robotics applications such as for intuitive teleoperated control, for fast collision checking in path planning or for obstacle avoidance algorithms. The LPT consists of a large but fixed set of paths in the mobile robot’s local neighborhood. Based on an efficient look-up table, the LPT paths’ lengths are adjusted such that they are collisionfree. However, experiments have shown that in dense environments insufficient circular paths are found, which impedes correct intention estimation and thus navigation assistance. In this work, the use of clothoidal paths rather than circular paths is evaluated. This substantially improves the capability to find complex paths in dense areas. Furthermore, we adapt the LPT to deal with dynamic obstacles of random shape using motion prediction estimates of these objects.

Development of 3-degree-of-freedom HMI for omnidirectional wheelchair typed mobile assistive robot using simple force detection mechanism

Development of 3-degree-of-freedom HMI for omnidirectional wheelchair typed mobile assistive robot using simple force detection mechanism

A Multimodal User Interface for an Assistive Robotic Shopping Cart

This paper presents the research and development of the prototype of the assistive mobile information robot (AMIR). The main features of the presented prototype are voice and gesture-based interfaces with Russian speech and sign language recognition and synthesis techniques and a high degree of robot autonomy. AMIR prototype’s aim is to be used as a robotic cart for shopping in grocery stores and/or supermarkets. Among the main topics covered in this paper are the presentation of the interface (three modalities), the single-handed gesture recognition system (based on a collected database of Russian sign language elements), as well as the technical description of the robotic platform (architecture, navigation algorithm). The use of multimodal interfaces, namely the speech and gesture modalities, make human-robot interaction natural and intuitive, as well as sign language recognition allows hearing-impaired people to use this robotic cart. AMIR prototype has promising perspectives for real usage in supermarkets, both due to its assistive capabilities and its multimodal user interface.

HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification

Objective. Electroencephalography (EEG) motor imagery classification has been widely used in healthcare applications such as mobile assistive robots and post-stroke rehabilitation. Recently, EEG motor imagery classification methods based on convolutional neural networks (CNNs) have been proposed and have achieved relatively high classification accuracy. However, these methods use single convolution scale in the CNN, while the best convolution scale differs from subject to subject. This limits the classification accuracy. Another issue is that the classification accuracy degrades when training data is limited. Approach. To address these issues, we have proposed a hybrid-scale CNN architecture with a data augmentation method for EEG motor imagery classification. Main results. Compared with several state-of-the-art methods, the proposed method achieves an average classification accuracy of 91.57% and 87.6% on two commonly used datasets, which outperforms several state-of-the-art EEG motor imagery classification methods. Significance. The proposed method effectively addresses the issues of existing CNN-based EEG motor imagery classification methods and improves the classification accuracy.

A safety control method of assistive autonomous mobile robot using deep learning on nursing assistant system for the patients

A safety control method of assistive autonomous mobile robot using deep learning on nursing assistant system for the patients

Towards an All-Day Assignment of a Mobile Service Robot for Elderly Care Homes

Due to the demographic change in many industrial nations, the proportion of the older population is increasing. With this increase, the number of people who are dependent on outpatient or inpatient care is also rising across the board. Against this background, digital assistive systems could play an important role for improving the situation within those sectors. Therefore, the proposed novel approach describes a possible all-day use of a mobile assistive robot within an inpatient geriatric care facility, which should both relieve the staff and provide a therapeutic and entertaining contribution for the residents. The design of the components of the robot platform required for all-day use was carried out in an iterative development process. This process was started by convening a focus group, which first analyzed the requirements and then critically questioned the current status and actual benefits. Additionally, the accompanying occupational therapists and care assistants (N = 6) answered questionnaires after each of the 32 completed assignments, which were intended to draw attention to existing weaknesses and positive aspects. The main focus was to answer the question of how an assistive robot can be used meaningfully within an inpatient geriatric care facility with the means of the current state of science and whether this platform is perceived as support by the groups of people concerned. Due to the predominantly positive response to this question, the concept presented here for all-day use could be realized. Even if the response and operational capability were predominantly positive, there are still wishes from the staff and residents. These demands cannot yet be guaranteed with the current state of science to the required high degree of robustness under real world conditions. Consequently, the components identified as still in development or conceptually conceived require further research in the respective fields.

A Socially Aware SLAM Technique Augmented by Person Tracking Module

In recent years the development of Simultaneous Localization and Mapping (SLAM) techniques have enabled social robots to autonomously navigate in routine human workplaces. However, most common SLAM techniques are developed for mapping and localization in static worlds. In this paper, we have developed and analyzed a novel augmentation of the FastSLAM algorithm, including a person tracking module using 2D LiDAR sensor data. Utilizing this module, the SLAM algorithm is capable of filtering measurements coming from walking people who produce noises due to their intrinsic dynamic and unstableness. This augmentation was developed and then tested on our socially assistive mobile robot platform, Arash, while moving in populated environments utilizing Robotic Operating System (ROS) as a middleware. This new approach demonstrated a clearer representation of the mapped environment and therefore more accurate localization and navigation compared to its static SLAM predecessor.

A clinical pilot study on posture stabilization via light contact with cane-type companion robot

In this paper, toward robotic gait assistance, we investigate the feasibility of a cane-type assistive mobile robot accompanying the user autonomously through a clinical pilot experiment. As widely known, gait ability is important for all people to keep their quality of life. However, for people having weakened lower limbs such as elderly people, their postural sway during walking could be insuppressible and cause falling. To support the gait motion of elderly people, our group has been developing a series of hand-holding cane robots named Intelligent Cane. Such assistive robots are expected to remove barriers to the independent lives of elderly people. Recently, we have focused on the potential of a companion robot that follows the user ahead and can be touched or grasped for bracing whenever the user needs it. In order to demonstrate proof of our concept through an experiment with a motion capture system, we propose a user companion strategy that enables our cane robot to keep a constant relative distance between the robot and the user walking on a treadmill. We evaluate the accuracy of the user companion in an experiment where a user walks on a treadmill. Then, we conduct a clinical experiment with three healthy subjects walking on with the treadmill with our cane robot as a pilot study. Through the clinical experiment, we evaluate a postural stabilization effect of physical interaction with the robot and discuss the feasibility of our robotic gait assistance methodology.

Admittance control based robotic clinical gait training with physiological cost evaluation

In this paper, we investigate an effective control strategy for a cane-type assistive mobile robot toward clinical gait training. Assistive robots are expected to provide aid in order to reduce the burden of caregivers and physical therapists, e.g., in gait rehabilitation of elderly people. Our group has been developing a series of cane-type walking assistive robots named Intelligent Cane as a mobile hand-holding device based on admittance control to provide safe and efficient gait training. This paper explores a systematic design methodology of the admittance control model in order to provide suitable walking load during gait training. We first conduct a pilot experiment to investigate the relationship between the physiological cost of user’s walking and the coefficients in the admittance control model of the cane robot. Then, we present a clinical gait training study conducted in a hospital to evaluate the feasibility in practical use of the proposed control strategy of our cane robot in gait rehabilitation. These experimental results suggest the effectiveness of the proposed gait rehabilitation strategy with our robot.

Methodology for Autonomous Crossing Narrow Passages Applied on Assistive Mobile Robots

The assistive robotics field works with technologies aimed at assisting people with physical impairments. Simple tasks like passing through doorways can be a significant challenge for motored wheelchairs users. Besides, depending on the disability degree, the task may be impossible to complete without help. Therefore, mobile robotics technologies have been implemented in wheelchairs to give them intelligence, and a supervisory interface inserts the user in the control loop to complete the objectives. Thus, this research proposes an innovative method to assist the user to cross narrow passages automatically. The methodology works in known and unknown environments. Besides, it does not require mapping and global localization algorithms. In this technique, a depth camera recognizes the edges between a narrow path. A Kalman filter method is used to improve the precision of these points even when the passage leaves the camera field of view. Finally, a nonlinear controller performs the narrow passage crossing. This methodology was analyzed using a MATLAB model, Gazebo simulation platform, and a real mobile robot Pioneer-P3DX. The results demonstrated promising applicability.