Assistive Robotic Arm Research Articles

Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments.

This paper presents a new architecture that integrates Internet of Things (IoT) devices, service robots, and users in a smart assistive environment. A new intuitive and multimodal interaction system supporting people with disabilities and bedbound patients is presented. This interaction system allows the user to control service robots and devices inside the room in five different ways: touch control, eye control, gesture control, voice control, and augmented reality control. The interaction system is comprised of an assistive robotic arm holding a tablet PC. The robotic arm can place the tablet PC in front of the user. A demonstration of the developed technology, a prototype of a smart room equipped with home automation devices, and the robotic assistive arm are presented. The results obtained from the use of the various interfaces and technologies are presented in the article. The results include user preference with regard to eye-base control (performing clicks, and using winks or gaze) and the use of mobile phones over augmented reality glasses, among others.

Self‐feeding assistive 7‐DOF robotic arm simulator using solving method of inverse kinematics

AbstractIn order to help people with disabilities of the upper extremities drink and eat, a simulator for self‐feeding assistive robotic arm using solving method of inverse kinematics has been developed in this paper. Using inverse kinematics equations of 7 degrees of freedom (DOF) with an arm angle parameter, the robotic arm model was implemented in the simulator and its controller with mouse device was also designed. The arm angle parameter defined the angle between the arm plane and a reference plane was used to resolve the robotic arm redundancy. Three able‐bodied persons participated in simulation experiments. They manipulated the robotic arm model in a virtual space using mouse device. It was found from the simulation results they successfully performed the drinking and eating tasks in our simulator.

Decoding Hand Movement Types and Kinematic Information From Electroencephalogram.

Brain-computer interfaces (BCIs) have achieved successful control of assistive devices, e.g. neuroprosthesis or robotic arm. Previous research based on hand movements Electroencephalogram (EEG) has shown limited success in precise and natural control. In this study, we explored the possibilities of decoding movement types and kinematic information for three reach-and-execute actions using movement-related cortical potentials (MRCPs). EEG signals were acquired from 12 healthy subjects during the execution of pinch, palmar and precision disk rotation actions that involved two levels of speeds and forces. In the case of discrimination between hand movement types under each of four different kinematics conditions, we obtained the average peak accuracies of 83.44% and 73.83% for the binary and 3-class classification, respectively. In the case of discrimination between different movement kinematics for each of three actions, the average peak accuracies of 82.9% and 58.2% could be achieved for the two and 4-class scenario. In both cases, peak decoding performance was significantly higher than the subject-specific chance level. We found that hand movement types all could be classified when these actions were executed at four different kinematic parameters. Meanwhile, for each of three hand movements, we decoded movement parameters successfully. Furthermore, the feasibility of decoding hand movements during hand retraction process was also demonstrated. These findings are of great importance for controlling neuroprosthesis or other rehabilitation devices in a fine and natural way, which would drastically increase the acceptance of motor impaired users.

Aprendizaje profundo con imágenes RGB-D: clasificación de objetos y estimación de la pose

En el marco de la tesis de doctorado, se plantea como objetivo el desarrollo de una Interfaz Hombre-Máquina para comandar un brazo robótico asistencial de más de 6 grados de libertad. Se presenta el uso de técnicas de aprendizaje profundo para el reconocimiento de objetos y estimación de la pose a fin de poder interactuar con los mismos. Se implementaron 3 modelos de redes neuronales convolucionales multimodales para imágenes RGB-D de la base de datos BigBIRD, con tres salidas de clasificación: 22 Objetos - 5 Cámaras - 8 etiquetas de Rotación. Para el mejor de los modelos se alcanzaron valores de precisión de 96% para objetos, 98% para cámara y 56% para la rotación.

Development of assistive robotic arm for power line maintenance

In Japan, deterioration of electric power supply systems has been accompanied by a steady increase in the quantity of maintenance work. Since such work on the bucket of an elevated work vehicle is extremely hard for the workers, innovations are required to save labor. Some robotic systems attached to the bucket have been researched, however, they are remotely operated and their operability is not so good. Therefore, we propose and develop a new assistive robotic arm that can be operated directly and easily by only one worker on the bucket. It is practical, easy to install, and relatively light weight. This system consists of three arms and each arm has 4 DOF and one semi-fixed joint. The arm end can be extended from the level of the operator's hands to a height of about 1 m above his head for safety work then retracted to replace one specialized tool with another. An operation grip is located near the arm base and the arm moves in direction the grip is pushed. This grip also equips smart mechanism for permitting the operator to sense the reaction force on the end tools. A prototype robotic arm was made and some operating tests were carried out to ensure the effectiveness of the proposed human-robot cooperation system.

Self-feeding Assistive 7-DOF Robotic Arm Simulator Using Solving Method of Inverse Kinematics

Self-feeding Assistive 7-DOF Robotic Arm Simulator Using Solving Method of Inverse Kinematics

AMiCUS 2.0-System Presentation and Demonstration of Adaptability to Personal Needs by the Example of an Individual with Progressed Multiple Sclerosis.

AMiCUS is a human–robot interface that enables tetraplegics to control an assistive robotic arm in real-time using only head motion, allowing them to perform simple manipulation tasks independently. The interface may be used as a standalone system or to provide direct control as part of a semi-autonomous system. Within this work, we present our new gesture-free prototype AMiCUS 2.0, which has been designed with special attention to accessibility and ergonomics. As such, AMiCUS 2.0 addresses the needs of tetraplegics with additional impairments that may come along with multiple sclerosis. In an experimental setup, both AMiCUS 1.0 and 2.0 are compared with each other, showing higher accessibility and usability for AMiCUS 2.0. Moreover, in an activity of daily living, a proof-of-concept is provided that an individual with progressed multiple sclerosis is able to operate the robotic arm in a temporal and functional scope, as would be necessary to perform direct control tasks for use in a commercial semi-autonomous system. The results indicate that AMiCUS 2.0 makes an important step towards closing the gaps of assistive technology, being accessible to those who rely on such technology the most.

Semi-Autonomous Robotic Arm Reaching With Hybrid Gaze-Brain Machine Interface.

Recent developments in the non-muscular human–robot interface (HRI) and shared control strategies have shown potential for controlling the assistive robotic arm by people with no residual movement or muscular activity in upper limbs. However, most non-muscular HRIs only produce discrete-valued commands, resulting in non-intuitive and less effective control of the dexterous assistive robotic arm. Furthermore, the user commands and the robot autonomy commands usually switch in the shared control strategies of such applications. This characteristic has been found to yield a reduced sense of agency as well as frustration for the user according to previous user studies. In this study, we firstly propose an intuitive and easy-to-learn-and-use hybrid HRI by combing the Brain–machine interface (BMI) and the gaze-tracking interface. For the proposed hybrid gaze-BMI, the continuous modulation of the movement speed via the motor intention occurs seamlessly and simultaneously to the unconstrained movement direction control with the gaze signals. We then propose a shared control paradigm that always combines user input and the autonomy with the dynamic combination regulation. The proposed hybrid gaze-BMI and shared control paradigm were validated for a robotic arm reaching task performed with healthy subjects. All the users were able to employ the hybrid gaze-BMI for moving the end-effector sequentially to reach the target across the horizontal plane while also avoiding collisions with obstacles. The shared control paradigm maintained as much volitional control as possible, while providing the assistance for the most difficult parts of the task. The presented semi-autonomous robotic system yielded continuous, smooth, and collision-free motion trajectories for the end effector approaching the target. Compared to a system without assistances from robot autonomy, it significantly reduces the rate of failure as well as the time and effort spent by the user to complete the tasks.

Proof of Concept of an Assistive Robotic Arm Control Using Artificial Stereovision and Eye-Tracking.

Assistive robotic arms have become popular to help users with upper limb disabilities achieve autonomy in their daily tasks, such as drinking and grasping objects in general. Usually, these robotic arms are controlled with an adapted joystick. Joysticks are user-friendly when it comes to a general approach to an object. However, they are not as intuitive when having to accurately approach an object, especially when obstacles are present. Alternatively, the combined use of artificial stereovision and eye-tracking seems to be a promising solution, as the user's vision is usually dissociated from their upper limb disability. Hence, the objective of this study was to develop a proof of concept for the control of an assistive robotic arm using a low-cost combination of stereovision and eye-tracking. Using the developed control system, a typically developed person was able to control the robotic arm successfully reaching and grasping an object for 92% of the trials without obstacles with an average time of 13.8 seconds. Then, another set of trials with one obstacle had a success rate of 91% with an average time of 17.3 seconds. Finally, the last set of trials with two obstacles had a success rate of 98% with an average time of 18.4 seconds. Furthermore, the cost of an eye-tracker and stereovision remains below 400$.

Integration of Forearm sEMG Signals with IMU Sensors for Trajectory Planning and Control of Assistive Robotic Arm.

Patients with issues such as cerebral palsy, spinal cord injury, and multiple sclerosis have difficulties with activities of daily living (ADL). Their abilities to perform tasks can be improved through vigorous physical therapy. When that therapy is either not effective or lacking in its progression an assistive robotic device can be used to improve patients' quality of life and help them in accomplishing ADL's. This study presents implementation of an EMG controlled assistive robotic arm to aid patients with upper limb mobility limitations. Using the MYO armband, EMG signals were obtained from three healthy human test subjects and were analyzed in MATLAB® Simulink®. Post signal acquisition, signals were classified to be used as inputs for a Kinova MICO 6 DOF manipulator. Trajectories are planned based on the user EMG signals and robot position data obtained from the Polhemus 6D motion tracker, an IMU-type sensor, which automatically provide position and orientation data. An inverse dynamics controller is developed to drive the robot joints accordingly. Results have shown that the classification accuracy of the EMG signals to control commands for the robot was greater than 90%. The classification accuracy was achieved through the use of a pattern recognition neural network. This preliminary investigation demonstrates the possible future implementation of the system for its intended application.

A high-resolution tongue-based joystick to enable robot control for individuals with severe disabilities.

Assistive robotic arms have shown the potential to improve the quality of life of people with severe disabilities. However, a high performance and intuitive control interface for robots with 6-7 DOFs is still missing for these individuals. An inductive tongue computer interface (ITCI) was recently tested for control of robots and the study illustrated potential in this field. The paper describes the investigation of the possibility of developing a high performance tongue-based joystick-like controller for robots through two studies. The first compared different methods for mapping the 18 sensor signals to a 2D coordinate, as a touchpad. The second evaluated the performance of a novel approach for emulating an analog joystick by the ITCI based on the ISO9241-411 standard. Two subjects performed a multi-directional tapping test using a standard analog joystick, the ITCI system held in hand and operated by the other hand, and finally by tongue when mounted inside the mouth. Throughput was measured as the evaluation parameter. The results show that the contact on the touchpads can be localized by almost 1 mm accuracy. The effective throughput of ITCI system for the multi-directional tapping test was 2.03 bps while keeping it in the hand and 1.31 bps when using it inside the mouth.

Semi-Autonomous Tongue Control of an Assistive Robotic Arm for Individuals with Quadriplegia.

Individuals suffering from quadriplegia can achieve increased independence by using an assistive robotic manipulator (ARM). However, due to their disability, the interfaces that can be used to operate such devices become limited. A versatile intraoral tongue control interface (ITCI) has previously been develop for this user group, as the tongue is usually spared from disability. A previous study has shown that the ITCI can provide direct and continuous control of 6-7 degrees of freedom (DoF) of an ARM, due to a high number of provided inputs (18). In the present pilot study we investigated whether semi-automation might further improve the efficiency of the ITCI, when controlling an ARM. This was achieved by adding a camera to the end effector of the ARM and using computer vision algorithms to guide the ARM to grasp a target object. Three ITCI and one joystick control scheme were tested and compared: 1) manual Cartesian control with a base frame reference point, 2) manual Cartesian control with an end effector reference point 3) manual Cartesian control with an end effector reference point and an autonomous grasp function 4) regular JACO2 joystick control. The results indicated that end effector control was superior to the base frame control in total task time, number of commands issued and path efficiency. The addition of the automatic grasp function did not improve the performance, but resulted in fewer collisions/displacements of the target object when grasping.

Voice Control Interface Prototype for Assistive Robots for People Living with Upper Limb Disabilities.

This paper presents a voice control interface prototype for assistive robots aiming to help people living with upper limb disabilities to perform daily activities autonomously. Assistive robotic devices can be used to help people with upper-body disabilities gain more autonomy in their daily life. However, it is very difficult or even impossible for certain users to control the robot with conventional control systems (e.g. joystick, sip-and-puff). This paper presents the design and preliminary evaluation of a voice command system prototype for the control of assistive robotic arms' movements. This work aims at making the control of assistive robots more intuitive and fluid, and to perform various tasks in less time and with a lesser effort. The prototype of the voice command interface developed is first presented, followed by two experiments with five able-bodied subjects in order to assess the system's performance and guide future development.

Pattern recognition based on HD-sEMG spatial features extraction for an efficient proportional control of a robotic arm

To enable an efficient alternative control of an assistive robotic arm using electromyographic (EMG) signals, the control method must simultaneously provide both the direction and the velocity. However, the contraction variations of the forearm muscles, used to proportionally control the device’s velocity using a regression method, can disturb the accuracy of the classification used to estimate its direction at the same time. In this paper, the original set of spatial features takes advantage of the 2D structure of an 8 × 8 high-density surface EMG (HD-sEMG) sensor to perform a high accuracy classification while improving the robustness to the contraction variations. Based on the HD-sEMG sensor, different muscular activity images are extracted by applying different spatial filters. In order to characterize their distribution specific to each movement, instead of the EMG signals’ amplitudes, these muscular images are divided in sub-images upon which the proposed spatial features, such as the centers of the gravity coordinates and the percentages of influence, are computed. These features permits to achieve average accuracies of 97% and 96.7% to detect respectively 16 forearm movements performed by a healthy subject with prior experience with the control approach and 10 movements by ten inexperienced healthy subjects. Compared with the time-domain features, the proposed method exhibits significant higher accuracies in presence of muscular contraction variations, requires less training data and is more robust against the time of use. Furthermore, two fine real-time tasks illustrate the potential of the proposed approach to efficiently control a robotic arm.

Assistive robotic arm: Evaluation of the performance of intelligent algorithms

ABSTRACT People with upper body disabilities may be limited in their activities of daily living. Robotic arms, such as JACO, are assistive devices that could improve their abilities, independent living, and social participation. However, performing complex tasks with JACO can be time-consuming or tedious. Therefore, some advanced functionalities have been developed to enhance the performance of users. The main objective of this study is to evaluate the performance, in terms of ease of use, task completion time, and participants’ perception of usability, of three new algorithms applied to the JACO robotic arm: (1) predefined position, (2) fluidity filter, and (3) drinking mode. The secondary objective is to evaluate differences in performance variables between proportional and non-proportional control modes. Fourteen participants with upper body disabilities completed various tasks with and without these functionalities. Using JACO with the algorithms led to a significant decrease of up to 72% in task completion time and improvements of 2.3 and 2.9 on a 7-point Likert scale for perceived ease of use and usability, respectively. There was no significant difference between control modes. Our results demonstrate that algorithms could produce significant improvements in performing daily living activities.

A Pilot Study of Endoscopic Submucosal Dissection Using an Endoscopic Assistive Robot in a Porcine Stomach Model.

Background/AimsEndoscopic assistive devices have been developed to reduce the complexity and improve the safety of surgeries involving the use of endoscopes. We developed an assistive robotic arm for endoscopic submucosal dissection (ESD) and evaluated its efficiency and safety in this in vitro pilot study.MethodsESD was performed using an auxiliary transluminal endoscopic robot. An in vitro test bed replicating the intra-abdominal environment and pig stomachs were used for the experiment. Participants were divided into skilled operators and unskilled operators. Each group performed ESD 10 times by using both conventional and robot-assisted methods. The perforation incidence, operation time, and resected mucous membrane size were measured.ResultsFor the conventional method, significant differences were noted between skilled and unskilled operators regarding operation time (11.3 minutes vs 26.7 minutes) and perforation incidence (0/10 vs 6/10). Unskilled operators showed a large decrease in the perforation incidence with the robot-assisted method (conventional method vs robot-assisted method, 6/10 vs 1/10). However, the operation time did not differ between the conventional and robot-assisted methods. On the other hand, skilled operators did not show differences in the operation time and perforation incidence between the conventional and robot-assisted methods. Among both skilled and unskilled operators, the operation time decreased with the robot-assisted method as the experiment proceeded.ConclusionsThe surgical safety of unskilled operators greatly improved with robotic assistance. Thus, our assistive robotic arm was beneficial for ESD. Our findings suggest that endoscopic assistive robots have positive effects on surgical safety.

Analysis on Load Torque Effect for Assistive Robotic Arms

Analysis on Load Torque Effect for Assistive Robotic Arms

An Assistive Robotic Arm For People With Physical Disabilities Of The Extremities: HOG Based Food Detection.

An assistive robotic arm for helping feed people with physical disabilities of the extremities is presented in this paper. In order to detect foods on a plate, HOG based food detectors were created and implemented in our robotic arm system. The system consists of a 6-axis robotic arm, Web camera, laptop computer, and microcontroller. Using the robotic arm system, we performed two experiments: (1) eating chicken nuggets task and (2) eating shao mais task. It was found from the experimental results that the proposed assistive robotic arm system could stably transfer chicken nuggets and shao mais to the user's mouth.

A Multimodal Adaptive Wireless Control Interface for People With Upper-Body Disabilities.

This paper describes a multimodal body-machine interface (BoMI) to help individuals with upper-limb disabilities using advanced assistive technologies, such as robotic arms. The proposed system uses a wearable and wireless body sensor network (WBSN) supporting up to six sensor nodes to measure the natural upper-body gesture of the users and translate it into control commands. Natural gesture of the head and upper-body parts, as well as muscular activity, are measured using inertial measurement units (IMUs) and surface electromyography (sEMG) using custom-designed multimodal wireless sensor nodes. An IMU sensing node is attached to a headset worn by the user. It has a size of 2.9 cm 2.9 cm, a maximum power consumption of 31 mW, and provides angular precision of 1. Multimodal patch sensor nodes, including both IMU and sEMG sensing modalities are placed over the user able-body parts to measure the motion and muscular activity. These nodes have a size of 2.5 cm 4.0 cm and a maximum power consumption of 11 mW. The proposed BoMI runs on a Raspberry Pi. It can adapt to several types of users through different control scenarios using the head and shoulder motion, as well as muscular activity, and provides a power autonomy of up to 24h. JACO, a 6-DoF assistive robotic arm, is used as a testbed to evaluate the performance of the proposed BoMI. Ten able-bodied subjects performed ADLs while operating the AT device, using the Test d'Évaluation des Membres Supérieurs de Personnes Âgées to evaluate and compare the proposed BoMI with the conventional joystick controller. It is shown that the users can perform all tasks with the proposed BoMI, almost as fast as with the joystick controller, with only 30% time overhead on average, while being potentially more accessible to the upper-body disabled who cannot use the conventional joystick controller. Tests show that control performance with the proposed BoMI improved by up to 17% on average, after three trials.

Wireless intraoral tongue control of an assistive robotic arm for individuals with tetraplegia

BackgroundFor an individual with tetraplegia assistive robotic arms provide a potentially invaluable opportunity for rehabilitation. However, there is a lack of available control methods to allow these individuals to fully control the assistive arms.MethodsHere we show that it is possible for an individual with tetraplegia to use the tongue to fully control all 14 movements of an assistive robotic arm in a three dimensional space using a wireless intraoral control system, thus allowing for numerous activities of daily living. We developed a tongue-based robotic control method incorporating a multi-sensor inductive tongue interface. One abled-bodied individual and one individual with tetraplegia performed a proof of concept study by controlling the robot with their tongue using direct actuator control and endpoint control, respectively.ResultsAfter 30 min of training, the able-bodied experimental participant tongue controlled the assistive robot to pick up a roll of tape in 80% of the attempts. Further, the individual with tetraplegia succeeded in fully tongue controlling the assistive robot to reach for and touch a roll of tape in 100% of the attempts and to pick up the roll in 50% of the attempts. Furthermore, she controlled the robot to grasp a bottle of water and pour its contents into a cup; her first functional action in 19 years.ConclusionTo our knowledge, this is the first time that an individual with tetraplegia has been able to fully control an assistive robotic arm using a wireless intraoral tongue interface. The tongue interface used to control the robot is currently available for control of computers and of powered wheelchairs, and the robot employed in this study is also commercially available. Therefore, the presented results may translate into available solutions within reasonable time.