Wheelchair-mounted Robotic Arm Research Articles

Design of a Wheelchair-Mounted Robotic Arm for Feeding Assistance of Upper-Limb Impaired Patients

This paper delineates the design and realization of a Wheelchair-Mounted Robotic Arm (WMRA), envisioned as an autonomous assistance apparatus for individuals encountering motor difficulties and/or upper limb paralysis. The proposed design solution is based on employing a 3D printing process coupled with optimization design techniques to achieve a cost-oriented and user-friendly solution. The proposed design is based on utilizing commercial Arduino control hardware. The proposed device has been named Pick&Eat. The proposed device embodies reliability, functionality, and cost-effectiveness, and features a modular structure housing a 4-degrees-of-freedom robotic arm with a fixing frame that can be attached to commercial wheelchairs. The arm is integrated with an interchangeable end-effector facilitating the use of various tools such as spoons or forks tailored to different food types. Electrical and sensor components were meticulously designed, incorporating sensors to ensure user safety throughout operations. Smooth and secure operations are achieved through a sequential procedure that is depicted in a specific flowchart. Experimental tests have been carried out to demonstrate the engineering feasibility and effectiveness of the proposed design solution as an innovative assistive solution for individuals grappling with upper limb impairment. Its capacity to aid patients during the eating process holds promise for enhancing their quality of life, particularly among the elderly and those with disabilities.

Optimization of Wheelchair-Mounted Robotic Arms’ Base Placement by Fusing Occupied Grid Map and Inverse Reachability Map

In a household setting, a wheelchair-mounted robotic arm (WMRA) can be useful for assisting elderly and disabled individuals. However, the current WMRA can only perform movement and grasping tasks through joystick remote control. This method results in low efficiency due to poor coordination between the mobile platform and the robotic arm as well as the numerous operational steps required. To improve the efficiency and success rate of the robot in task execution, this paper proposes a parking location optimization method that combines the occupied grid map (OGM) and the inverse reachability map (IRM). Firstly, the SLAM algorithm is used to collect environment information, which is then stored in the form of an occupied grid map. The robotic arm workspace is then gridded, and the inverse reachability map is calculated based on the grasping pose of the target object. Finally, the optimal position of the mobile platform is obtained by comparing the optimal location point in the inverse reachability map and the obstacle information in the occupied grid map. This process achieves base placement optimization based on the grasping pose. The experimental results demonstrate that this method reduces the user operation time by 97.31% and overall task completion time by 40.57% when executing household environment tasks compared with the joystick control, increasing the range of executable tasks compared with the algorithm of the EL-E robot and reducing task completion time by 23.48% for the same task. This paper presents a parking location optimization method that can improve the grasping efficiency of the robotic arm and achieve parking location position selection for the WMRA in a household environment.

Object Affordance-Based Implicit Interaction for Wheelchair-Mounted Robotic Arm Using a Laser Pointer.

With the growth of the world's population, limited healthcare resources cannot provide adequate nursing services for all people in need. The wheelchair-mounted robotic arm (WMRA) with interactive technology could help to improve users' self-care ability and relieve nursing stress. However, the users struggle to control the WMRA due to complex operations. To use the WMRA with less burden, this paper proposes an object affordance-based implicit interaction technology using a laser pointer. Firstly, a laser semantic identification algorithm combined with the YOLOv4 and the support vector machine (SVM) is designed to identify laser semantics. Then, an implicit action intention reasoning algorithm, based on the concept of object affordance, is explored to infer users' intentions and learn their preferences. For the purpose of performing the actions about task intention in the scene, the dynamic movement primitives (DMP) and the finite state mechanism (FSM) are respectively used to generalize the trajectories of actions and reorder the sequence of actions in the template library. In the end, we verified the feasibility of the proposed technology on a WMRA platform. Compared with the previous method, the proposed technology can output the desired intention faster and significantly reduce the user's limb involvement time (about 85%) in operating the WMRA under the same task.

Adaptive control of a wheelchair mounted robotic arm with neuromorphically integrated velocity readings and online-learning.

Wheelchair-mounted robotic arms support people with upper extremity disabilities with various activities of daily living (ADL). However, the associated cost and the power consumption of responsive and adaptive assistive robotic arms contribute to the fact that such systems are in limited use. Neuromorphic spiking neural networks can be used for a real-time machine learning-driven control of robots, providing an energy efficient framework for adaptive control. In this work, we demonstrate a neuromorphic adaptive control of a wheelchair-mounted robotic arm deployed on Intel’s Loihi chip. Our algorithm design uses neuromorphically represented and integrated velocity readings to derive the arm’s current state. The proposed controller provides the robotic arm with adaptive signals, guiding its motion while accounting for kinematic changes in real-time. We pilot-tested the device with an able-bodied participant to evaluate its accuracy while performing ADL-related trajectories. We further demonstrated the capacity of the controller to compensate for unexpected inertia-generating payloads using online learning. Videotaped recordings of ADL tasks performed by the robot were viewed by caregivers; data summarizing their feedback on the user experience and the potential benefit of the system is reported.

Control of a Wheelchair-Mounted 6DOF Assistive Robot With Chin and Finger Joysticks.

Throughout the last decade, many assistive robots for people with disabilities have been developed; however, researchers have not fully utilized these robotic technologies to entirely create independent living conditions for people with disabilities, particularly in relation to activities of daily living (ADLs). An assistive system can help satisfy the demands of regular ADLs for people with disabilities. With an increasing shortage of caregivers and a growing number of individuals with impairments and the elderly, assistive robots can help meet future healthcare demands. One of the critical aspects of designing these assistive devices is to improve functional independence while providing an excellent human–machine interface. People with limited upper limb function due to stroke, spinal cord injury, cerebral palsy, amyotrophic lateral sclerosis, and other conditions find the controls of assistive devices such as power wheelchairs difficult to use. Thus, the objective of this research was to design a multimodal control method for robotic self-assistance that could assist individuals with disabilities in performing self-care tasks on a daily basis. In this research, a control framework for two interchangeable operating modes with a finger joystick and a chin joystick is developed where joysticks seamlessly control a wheelchair and a wheelchair-mounted robotic arm. Custom circuitry was developed to complete the control architecture. A user study was conducted to test the robotic system. Ten healthy individuals agreed to perform three tasks using both (chin and finger) joysticks for a total of six tasks with 10 repetitions each. The control method has been tested rigorously, maneuvering the robot at different velocities and under varying payload (1–3.5 lb) conditions. The absolute position accuracy was experimentally found to be approximately 5 mm. The round-trip delay we observed between the commands while controlling the xArm was 4 ms. Tests performed showed that the proposed control system allowed individuals to perform some ADLs such as picking up and placing items with a completion time of less than 1 min for each task and 100% success.

A Shared Control Strategy for Reach and Grasp of Multiple Objects Using Robot Vision and Noninvasive Brain–Computer Interface

It is ambitious to develop a brain-controlled robotic arm for some patients with motor impairments to perform activities of daily living using brain&#x2013;computer interfaces (BCIs). Despite much progress achieved, this mission is still very challenging mainly due to the poor decoding performance of BCIs. The problem is even exacerbated in the case of noninvasive BCIs. A shared control strategy is developed in this work to realize flexible robotic arm control for reach and grasp of multiple objects. With the intelligent assistance provided by robot vision, the subject was only required to finish gross reaching movement and target selection using a simple motor imagery-based BCI with binary output. Along with the user control, the robotic arm, which identified and localized potential targets within the workspace in the background, was capable of providing both trajectory correction in the reaching phase to reduce trajectory redundancy and autonomous grasping assistance in the phase of grasp. Ten subjects participated in the experiments containing one session of two-block grasping tasks with fixed locations and another one of randomly placed three-block grasping tasks. The results of the experiments demonstrated substantial improvement with the shared control system. Compared with the single BCI control, the success rate of shared control was significantly higher (<inline-formula> <tex-math notation="LaTeX">$p &lt; 0.001$ </tex-math></inline-formula> for group performance), and moreover, the task completion time and perceived difficulty were significantly lower (<inline-formula> <tex-math notation="LaTeX">$p &lt; 0.001$ </tex-math></inline-formula> for group performance both), indicating the potential of our proposed shared control system in real applications. <i>Note to Practitioners</i>&#x2014;This article is motivated by the problem of dexterous robotic arm control based on a brain&#x2013;computer interface (BCI). For people suffering from severe neuromuscular disorders or accident injuries, a brain-controlled robotic arm is expected to provide assistance in their daily lives. A primary bottleneck to achieve the objective is that the information transfer rate of current BCIs is not high enough to produce multiple and reliable commands during the online robotic control. In this work, machine autonomy is incorporated in a BCI-controlled robotic arm system, where the user and machine can work together to reach and grasp multiple objects in a given task. The intelligent robot system autonomously localized the potential targets and provided trajectory correction and grasping assistance accordingly. Meanwhile, the user only needed to complete gross reaching movement and target selection with a basic binary motor imagery-based BCI, which reduced the task difficulty and retained the volitional involvement of the user at the same time. The results of the experiments showed that the accuracy and efficiency of grasping tasks increased significantly in the shared control mode together with a significant decrease in the perceived mental workload, which indicates that our proposed shared control system is effective and user-friendly in practice. In the future, more feedback information will be introduced to enhance the task performance further, and a wheelchair-mounted robotic arm system will be developed for greater flexibility. In addition, more functional task modules (e.g., self-feeding and opening doors) should be integrated for more practical utilities.

User Experiences with Wheelchair Mounted Robotic Arm (WMRA)

To investigate the current use and outcomes of Wheelchair Mounted Robotic Arm (WMRA) as perceived by users.

Learning, Generalization, and Obstacle Avoidance with Dynamic Movement Primitives and Dynamic Potential Fields

In order to offer simple and convenient assistance for the elderly and disabled to take care of themselves, we propose a general learning and generalization approach for a service robot to accomplish specified tasks autonomously in an unstructured home environment. This approach firstly learns the required tasks by learning from demonstration (LfD) and represents the learned tasks with dynamic motion primitives (DMPs), so as to easily generalize them to a new environment only with little modification. Furthermore, we integrate dynamic potential field (DPF) with the above DMPs model to realize the autonomous obstacle avoidance function of a service robot. This approach is validated on the wheelchair mounted robotic arm (WMRA) by performing serial experiments of placing a cup on the table with an obstacle or without obstacle on its motion path.

Recent Advances on Human-Robot Interface of Wheelchair-Mounted Robotic Arm

Background: Wheelchair mounted robotic arm is a typical assistive robot, which is widely used to help the elders and the disabled to complete the activities of daily life. But limited by the restrictions of the users’ athletic ability and cognitive ability, how to flexibly manipulate such robot is still a problem in front of them. The human-computer interaction technology is the core technology of the robot. Its performance directly affects the user&amp;#039;s acceptability, satisfaction and promotion of intelligent wheelchairs. Objective: The study aims to give a general summary of recent human-robot interface of wheelchair mounted robotic arm and introduce their respective characteristics. Methods: Based on various patents and research developments about the human-robot interface of the assistive robot at home and abroad, this paper puts forward the basic principle of designing the humanrobot interaction mode, divides the man-robot interface into two categories based on the perspective of user control robot arm, and describes in detail, the typical human-robot interface and its related characteristics contained in each classification. Results: The development trends of the human-robot interface in future are predicted, so as to provide some research reference for the related scientific researchers. Conclusion: Wheelchair mounted robotic arm has important practical significance. Further improvements are needed in the design of the human-robot interface. It can effectively improve the operation performance of the WMRA, and take full advantage of the user’s existing movement ability to meet the requirement of dominating the control process. Furthermore, these improvements in the human-robot interface will allow more and more users to accept the WMRA, manipulate the WMRA, and enjoy improvements in the quality of their life for these assistive robots.

Learning motion primitives from demonstration

With the purpose to offer simple and convenient assistance for the elders with disabilities to take care of themselves in activities of daily living, we present a motion primitives learning method based on robot learning from demonstration to improve the intelligence and adaptability of the wheelchair-mounted robotic arm. This method adopts the beta process autoregressive hidden Markov model to segment the demonstrations of related task, acquire the contained motion primitives, and recognize the repeated motion primitives. After that, it adopts the dynamic movement primitives to adjust the related motion primitives according to the task instructions by the wheelchair-mounted robotic arm users, so as to replay the demonstrated task in a new environment. This learning framework is validated on a 6-degree-of-freedom JACO robotic arm, performing the tasks of drinking water from the bottle through a straw and pouring water from the bottle to the cup.

Design of inverse dynamics tracking control system and voice based interfacing mechanism to the WMRA

Majority of wheelchair users with neuromuscular diseases experience upperbody muscular weakness, which limit their ability to perform common activities of daily living. A Wheelchair Mounted Robotic Arm (WMRA) can assist these individuals to perform various tasks independently. This research paper is focused on designing a control system and interfacing mechanism to a newly designed WMRA. The proposed interfacing mechanism is based on virtual image and voice activated signals. The paper outlined the implementation through three dimensional imagining and a virtual workspace model of the real world for the arm to navigate through. The user speaks a simple command through a microphone and then be run through a control algorithm to drive the robotic arm. Inverse dynamics based control scheme is implemented to command the active joints of the robot, based on an input from the virtual CAD environment. The preliminary simulation results show that the virtual model joint trajectories are well tracked with minimal error. The paper also outlined the overall implementation of the system using Robot Operating System based interfacing system.

How Autonomy Impacts Performance and Satisfaction: Results From a Study With Spinal Cord Injured Subjects Using an Assistive Robot

We report a small dual cohort pilot study with traumatic spinal cord injured (SCI) subjects designed to investigate the utility of a wheelchair-mounted robotic arm for these subjects. The UCF-MANUS, a vision-based 6DOF assistive robotic arm, has been designed to aid individuals with upper limb extremities to complete tasks of daily living that they would otherwise be unable to complete themselves. Pick-and-place IADL tasks were designed and ten (10) users post-SCI were selected under IRB guidelines to be trained and tested with the system for 1 to 2 h weekly over a period of three weeks. During this time, they controlled the robot either through a manual or an autonomous (supervised) mode of operation. Baseline characteristics (pre-study), quantitative performance metrics (during study), and psychometrics (post-study) were obtained and statistically analyzed to test a set of hypotheses related to performance and satisfaction with the two control modes. At the end of the study, both the autonomous and the manual mode had comparable task completion times while user effort required for operating the robot in autonomous mode was significantly less than that for the manual mode. However, the autonomous mode failed to commensurately raise the user's level of satisfaction. Over the three-week study, the manual mode users showed a pronounced learning effect in terms of reducing mean task completion time and number of commands while the auto mode users showed improvement in terms of reduction of variability. Based on qualitative feedback and quantitative results, possible directions for system design are presented to concurrently achieve better performance and satisfaction outcomes.

“I Want That”: Human-in-the-Loop Control of a Wheelchair-Mounted Robotic Arm

Wheelchair-mounted robotic arms have been commercially available for a decade. In order to operate these robotic arms, a user must have a high level of cognitive function. Our research focuses on replacing a manufacturer-provided, menu-based interface with a vision-based system while adding autonomy to reduce the cognitive load. Instead of manual task decomposition and execution, the user explicitly designates the end goal, and the system autonomously retrieves the object. In this paper, we present the complete system which can autonomously retrieve a desired object from a shelf. We also present the results of a 15-week study in which 12 participants from our target population used our system, totaling 198 trials.

“I Want That”: Human-in-the-Loop Control of a Wheelchair-Mounted Robotic Arm

Wheelchair-mounted robotic arms have been commercially available for a decade. In order to operate these robotic arms, a user must have a high level of cognitive function. Our research focuses on replacing a manufacturer-provided, menu-based interface with a vision-based system while adding autonomy to reduce the cognitive load. Instead of manual task decomposition and execution, the user explicitly designates the end goal, and the system autonomously retrieves the object. In this paper, we present the complete system which can autonomously retrieve a desired object from a shelf. We also present the results of a 15-week study in which 12 participants from our target population used our system, totaling 198 trials.

Article 14: Effect of a wheelchair-mounted robotic arm on functional outcomes in persons with spinal cord injury

Objective: To determine the effect of a wheelchair-mounted robotic arm (WMRA) on function in persons with spinal cord injury (SCI). Design: Participants enrolled in the 2-week protocol. Setting: Veterans Affairs medical centers in Houston and Pittsburgh. Participants: 11 men (mean age, 42±12y) with cervical SCI (C3–4 to C6–7). All used power wheelchairs with joystick controls. Intervention: Training took place for 6 hours over 3 days. Subjects practiced activities of daily (ADL) tasks for 14 hours over 7 days. Tasks were timed and subjects were classified functionally as dependent (0), needs assistance (1), or independent (2). After 2 weeks, they were reevaluated for functional improvement with the WMRA. Main Outcome Measures: Number and time to complete tasks were calculated. Results: Mean ADL scores (independence) at reevaluation were significantly higher ( P<.02) than at baseline. Average ADL performance times decreased from 226 to 171 seconds, a statistically significant reduction in time ( P<.0001). Data revealed significant differences in subjects’ functional independence and time to complete tasks with the WMRA in 7 activities ( P<.05). Conclusions: The WMRA has the potential to enable persons with SCI to perform functional tasks that would otherwise be impossible for them. However, design problems (ie, added wheelchair width when WMRA is installed) interfered with maximum utilization and satisfaction.

KARES: Intelligent wheelchair-mounted robotic arm system using vision and force sensor

KARES, an acronym of KAIST Rehabilitation Engineering System, is the intelligent rehabilitation system with a 6 DOF robotic arm mounted on the powered wheelchair. It is developed to assist the disabled and the elderly for the independent activities. Human-machine interaction is a key issue of design for intelligent systems such as KARES. A special attention is paid in bestowing certain degree of autonomy to the robotic sub-system since the direct control of the robotic arm takes a high cognitive load on the user part while physically disabled persons may have difficulties in dexterously operating a joystick or push buttons for delicate movements. To perceive environment, one color vision sensor and one force/torque sensor are mounted on the end-effector of the robotic arm of KARES. To test the system, four basic tasks are defined as picking up a cup on the table, picking up a pen on the floor, moving an object to the user's face, and operating a switch on the wall. These tasks are performed autonomously in a semi-structured environment.

An Electric Wheelchair Mounted Robotic Arm—A Survey of Potential Users

This paper describes the results of a survey which investigated and evaluated the needs and abilities of electric wheelchair users. The results of this survey will be used to develop a low-cost electric wheelchair-mounted robotic arm for use by physically disabled people to facilitate rehabilitation. The survey was undertaken by the author together with staff and students from occupational therapist training colleges, using a four-page questionnaire containing over 110 questions. The questionnaire was developed by the author together with Dr Robin Platts (Director of Orthotics), Mr Ian Bayley (Director of the London Spinal Unit) and senior occupational therapists at the Royal National Orthopaedic Hospital, Stanmore, Middlesex. After a successful trial the questionnaire was used with 50 severely disabled people from various backgrounds and social circumstances. The results of this survey show that the average electric wheelchair user is 40 years old, single (68%), living at home (58%) with family support (69%) and without any paid employment (79%). The most prevalent disability is spinal cord injury (24%) followed by multiple sclerosis (16%). The survey has identified several tasks which electric wheelchair users find impossible to do, and some of these will form part of the design specification. Finally 84% of the survey subjects would consider buying such a robotic aid.