Wheelchair Control Research Articles

Combined control of rehabilitation wheelchair using periocular electromyography and electroencephalography

To improve the convenience of life for the people with reduced mobility, a combined control method of wheelchair utilizing periocular electromyography (Per-EMG) and electroencephalography (EEG) is presented. Based on the Per-EMG and EEG signals obtained from the bioelectric sensors, a novel feature classification combined model is proposed by combining convolutional neural network (CNN) and long short-term memory (LSTM) neural network. These two deep learning architectures enable the comprehensive analysis and accurate classification of the acquired signals. Then the inferencing results can be converted to the corresponding driving command of the rehabilitation wheelchair. Furthermore, the important metrics such as accuracy, precision and recall are adopted to evaluate the performance of this combined model. These metrics provide a quantitative assessment of the model’s classification capabilities. By practical experiments, the proposed combined control method for rehabilitation wheelchair demonstrates its reasonability and effectiveness. And the wheelchair with combined control method can enhance the mobility and independence of the people with reduced mobility. These findings contribute to the development of assistive technologies in the field of rehabilitation.

Signature methods for brain-computer interfaces

Brain-computer interfaces (BCIs) allow direct communication between one’s central nervous system and a computer without any muscle movement hence by-passing the peripheral nervous system. They can restore disabled people’s ability to interact with their environment, e.g. communication and wheelchair control. However, to this day their performance is still hindered by the non-stationarity of electroencephalography (EEG) signals, as well as their susceptibility to noise from the users’ environment and from their own physiological activity. Moreover, a non-negligible amount of users struggle to use BCI systems based on motor imagery. In this paper, a new method based on the path signature is introduced to tackle this problem by using features which are different from the usual power-based ones. The path signature is a series of iterated integrals computed from a multidimensional path. It is invariant under translation and time reparametrization, which makes it a robust feature for multichannel EEG time series. The performance can be further boosted by combining the path signature with the gold standard Riemannian classifier in the BCI field exploiting the geometric structure of symmetric positive definite (SPD) matrices. The results obtained on publicly available datasets show that the signature method is more robust to inter-user variability than classical ones, especially on noisy and low-quality data. Hence, this study paves the way towards the use of mathematical tools that until now have been neglected, in order to tackle the EEG-based BCI variability issue. It also sheds light on the lead-lag relationship captured by path signature which seems relevant to assess the underlying neural mechanisms.

Dual-Quaternion-Based SLERP MPC Local Controller for Safe Self-Driving of Robotic Wheelchairs

In this work, the motion control of a robotic wheelchair to achieve safe and intelligent movement in an unknown scenario is proposed. The primary objective is to develop a comprehensive framework for a robotic wheelchair that combines a global path planner and a model predictive control (MPC) local controller. The A* algorithm is employed to generate a global path. To ensure safe and directional motion for the wheelchair user, an MPC local controller is implemented taking into account the via points generated by an approach combined with dual quaternions and spherical linear interpolation (SLERP). Dual quaternions are utilized for their simultaneous handling of rotation and translation, while SLERP enables smooth and continuous rotation interpolation by generating intermediate orientations between two specified orientations. The integration of these two methods optimizes navigation performance. The system is built on the Robot Operating System (ROS), with an electric wheelchair equipped with 3D-LiDAR serving as the hardware foundation. The experimental results reveal the effectiveness of the proposed method and demonstrate the ability of the robotic wheelchair to move safely from the initial position to the destination. This work contributes to the development of effective motion control for robotic wheelchairs, focusing on safety and improving the user experience when navigating in unknown environments.

Compressive sensing applied to SSVEP-based brain–computer interface in the cloud for online control of a virtual wheelchair

People with severe motor disabilities have great challenges in achieving independence, particularly driving or controlling manually robotic wheelchairs. Existing assistive technologies for robotic wheelchair control often fail to address user needs adequately, becoming necessary an appropriate evaluation of usability and user’s satisfaction for these systems. In fact, user experience and satisfaction are crucial in determining effectiveness, usability and acceptance of these technologies. The aim is to enhance the quality of life of individuals with severe motor disabilities, increasing their independence to perform some daily activities, such as locomotion. This study proposes a cloud Steady-State Visual Evoked Potential (SSVEP)-based Brain–Computer Interface (BCI) system to evaluate its performance through the user experience in commanding a virtual electric-powered wheelchair in a training simulator. Moreover, this study compares traditional wheelchair command modes, such as input interfaces using JoyStick (JS) and Eye Tracker (ET), through standardized questionnaires to measure motivation, ease of use, workload, discomfort, sense of presence and other relevant parameters. The analysis of user experience reveals high levels of motivation, enjoyment, and usability when using the cloud SSVEP-based BCI proposed here. Although our BCI demands higher mental work and discomforts than both ET and JS, its potential for improving the user’s performance is demonstrated here. The findings of this research provide valuable insights for researchers and practitioners in the field of assistive technologies, enabling the development of more personalized systems based on the user’s experience. The main contributions of this study lie in a cloud BCI approach and a comprehensive evaluation methodology to evaluate the user’s experience.

Combining Mindwave, MPU6050, Ultrasonic, and Internet of Things for a reliable, safe, and monitored wheelchair control system

This paper proposes the development of a wheelchair control system by combining a Mindwave EEG (electroencephalogram) sensor, an MPU6050 inertial sensor, an ultrasonic sensor, and IoT technology. Stroke survivors often face challenges in controlling their wheelchairs due to cognitive and physical impairments, requiring specific solutions to ensure their safety and comfort. The integration of the Mindwave EEG sensor captures the user's cognitive state, such as level of attention and focus, while the MPU6050's inertial sensor monitors physical movement and changes in head orientation. Additionally, the ultrasonic sensor is incorporated to detect obstacles and provide proximity information for safe navigation. This wheelchair control system is also linked with IoT technology to enable monitoring of the user's cognitive, physical, and environmental condition and remote assistance. In this paper, to implement this wheelchair control system, a T-BEAM with an ESP32 controller is used, which is equipped with GPS to provide wheelchair location data and LORA radio communication for remote data transmission, and a TTGO LORA32 to forward the data to Arduino IoT Cloud service over Wi-Fi. The results of the development show that the wheelchair control system is safer and more monitored, and users are easier and more flexible in operating wheelchairs.

A hybrid BCI combining SSVEP and EOG and its application for continuous wheelchair control

Brain-computer interfaces (BCIs) have emerged as one of the most promising techniques for individuals with motor disabilities to control external devices directly without peripheral pathways. While the discrete control strategy is widely adopted in BCI systems, the continuous control strategy provides intuitive and fluent control of robotic devices. However, the steady-state visual evoked potential (SSVEP)-based BCI is still far from practical applications of continuous control. One of the existing challenges is the long erroneous transition time, i.e., the duration from when the subject switches towards a new target to when the BCI system recognizes this new target. Therefore, this study proposed a hybrid BCI combining SSVEP and electrooculography (EOG). First, the prior probability distribution of the intended target was obtained by the SSVEP detection method. Different from directly picking the one with the maximum probability as the conventional SSVEP-based BCI did, the captured saccade event in the EOG signal served as an additional event to optimize this probability distribution, and the target was then extrapolated and outputted. An offline experiment simulating the target switching process and an online experiment of virtual wheelchair control were designed and conducted. The proposed hybrid BCI significantly outperformed the conventional SSVEP-based BCI in both the offline experiment (continuous accuracy:64.99 ± 3.44 % vs. 55.76 ± 3.67 %, transition time: 1.19 ± 0.073 s vs. 1.65 ± 0.08 s) and the online wheelchair control experiment (success rate: 81.67 ± 5.75 % vs. 16.67 %± 3.76 %; collision times with the dynamic obstacle: 1.50 ± 0.50 vs. 8.25 ± 0.37; minimum distance to dynamic obstacle: 1.41 ± 0.13 m vs. 0.31 ± 0.08 m). The results revealed that by combining the SSVEP and EOG signals, the proposed hybrid BCI can significantly diminish the transition time in SSVEP-based BCI and achieve continuous, fluent control. The present study provides a novel framework for integrating the SSVEP and EOG, and promotes the potential applications of continuous brain-actuated robotic devices.

Augmented wheelchair control for collision avoidance

An electric wheelchair gives people with disabilities the ability to move independently and participate in social activities. However, the control of wheelchairs requires concentration, foresight, and spatial orientation, which often deteriorate with the progression of a disease and with age. The loss of these efficiency can be significantly compensated by the safety system. The research presented on augmentation in manual control of the wheelchair is carried out. Although most of the existing propositions focus on stopping before an obstacle, bypassing it, or automatic navigation to the indicated area, they do not resolve safe motion in tight spaces and areas with many obstacles. This article presents a new approach to collision controller for manual control. The proposed solution takes into account the situation in which the driver of the electric wheelchair uses it as a tool to move moveable objects in its surroundings. The developed and tested system provides safe and controlled contact with the obstacle. As a result, the intent to move resulting from joystick deflection causes a controlled low-speed motion of the object. This article contains descriptions of the control method and the research stand with the wheelchair, simulation results and experimental studies. The simulation model was used to analyse dynamics of the controlled object, select collision controller parameters and test controller operation. The results of wheelchair tests and simulations are similar and consistent with the assumed mode of operation, confirming correctness of the proposed control method.

Electric powered wheelchair control using user-independent classification methods based on surface electromyography signals.

Wheelchairs are one of the most popular assistive technology (AT) among individuals with motor impairments due to their comfort and mobility. People with finger problems may find it difficult to operate wheelchairs using the conventional joystick control method. Therefore, in this research study, a hand gesture-based control method is developed for operating an electric-powered wheelchair (EPW). This study selected a comfort-based hand position to determine the stop maneuver. An additional exploration was undertaken to investigate four gesture recognition methods: linear regression (LR), regularized linear regression (RLR), decision tree (DT), and multi-class support vector machine (MC-SVM). The first two methods, LR and RLR, have promising accuracy values of 94.85% and 95.88%, respectively, but each new user must be trained. To overcome this limitation, this study explored two user-independent classification methods: MC-SVM and DT. These methods effectively addressed the finger dependency issue and demonstrated remarkable success in recognizing gestures across different users. MC-SVM has about 99.05% of both precision and accuracy, and the DT has about 97.77% accuracy and precision. All six participants were successful in controlling the EPW without any collisions. According to the experimental results, the proposed approach has high accuracy and can address finger dependency issues.

Environmental Intervention Evaluation to Support Motor Planning of Wheelchair Movements: A Pilot Study

Environmental interventions provide a promising opportunity to support inclusive mobility in our rapidly changing society and workforce. Concurrently, the trends for aging into mobility disability, working into older age, and health and economic burdens of unpaid caregiving continue to rise. Hence, providing equitable mobility for all is an urgent need. In the context of manual wheelchair control, a novel method investigating the impact of environmental interventions on individual’s internal representation of motor actions is proposed. In addition to measuring objective and subjective performance metrics, we propose that an understanding of how internal representations of tasks are influenced by user perception of the environment has promising potential towards informing effective and inclusive task-environment interactions. The application explored in this pilot study aims to facilitate mobility.

Towards hippocampal navigation for brain–computer interfaces

Automatic wheelchairs directly controlled by brain activity could provide autonomy to severely paralyzed individuals. Current approaches mostly rely on non-invasive measures of brain activity and translate individual commands into wheelchair movements. For example, an imagined movement of the right hand would steer the wheelchair to the right. No research has investigated decoding higher-order cognitive processes to accomplish wheelchair control. We envision an invasive neural prosthetic that could provide input for wheelchair control by decoding navigational intent from hippocampal signals. Navigation has been extensively investigated in hippocampal recordings, but not for the development of neural prostheses. Here we show that it is possible to train a decoder to classify virtual-movement speeds from hippocampal signals recorded during a virtual-navigation task. These results represent the first step toward exploring the feasibility of an invasive hippocampal BCI for wheelchair control.

Development of a Gaze-Driven Electric Wheelchair with 360° Camera and Novel Gaze Interface

A novel gaze-based user interface (UI) is proposed for the remote control of robots and electric wheelchairs. A task-based experiment showed that this UI is more suitable for remote control than a conventional interface. The UI was applied to the control of a commercially available low-cost electric wheelchair. By using a 360° camera and an eye tracker that can be used outdoors, the visibility of obstacles and the usability of the gaze-driven electric wheelchair were greatly improved. The gaze-driven electric wheelchair exhibited good performance in a task-based evaluation experiment.

Design and Evaluation of the Extended FBS Model Based Gaze-Control Power Wheelchair for Individuals Facing Manual Control Challenges.

This study addresses the challenges faced by individuals with upper limb disadvantages in operating power wheelchair joysticks by utilizing the extended Function-Behavior-Structure (FBS) model to identify design requirements for an alternative wheelchair control system. A gaze-controlled wheelchair system is proposed based on design requirements from the extended FBS model and prioritized using the MosCow method. This innovative system relies on the user's natural gaze and comprises three levels: perception, decision making, and execution. The perception layer senses and acquires information from the environment, including user eye movements and driving context. The decision-making layer processes this information to determine the user's intended direction, while the execution layer controls the wheelchair's movement accordingly. The system's effectiveness was validated through indoor field testing, with an average driving drift of less than 20 cm for participates. Additionally, the user experience scale revealed overall positive user experiences and perceptions of the system's usability, ease of use, and satisfaction.

MSATNet: multi-scale adaptive transformer network for motor imagery classification.

Motor imagery brain-computer interface (MI-BCI) can parse user motor imagery to achieve wheelchair control or motion control for smart prostheses. However, problems of poor feature extraction and low cross-subject performance exist in the model for motor imagery classification tasks. To address these problems, we propose a multi-scale adaptive transformer network (MSATNet) for motor imagery classification. Therein, we design a multi-scale feature extraction (MSFE) module to extract multi-band highly-discriminative features. Through the adaptive temporal transformer (ATT) module, the temporal decoder and multi-head attention unit are used to adaptively extract temporal dependencies. Efficient transfer learning is achieved by fine-tuning target subject data through the subject adapter (SA) module. Within-subject and cross-subject experiments are performed to evaluate the classification performance of the model on the BCI Competition IV 2a and 2b datasets. The MSATNet outperforms benchmark models in classification performance, reaching 81.75 and 89.34% accuracies for the within-subject experiments and 81.33 and 86.23% accuracies for the cross-subject experiments. The experimental results demonstrate that the proposed method can help build a more accurate MI-BCI system.

IOT Based Voice Controlled Wheel Chair

IOT Based Voice Controlled Wheel Chair

Wheelchair control system with hand movement using accelerometer sensor

Disability is a condition where there is an imbalance of interaction between biological conditions with the surrounding environment, which includes disruption, limited activities, and restrictions on participation. With the increasing population of people with disabilities in walking, either due to congenital physical disabilities, accidents, or motor functions due to aging, the need for wheelchairs increases. In this research, a wheelchair control system with hand movements using an accelerometer sensor was made. The system consists of a transmitter on the back of the hand and a receiver on the wheelchair. In the transmitter's part, MPU-6050 sensor detects hand movements, whereby hand movements can be upward, downward, straight, left tilted, and right tilted. The data from MPU-6050 sensor was processed by Arduino Nano then sent using the nRF24L01 module. In the receiver's part, the data were processed by Arduino Uno which activates the BTS7960 motor drive module and drives the DC motor. The test results show that hand movements with an angle of 45° can move the wheelchair forward, backward, turn right, turn left, and stop at a speed of 2 km/h with a maximum test load of 80 kg.

Intelligent Eye-Controlled Electric Wheelchair Based on Estimating Visual Intentions Using One-Dimensional Convolutional Neural Network and Long Short-Term Memory.

When an electric wheelchair is operated using gaze motion, eye movements such as checking the environment and observing objects are also incorrectly recognized as input operations. This phenomenon is called the "Midas touch problem", and classifying visual intentions is extremely important. In this paper, we develop a deep learning model that estimates the user's visual intention in real time and an electric wheelchair control system that combines intention estimation and the gaze dwell time method. The proposed model consists of a 1DCNN-LSTM that estimates visual intention from feature vectors of 10 variables, such as eye movement, head movement, and distance to the fixation point. The evaluation experiments classifying four types of visual intentions show that the proposed model has the highest accuracy compared to other models. In addition, the results of the driving experiments of the electric wheelchair implementing the proposed model show that the user's efforts to operate the wheelchair are reduced and that the operability of the wheelchair is improved compared to the traditional method. From these results, we concluded that visual intentions could be more accurately estimated by learning time series patterns from eye and head movement data.

Brain-Computer Interface Based on Steady-State Visual Evoked Potential Using Quick-Response Code Pattern for Wheelchair Control.

Brain-computer interfaces (BCIs) are widely utilized in control applications for people with severe physical disabilities. Several researchers have aimed to develop practical brain-controlled wheelchairs. An existing electroencephalogram (EEG)-based BCI based on steady-state visually evoked potential (SSVEP) was developed for device control. This study utilized a quick-response (QR) code visual stimulus pattern for a robust existing system. Four commands were generated using the proposed visual stimulation pattern with four flickering frequencies. Moreover, we employed a relative power spectrum density (PSD) method for the SSVEP feature extraction and compared it with an absolute PSD method. We designed experiments to verify the efficiency of the proposed system. The results revealed that the proposed SSVEP method and algorithm yielded an average classification accuracy of approximately 92% in real-time processing. For the wheelchair simulated via independent-based control, the proposed BCI control required approximately five-fold more time than the keyboard control for real-time control. The proposed SSVEP method using a QR code pattern can be used for BCI-based wheelchair control. However, it suffers from visual fatigue owing to long-time continuous control. We will verify and enhance the proposed system for wheelchair control in people with severe physical disabilities.

Wheelchair Control Using Bluetooth-Based Electromyography Signals

In modern times like this, many wheelchairs have been developed with various controls, ranging from manual ones, namely by being pushed by other people or using their hands to turn the wheel, to automatic ones, such as electric wheelchairs that use joysticks and Electromyography control. The control of the electromyography signal utilizes muscles that can still be used to move the wheelchair, in this case, using the hand muscles. The use of a Bluetooth wireless system in sending electromyography signals aims to facilitate the use of a wheelchair without interference from the many connected cables so that users are more flexible in placing the electromyography sensor on the user's hand muscles. By placing the electromyography sensor on the user's arm, the electromyography sensor detects a contraction or relaxation, which is indicated by the LED flame. The output value of the sensor will be compared with a predetermined limit value. When the value is greater than the limit value, it will produce a logic low; when the value is less than the limit value, it will build a logic high. The Arduino microcontroller will calculate every low logic. The results of these calculations will be processed into serial data. The serial data will be sent to the HC-05 enslaved person via the HC-05 master wirelessly. The motor driver will execute the data so that it produces motion forward, backward, turn right, turn left and stop. It is hoped that this tool can help individuals with limited movement so that they do not have difficulty in mobility.

A Survey on Visual based Wheelchair Control System for ALS Impaired Patients

A Survey on Visual based Wheelchair Control System for ALS Impaired Patients

SmartRolling: A human–machine interface for wheelchair control using EEG and smart sensing techniques

Smart wheelchairs based on brain–computer interface (BCI) have been widely utilized recently to address certain mobility problems for people with disability. In this paper, we present SmartRolling, an intuitive human–machine interaction approach for the direct control of robotic wheelchair that jointly leverages EEG signals and motion sensing techniques. Specifically, SmartRolling offers two wheelchair-actuation modes for users with different physical conditions: (1) head motion only — people who are severely disabled but able to do basic tasks using eyes and head, and (2) head and hands motion — in addition to type 1, people who can use functioning hands/arms for extra tasks. The system issues operation commands by recognizing different EEG patterns elicited by motor execution (ME) tasks including eye blink, jaw clench, and fist open/close, while at the same time estimates users’ steering intentions based on their facing direction by leveraging inertial measurements and computer vision techniques. The experiment results demonstrate that the proposed system is robust and effective to meets the individual’s needs and has great potential to promote better health.