Standard Joystick Research Articles

SpheriCol: A Driving Assistant for Power Wheelchairs Based on Spherical Vision

This paper presents a new vision-based navigation assistant for power wheelchairs, called SpheriCol. Inspired by the parking assist system in cars, an omnidirectional image captured by a twin-fisheye camera is overlaid with colored distance markers from range sensors mounted on the wheelchair, to generate an augmented view of the surrounding environment. Such an image is displayed in real time to the user on a screen. To evaluate the effectiveness of the proposed system, able-bodied subjects and older adults with motor impairments have been asked to drive a consumer-grade power wheelchair equipped with SpheriCol, via a standard joystick. Our clinical trials with different obstacle courses, consistently indicate that SpheriCol is effective in improving safety and comfort, and in supporting a driver’s decision during challenging but prevalent maneuvers, such as reversing out of an elevator, corridor centering, and turning on the spot.

Design and Evaluation of an Augmented Reality Head-mounted Display Interface for Human Robot Teams Collaborating in Physically Shared Manufacturing Tasks

We provide an experimental evaluation of a wearable augmented reality (AR) system we have developed for human-robot teams working on tasks requiring collaboration in shared physical workspace. Recent advances in AR technology have facilitated the development of more intuitive user interfaces for many human-robot interaction applications. While it has been anticipated that AR can provide a more intuitive interface to robot assistants helping human workers in various manufacturing scenarios, existing studies in robotics have been largely limited to teleoperation and programming. Industry 5.0 envisions cooperation between human and robot working in teams. Indeed, there exist many industrial tasks that can benefit from human-robot collaboration. A prime example is high-value composite manufacturing. Working with our industry partner towards this example application, we evaluated our AR interface design for shared physical workspace collaboration in human-robot teams. We conducted a multi-dimensional analysis of our interface using established metrics. Results from our user study (n = 26) show that, subjectively, the AR interface feels more novel and a standard joystick interface feels more dependable to users. However, the AR interface was found to reduce physical demand and task completion time, while increasing robot utilization. Furthermore, user’s freedom of choice to collaborate with the robot may also affect the perceived usability of the system.

A Dedicated Tool Frame Based Tongue Interface Layout Improves 2D Visual Guided Control of an Assistive Robotic Manipulator: A Design Parameter for Tele-Applications

It is crucial to increase the independence of severely disabled individuals. Assistive robotics can aid in the desired activities of daily living, including tasks requiring remote performance e.g. grasping remote objects, turning switches on/off, and opening/closing doors. The robot control is compromised by the lack of efficient interfaces for individuals with disabilities and the lack of depth perception. This paper addresses these challenges by presenting the development and evaluation of efficient tongue-based robot interfaces and low-level robot control schemes targeting tele-robotic control through a 2D display. Ten able-bodied participants were successful in completing ten rounds of controlling a JACO robot to perform a pouring water task, using five different control methods, under 2D or 3D visual feedback. The tool-frame based tongue interface layout, L2_TF (with emulated joystick, mode switch button and a “GO” button) improved the 2D visual guided control of the JACO robot compared with the other tongue control methods. The mean trajectory length of completing the task using L2_TF was 3% longer compared with the standard joystick when controlling through 2D. The trajectory length for reaching and grabbing a bottle was shortest for L2_TF compared with all other control methods, including the joystick. The iTongue control layouts performed well in gripping time, showing no significant difference between 2D and 3D. The transition from 2D to 3D resulted in a mean decrease of 27.7% for task completion time across all interfaces. L2_TF and the joystick had the strongest and most similar robustness to the transition between 3D and 2D.

Wheelchair driving strategies: A comparison between standard joystick and gaze-based control

ABSTRACT This paper aims to evaluate and compare the driving performances achieved with a power wheelchair using a standard joystick versus a novel gaze-based technology. The gaze-based interface, called RoboEYE, involves a novel paradigm of computer interaction that handles the receipt of information from an eye tracker, using it as a continuous input for wheelchair navigation. A pool of 36 subjects has tested both technologies in a circuit designed considering the Wheelchair Skill Test. The experimental analysis involved evaluations of specific metrics of motion and the submission of questionnaires to collect required information about perceived feelings and mental workload. The joystick proved to be the best driving interface. It turned out to be more accurate and efficient than the gaze-based solution. However, the latter achieved only small differences in driving kinematics. These differences can be considered negligible from an operational point of view, offering a driving experience similar to that achievable with the joystick. Testers reported no particular stress, fatigue, or frustration when switching from one interface to another. These elements suggest that the proposed gaze-based solution is an appropriate alternative for a technology transition driven by a pathological change in the user’s condition.

Implementation of wheelchair controller using mouth and tongue gesture

In this paper, a simple human-machine interface allowing people with severe disabilities to control a motorized wheelchair using mouth and tongue gesture is presented. The development of the proposed system consists of three principal phases: the first phase is mouth detection which performed by using haar cascade to detect the face area and template matching to detect mouth and tongue gestures from the lower face region. The second phase is command extraction; it is carried by determining the mouth and tongue gesture commands according to the detected gesture, the time taken to execute the gestures, and the previous command which is stored in each frame. Finally, the gesture commands are sent to the wheelchair as instruction using the Bluetooth serial port. The hardware used for this project were; laptop with universal serial bus (USB) webcam as a vision-based control unit, Bluetooth module to receive instructions comes from the vision control unit, standard joystick used in case of emergency, joystick emulator which delivers to the control board signals similar to the signals that are usually generated by the standard joystick, and ultrasonic sensors to provide safe navigation. The experimental results showed the success of the proposed control system based on mouth and tongue gestures.

Evaluation of an exercise-enabling control interface for powered wheelchair users: a feasibility study with Duchenne muscular dystrophy

BackgroundPowered wheelchairs are an essential technology to support mobility, yet their use is associated with a high level of sedentarism that can have negative health effects for their users. People with Duchenne muscular dystrophy (DMD) start using a powered wheelchair in their early teens due to the loss of strength in their legs and arms. There is evidence that low-intensity exercise can help preserve the functional abilities of people with DMD, but options for exercise when sitting in a powered wheelchair are limited.MethodsIn this paper, we present the design and the feasibility study of a new version of the MOVit device that allows powered-wheelchair users to exercise while driving the chair. Instead of using a joystick to drive the wheelchair, users move their arms through a cyclical motion using two powered, mobile arm supports that provide controller inputs to the chair. The feasibility study was carried out with a group of five individuals with DMD and five unimpaired individuals. Participants performed a series of driving tasks in a wheelchair simulator and on a real driving course with a standard joystick and with the MOVit 2.0 device.ResultsWe found that driving speed and accuracy were significantly lowered for both groups when driving with MOVit compared to the joystick, but the decreases were small (speed was 0.26 m/s less and maximum path error was 0.1 m greater). Driving with MOVit produced a significant increase in heart rate (7.5 bpm) compared to the joystick condition. Individuals with DMD reported a high level of satisfaction with their performance and comfort in using MOVit.ConclusionsThese results show for the first time that individuals with DMD can easily transition to driving a powered wheelchair using cyclical arm motions, achieving a reasonable driving performance with a short period of training. Driving in this way elicits cardiopulmonary exercise at an intensity found previously to produce health-related benefits in DMD.

Design and evaluation of an alternative wheelchair control system for dexterity disabilities.

This work details the design and development of a novel 3D printed, modular alternative wheelchair control system for powered wheelchair users afflicted with dexterity inhibiting disorders, which mechanically interfaces directly with the installed standard joystick. The proposed joystick manipulator utilises an accelerometer for gesture control input processed by the Arduino microprocessor and a mechanical control interface, which sits over a standard installed two-axis proportional joystick, the preferential control system for most powered chair manufacturers. When fitted, this allows powered electric wheelchair users with limited dexterity, independent to navigate their wheelchair unassisted. The mechanical system has been selected so that the joystick manipulator is as universal as possible and can be installed to almost any powered wheelchair that uses a two-axis joystick. The design process and key aspects of the operation of the joystick manipulator are presented as well as field testing on a wheelchair conducted. The test results show that the proposed joystick manipulator is a successful system that can be universally fitted to most powered chairs and offers potentially greater independence for the powered wheelchair user.

Development of a New Intelligent Joystick for People with Reduced Mobility.

Despite the diversity of electric wheelchairs, many people with physical limitations and seniors have difficulty using their standard joystick. As a result, they cannot meet their needs or ensure safe travel. Recent assistive technologies can help to give them autonomy and independence. This work deals with the real-time implementation of an artificial intelligence device to overcome these problems. Following a review of the literature from previous work, we present the methodology and process for implementing our intelligent control system on an electric wheelchair. The system is based on a neural algorithm that overcomes problems with standard joystick maneuvers such as the inability to move correctly in one direction. However, this implies the need for an appropriate methodology to map the position of the joystick handle. Experiments on a real wheelchair are carried out with real patients of the Mohamed Kassab National Institute Orthopedic, Physical and Functional Rehabilitation Hospital of Tunis. The proposed intelligent system gives good results compared to the use of a standard joystick.

Intelligent Control Wheelchair Using a New Visual Joystick.

A new control system of a hand gesture-controlled wheelchair (EWC) is proposed. This smart control device is suitable for a large number of patients who cannot manipulate a standard joystick wheelchair. The movement control system uses a camera fixed on the wheelchair. The patient's hand movements are recognized using a visual recognition algorithm and artificial intelligence software; the derived corresponding signals are thus used to control the EWC in real time. One of the main features of this control technique is that it allows the patient to drive the wheelchair with a variable speed similar to that of a standard joystick. The designed device “hand gesture-controlled wheelchair” is performed at low cost and has been tested on real patients and exhibits good results. Before testing the proposed control device, we have created a three-dimensional environment simulator to test its performances with extreme security. These tests were performed on real patients with diverse hand pathologies in Mohamed Kassab National Institute of Orthopedics, Physical and Functional Rehabilitation Hospital of Tunis, and the validity of this intelligent control system had been proved.

Differences in outcomes between the JoyBar control and standard wheelchair joystick control on two maneuverability tasks: a pilot study

Purpose: To determine if older adult, novice wheelchair users who drive a power wheelchair with a JoyBar control complete maneuverability tasks in less time and with less error than those who drive a power wheelchair with a standard joystick control.Materials and methods: A parallel randomized controlled trial design conducted at a medical rehabilitation and research centre with ambulatory older adults aged 60 and above (n = 27). The intervention was the JoyBar alternative wheelchair control. The primary outcome measure was total time to complete each of the two maneuverability tasks. The secondary outcome measure was total number of errors during each of the maneuverability tasks.Results: An independent, two sampled t-test was conducted and revealed that the JoyBar group took a greater amount of time to complete both maneuverability tasks than the control group (p < .05). No significant differences (p < .05) were found in rates of error on either task between the JoyBar and joystick groups.Conclusions: Maneuverability of a powered wheelchair by novice wheelchair users was not improved through the use of the JoyBar when compared to a standard wheelchair joystick, as measured by rates of error and time to complete maneuverability tasks.Implications for rehabilitationClients who are new to powered wheelchair use may perform maneuverability tasks faster, with equivalent accuracy, using a standard joystick versus the JoyBar.Clients who use a JoyBar may require adjustments to the programming of their wheelchair to ensure optimal performance.Additional training may be required to achieve proficiency in maneuverability tasks with a JoyBar versus a standard joystick.

A New Input Device for Spastics Based on Strain Gauge.

This article presents a new sensor for use by people with spastic disorders and similar conditions and enables them to steer and control medical devices such as electric powered wheelchairs. As spastic patients often suffer from cramping of their extremities, which can then no longer be controlled, using a standard joystick while operating a powered wheelchair can lead to dangerous situations. To prevent this, we designed a sensor based on strain gauges, which is shaped like a flat disc that can be operated using any body part. By shifting weight along the x- and y-axis, the disc tilts in all directions thereby generating proportionate output signals. The disc can also be pressed downward (z-axis), for example, to open a wheelchair’s menu. Thanks to the sensor’s flat disc-like construction and the option of mounting it into a control panel, users are not in danger of becoming stuck on the disc during spastic episodes. In the event of a spasm, body parts simply slide over the disc reducing risk of unintended actions. The sensor is adaptive and adjustable enabling it to fit a user’s range of strength and motion at any time. It was developed to ensure users can operate sensitive systems safely.

A New, Adaptable, Optical High-Resolution 3-Axis Sensor.

This article presents a new optical, multi-functional, high-resolution 3-axis sensor which serves to navigate and can, for example, replace standard joysticks in medical devices such as electric wheelchairs, surgical robots or medical diagnosis devices. A light source, e.g., a laser diode, is affixed to a movable axis and projects a random geometric shape on an image sensor (CMOS or CCD). The downstream microcontroller’s software identifies the geometric shape’s center, distortion and size, and then calculates x, y, and z coordinates, which can be processed in attached devices. Depending on the image sensor in use (e.g., 6.41 megapixels), the 3-axis sensor features a resolution of 1544 digits from right to left and 1038 digits up and down. Through interpolation, these values rise by a factor of 100. A unique feature is the exact reproducibility (deflection to coordinates) and its precise ability to return to its neutral position. Moreover, optical signal processing provides a high level of protection against electromagnetic and radio frequency interference. The sensor is adaptive and adjustable to fit a user’s range of motion (stroke and force). This recommendation aims to optimize sensor systems such as joysticks in medical devices in terms of safety, ease of use, and adaptability.

Design of voice-controlled wheelchair for quadriplegic patients

Powered wheelchairs with standard joysticks are difficult to control for 'Quadriplegics' patients. Wheelchair movement based on speaker's voice command will be helpful for the physically challenged people to mobilise themselves. So in this work a set of ten male and female voice commands (FORWARD, BACKWARD, LEFT, RIGHT and STOP) were recorded at different instances using a microphone and analysed how patient's voice differs from others by considering the parameters of amplitude, frequency and phase in the MATLAB software. FFT technique is used for feature extraction and smoothing the patient's voice signal. The processed signal is stored as a reference signal in LabVIEW for interfacing with hardware. The powered wheelchair movement depends on motor control and a drive system which consists of PIC 16F877A microcontroller and DC motor. Once the voice recognisation system identifies the voice command uttered by the patient, it is coded into digital signal and transferred to the PIC microcontroller which then controls the wheelchair accordingly.

Control System of Powered Wheelchairs Based on Tongue Motion Detection

A new intelligent powered wheelchair is urgently needed for the individuals with tetraplegia and similar impairments who are unable to use the standard joystick. Based on the tongue motion detection, a new control system is introduced in this paper which is helpful for users to operate powered wheelchair efficiently and easily. This article introduces two control modes, including tongue motion control mode and infrared control mode. Wherein the infrared control mode mainly use the infrared controller. The tongue motion can be detected with several vibration film sheets that were embedded in the headset and a standard analog signal can be generated with embedded controller to control the wheelchair. The tongue motion drive system integrated into headset was developed and the control of the wheelchair has been tested moving along the designed route. Preliminary results show that the system is simple and convenient to control powered wheelchair with low cost, which has potential application in intelligent control domain.

Analysis of Optimal Brainwave Concentration Model for Wheelchair Input Interface

Users with severe motor ability are unable to control their wheelchair using standard joystick and hence an alternative brainwave control input is preferred. To use such device a concentration is necessary to ensure the user can move as intended. However the model of concentration that can be used for such purpose is still an open problem and need further investigation. In this paper an analysis is performed to determine an optimal concentration model to enable the user to control the wheelchair movement easily. The model is analyzed under two conditions which are in visible mode (user look and focus at the picture) and invisible mode (user imagine about the picture). Each of the pictures represents the wheelchair's movements which are left, right, front and reverse. The EEG signal is analyzed by utilizing signal peak and average features generated by using Fast Fourier Transform (FFT) from the EEG's attention sensing channel. From the experiments, it is concluded that the EEG signal generate in visible mode with colour model shows high performance to distinguish between four wheelchairs movements compared to the invisible model.

Input device for electric wheelchairs considering physical functions of persons with severe Duchenne muscular dystrophy

Persons with severe Duchenne muscular dystrophy (DMD) cannot operate electric wheelchairs through standard joysticks because they lack muscular power and are affected by deformation of the hands. To address this difficulty, first the authors determined the requirements of input devices based on a survey of the daily life of persons with DMD and the authors developed new joysticks for an electric wheelchair considering the characteristics of the hand functions of persons with DMD. These devices were based on a quantitative evaluation of the hand functions of persons with severe DMD, which consists of fingertip force and range of motion of the fingers. Two new joysticks were developed, considering hand function, posture, and hand shape of two persons with severe DMD. Thus, the devices can be operated using slight force and movement of the fingers. Furthermore, the authors examined the shape of the hands of persons with severe DMD to determine the shape of the devices.

Comparing the Performance, Workload, and Usability of a Gamepad and Joystick in a Complex Task

Complex human system interface systems such as unmanned vehicles utilize controls that range from standard joystick and keyboard interfaces to Xbox controllers. However, few research studies have been conducted to compare Xbox controllers with other types of interfaces, showing mixed results between those controllers. The current study compared the performance of a joystick and keyboard interface with that of an Xbox controller in both a low and high difficulty task. The results indicate that the Xbox controller had lower tracking errors and trended to lower workload and a slightly higher usability score as measured by the system usability scale (SUS). An interaction between difficulty and controller was not found, however allowing for a longer practice time with the interfaces may have shown significant differences which are planned in a future study. Overall these results indicate favorable results that an Xbox controller may be a viable control interface for complex human system interaction tasks.

Enhancing Wheelchair Manoeuvrability for Severe Impairment Users

A significant number of individuals with severe motor impairments are unable to control their wheelchair using a standard joystick. Even when they can facilitate the control input, navigation in a confined space or crowded environments is still a great challenge. Here we propose a wheelchair framework that enables the user to issue the command via a multi-input hands free interface (HFI), which subsequently assists him/her to overcome difficult circumstances using a multimodal control strategy. Initially the HFI inputs are analysed to infer the desired control mode and the user command. Then environmental information is perceived using a combination of laser and Kinect sensors for determining all possible obstacle locations and outputting a safety map around the wheelchair's vicinity. Eventually, the user's command is validated with the safety map to moderate the final motion, which is collision free and the best for the user's preference. The proposed method can reduce the burden of severe impairment users when controlling wheelchairs by continuously monitoring the surroundings and can make them move easily according to the users' intention.

An initial evaluation of the Iowa Dental Surgical Simulator

The University of Iowa colleges of dentistry and engineering are collaborating to build an effective but low-cost surgical simulator that uses force feedback to teach and assess the tactile skills of dentistry. A prototype simulator was built, and a formative evaluation examined the realism of the haptics and identified directions for future work. Using a cross-over design, twelve experienced practitioners probed two virtual teeth using two instruments attached to the force feedback device. The session was videotaped, the forces recorded, and a standardized questionnaire completed. Two analyses were conducted: an ANOVA examined practitioners' questionnaire responses, and a t-test analyzed the probing forces. Significant tooth order by instrument order interaction and instrument effects were found. Practitioners were generally satisfied. They preferred a standard joystick to an explorer, felt that two-dimensional graphics were sufficient, and emphasized that the existing vibration be eliminated. Random placement of caries should help teach generalized skills.

The autonomous mobile robot SENARIO: a sensor aided intelligent navigation system for powered wheelchairs

The SENARIO project is develoing a sensor-aided intelligent navigation system that provides high-level navigational aid to users of powered wheelchairs. The authors discuss new and improved technologies developed within SENARIO concerning task/path planning, sensing and positioning for indoor mobile robots as well as user interface issues. The autonomous mobile robot SENARIO, supports semi- or fully autonomous navigation. In semi-autonomous mode the system accepts typical motion commands through a voice-activated or standard joystick interface and supports robot motion with obstacle/collision avoidance features. Fully autonomous mode is a superset of semi-autonomous mode with the additional ability to execute autonomously high-level go-to-goal commands. At its current stage, the project has succeeded in fully supporting semi-autonomous navigation, while experiments on the fully autonomous mode are very encouraging.