Virtual Joystick Research Articles

Design of Control Elements in Virtual Reality—Investigation of Factors Influencing Operating Efficiency, User Experience, Presence, and Workload

Virtual reality (VR) enables prototypes of devices to be evaluated in a simulation of a future usage environment. A disadvantage is the insufficient feedback design during the interaction with prototypes. In this study, we investigated how virtual control elements in VR must be designed to compensate for the lack of feedback. Therefore, 35 participants performed VR tasks using a virtual joystick and virtual rotary control. According to the design of experiments method, 12 factors, such as haptic feedback, sensitivity, size, and shape, were systematically varied. The control accuracy, task time, user experience, presence, and mental workload were recorded. The effect of a factor on the recorded parameters was examined using multifactorial ANOVA. Linear regression was used to calculate the mathematical models between the factors and parameters. These models were used to calculate the optimal design of the control elements in the VR. For rotary control, eight factors had a significant influence on the recorded parameters. There were seven factors for the joystick. With mathematical models, optimized control element designs for VR were calculated for the first time. These findings can help to better adapt prototypes and human–machine interfaces to different modalities in VR.

A web application for remote control of ROS robot based on WebSocket protocol and Django development environment

The development of teleoperation systems, robots, or any physical part of the system can be costly and if something goes wrong it can lead to development overdue. Precisely for these reasons, engineers and scientists today resort to the development of simulated systems before the construction of a real system. Robot Operating System (ROS) is one of the most popular solutions for robot development, manipulation, and simulation. In this paper, we present a web application for remote control of a ROS robot. The robot is controlled via a web application that is used as a virtual Joystick. Also, in this paper, an experimental work analysis of the projected system is performed. Further research possibilities include upgrading the presented web interface, adding certain motion autonomy sensors, or integrating some path planning algorithms.

Development of a Virtual Reality Simulation for Remote Controlled Container Crane Operator Training

Any large scale cargo port requires the use of container cranes for loading of intermodal containers as well as unloading them. The operator of such a large crane must be appropriately trained to prevent any accidents at the cargo port. This should be achieved by training crane operators on all the protocols of crane operation with significant hours of practice. The aim of this project is to develop a virtual reality (VR) simulation for operator training of container cranes which are remotely controlled, situated at the cargo ports. The simulation consists of a virtual control room equipped with joysticks and display screens, container crane, container ships, trucks and a cargo port environment. We used an actual super post-Panamax quay crane as a reference model for this simulation. The project has been built using Unity3d for the programming and 3DSMAX for the designing of models. The virtual joysticks in the simulation can be interacted by using the Oculus Rift VR setup thus allowing to maneuver the crane and the container load in all three dimensions. This paper mainly discusses the structure and implementation of the virtual reality simulation for crane operator training, along with a proposal for further improvements.

How Subtle Can It Get?

Flying drones have become common objects in our daily lives, serving a multitude of purposes. Many of these purposes involve outdoor scenarios where the user combines drone control with another activity. Traditional interaction methods rely on physical or virtual joysticks that occupy both hands, thus restricting drone usability. In this paper, we investigate one-handed human-to-drone-interaction by leveraging three modalities: force, touch, and IMU. After prototyping three different combinations of these modalities on a smartphone, we evaluate them against the current commercial standard through two user experiments. These experiments help us to find the combination of modalities that strikes a compromise between user performance, perceived task load, wrist rotation, and interaction area size. Accordingly, we select a method that achieves faster task completion times than the two-handed commercial baseline by 16.54% with the merits of subtle user behaviours inside a small-size ring-form device and implements this method within the ring-form device. The last experiment involving 12 participants shows that thanks to its small size and weight, the ring device displays better performance than the same method implemented on a mobile phone. Furthermore, users unanimously found the device useful for controlling a drone in mobile scenarios (AVG = 3.92/5), easy to use (AVG = 3.58/5) and easy to learn (AVG = 3.58/5). Our findings give significant design clues in search of subtle and effective interaction through finger augmentation devices with drone control. The users with our prototypical system and a multi-modal on-finger device can control a drone with subtle wrist rotation (pitch gestures: 43.24° amplitude and roll gestures: 46.35° amplitude) and unnoticeable thumb presses within a miniature-sized area of (1.08 * 0.61 cm2).

Tablet interface for direct vision teleoperation of an excavator for urban construction work

Abstract In recent years, worker shortages in the field of urban construction are looming as an important issue. To address this difficulty, direct-vision remote control of a construction machine can be an effective solution by which a worker can play not only the role of an operator but also play the role of a manual laborer.This study examines a new direct-vision remote control system for an excavator using a tablet device. To realize an intuitive interface, we propose the following graphical user interfaces (GUI) of two kinds: input bucket velocity using virtual joysticks, and input bucket position by dragging a bucket. We conducted a usability evaluation experiment simulating a horizontal digging operation. Operability of the GUIs were compared with that of a GUI resembling the conventional International Organization for Standardization (ISO) teleoperation interface. The experimental result concludes that proposed two GUIs superior to the conventional interface in task execution time, accuracy, and subjective usability evaluation.

Design of a solar tracking interactive kiosk

A two-axis solar tracker and its interactive kiosk were designed by an interdisciplinary team of students and faculty. The objective was to develop a publicly accessible kiosk that would facilitate the study of energy usage and production on campus. Tracking is accomplished by an open-loop algorithm, microcontroller, and ham radio rotator. Solar panel output is monitored in real time and displayed to the public with lights and digits that can be read by the casual passersby. While maximum power point tracking is the most accurate means of quantifying the output power of a photovoltaic panel, simplicity and design constraints dictated the use of short-circuit current as a proxy for power. A touchscreen display allows kiosk visitors to compare two solar panels, an automatic tracker that faces the sun, and an identical panel whose elevation and azimuth can be controlled with a virtual joystick. This project was a capstone experience for students in physics/engineering, computer science, and instructional technology. We discuss technical challenges and design choices, as well as the educational goals of the kiosk.

Near-Field Touch Interface Using Time-of-Flight Camera

[abstFig src='/00280005/20.jpg' width='300' text='Near-field touch interface system, NFTIS' ] The purpose of this study is to realize a near-field touch interface that is compact, flexible, and highly accurate. We applied a 3-dimensional image sensor (time-of-flight camera) to achieve the basic functions of conventional touch interfaces, such as clicking, dragging, and sliding, and we designed a complete projector-sensor system. Unlike conventional touch interfaces, such as those on tablet PCs, the system can sense the 3-dimensional positions of fingertips and 3-dimensional directions of fingers. Moreover, it does not require a real touch screen but instead utilizes a mobile projector for display. Nonetheless, the system is compact, with a working distance of as short as around 30 cm. Our methods solve the shadow and reflection problems of the time-of-flight camera and can provide robust detection results. Tests have shown that our approach has a high success rate (98.4%) on touch/hover detection and a small standard error (2.21 mm) on position detection on average for different participants, which is the best performance we have achieved. Some applications, such as the virtual keyboard and virtual joystick, are also realized based on the proposed projector-sensor system.

Skills based evaluation of alternative input methods to command a semi-autonomous electric wheelchair.

This paper presents the evaluation, under standardized metrics, of alternative input methods to steer and maneuver a semi-autonomous electric wheelchair. The Human-Machine Interface (HMI), which includes a virtual joystick, head movements and speech recognition controls, was designed to facilitate mobility skills for severely disabled people. Thirteen tasks, which are common to all the wheelchair users, were attempted five times by controlling it with the virtual joystick and the hands-free interfaces in different areas for disabled and non-disabled people. Even though the prototype has an intelligent navigation control, based on fuzzy logic and ultrasonic sensors, the evaluation was done without assistance. The scored values showed that both controls, the head movements and the virtual joystick have similar capabilities, 92.3% and 100%, respectively. However, the 54.6% capacity score obtained for the speech control interface indicates the needs of the navigation assistance to accomplish some of the goals. Furthermore, the evaluation time indicates those skills which require more user's training with the interface and specifications to improve the total performance of the wheelchair.

Remote Control User Interfaces for Service Mobile Robots for Elderly Care

Abstract: The following article discusses iPad and windows based user interfaces used to manage mobile robot for elderly care. Describe are specific areas like robot control, objects manipulation, task execution and real time robot status handling. Robot control discusses map handling, map navigation and manual robot handling and driving. Demonstrated are concepts like custom controls and virtual joysticks to manipulate robot. Object manipulation describes possible object related tasks and the way to execute them as well as getting feedback after task successful execution or failure. Task execution describes tasks, actions and action sequences. Also some safety and usability issues are evaluated.

A Virtual Joy-Stick Study of Emotional Responses and Social Motivation in Children with Autism Spectrum Disorder.

A new virtual reality task was employed which uses preference for interpersonal distance to social stimuli to examine social motivation and emotion perception in children with Autism Spectrum Disorders. Nineteen high function children with higher functioning Autism Spectrum Disorder (HFASD) and 23 age, gender, and IQ matched children with typical development (TD) used a joy stick to position themselves closer or further from virtual avatars while attempting to identify six emotions expressed by the avatars, happiness, fear, anger, disgust, sadness, and surprise that were expressed at different levels of intensity. The results indicated that children with HFASD displayed significantly less approach behavior to the positive happy expression than did children with TD, who displayed increases in approach behavior to higher intensities of happy expressions. Alternatively, all groups tended to withdraw from negative emotions to the same extent and there were no diagnostic group differences in accuracy of recognition of any of the six emotions. This pattern of results is consistent with theory that suggests that some children with HFASD display atypical social-approach motivation, or sensitivity to the positive reward value of positive social-emotional events. Conversely, there was little evidence that a tendency to withdraw from social-emotional stimuli, or a failure to process social emotional stimuli, was a component of social behavior task performance in this sample of children with HFASD.

Spatial Orientation Aware Smartphones for Tele-operated Robot Control in Military Environments

We report an experiment conducted to assess the usability of a smartphone-based controller for use by dismounted troops using unmanned ground vehicles for reconnaissance and surveillance. A virtual joystick controller is compared to a specially designed tilt-based controller intended to limit the device’s touch interface while providing intuitive control. Participants drove a small tracked robot on an indoor course using both controller options where metrics regarding performance, mental workload, and user satisfaction represented various measures of controller usability. Results indicate that the tilt controller is preferred by users and performs equally as well, if not better, on most performance metrics. These results support the development of a smartphone-based control option for military robotics, with a focus on gestural input methods which overcome deficiencies of current touch-based systems, namely lack of physical feedback, high attention demands, and unreliability in field environments.

Muscle Contraction: The Subtle Way of Human Computer Interaction

The advent of new technology has opened opportunities to the computer users to consider alternate ways for humans to interface with machines. Research groups are looking to use computer interfaces that can allow users to communicate with devices in a subtle way and allow virtual reality environments to create an immersing experience. Electromyography based wearable input device can interface with mobile devices in a subtle way. Electromyography can also be used for interactive computer gaming as well as to control flights through virtual joysticks and controllers. This study provides a brief review on various ways for humans electromyography (EMG) signal to interface with machines. Therefore, the crux of this study is to overview the up to date developments and research related to the EMG interface with the wearable devices. This study further opens up a passage for researchers and end users to advocate an excellent understanding of EMG interfacing with mechanical devices.

Shoulder-Surfing Resistant Smartphone Authentication Scheme Using Virtual Joystick

User authentication techniques such as the setting of passwords are gradually gaining importance as a means of managing important information stored in smartphones. Existing text–based password authentication schemes have the advantages of being quick and easy to use. However, they are problematic in that passwords are easily exposed to shoulder-surfing attack. In addition, a graphical password authentication scheme has the limitation of being difficult to apply to mobile device environments, in which a lot of information must be remembered and small-sized screens are provided. Therefore, in this paper, we propose a new hybrid password authentication scheme using a pocket billiard and a virtual joystick, which is secure against shoulder-surfing, brute force attack, and smudge attack and has excellent usability.

Gesture-based control and EMG decomposition

This paper presents two probabilistic developments for the use with electromyograms (EMGs). First described is a neuroelectric interface for virtual device control based on gesture recognition. The second development is a Bayesian method for decomposing EMGs into individual motor unit action potentials (MUAPs). This Bayesian decomposition method allows for distinguishing individual muscle groups with the goal of enhancing gesture recognition. All examples presented rely upon sampling EMG data from a subject's forearm. The gesture-based recognition uses pattern recognition software that has been trained to identify gestures from among a given set of gestures. The pattern recognition software consists of hidden Markov models, which are used to recognize the gestures as they are being performed in real time from moving averages of EMGs. Two experiments were conducted to examine the feasibility of this interface technology. The first replicated a virtual joystick interface, and the second replicated a keyboard. Moving averages of EMGs do not provide an easy distinction between fine muscle groups. To better distinguish between different fine motor skill muscle groups, we present a Bayesian algorithm to separate surface EMGs into representative MUAPs. The algorithm is based on differential variable component analysis, which was originally developed for electroencephalograms. The algorithm uses a simple forward model representing a mixture of MUAPs as seen across multiple channels. The parameters of this model are iteratively optimized for each component. Results are presented on both synthetic and experimental EMG data. The synthetic case has additive white noise and is compared with known components. The experimental EMG data were obtained using a custom linear electrode array designed for this study

Gestures as input: neuroelectric joysticks and keyboards

EMG (electromyogram) technology helps capture gestures as input for virtual joysticks and keyboards and thus could lead to new applications in flight control, space, and the video game industry.