Midas Touch Problem Research Articles

EyeGesener: Eye Gesture Listener for Smart Glasses Interaction Using Acoustic Sensing

The smart glasses market has witnessed significant growth in recent years. The interaction of commercial smart glasses mostly relies on the hand, which is unsuitable for scenarios where both hands are occupied. In this paper, we propose EyeGesener, an eye gesture listener for smart glasses interaction using acoustic sensing. To mitigate the Midas touch problem, we meticulously design eye gestures for interaction as two intentional consecutive saccades in a specific direction without visual dwell. The proposed system is a glass-mounted acoustic sensing system with two pairs of commercial speakers and microphones to sense eye gestures. To capture the subtle movements of the eyelid and surrounding skin induced by eye gestures, we design an Orthogonal Frequency Division Multiplexing (OFDM)-based channel impulse response (CIR) estimation schema that allows two speakers to transmit at the same time and in the same frequency band without collision. We implement eye gesture filtering and adversarial-based eye gesture recognition to identify eye gestures for interaction, filtering out daily eye movements. To address the differences in eye size and facial structure among different users, we employ adversarial training to achieve user-independent eye gesture recognition. We evaluate the performance of our system through experiments on data collected from 16 subjects. The experimental result shows that our system can recognize eight eye gestures with an average F1-score of 0.93, and the false alarm rate of our system is 0.03. We develop an interactive real-time audio-video player based on EyeGesener and then conduct a user study. The result demonstrates the high usability of the proposed system.

Magilock: a reliable control triggering method in multi-channel eye-control systems.

Eye-tracking technology brings a different human-computer interaction experience to users because of its intuitive, natural, and hands-free operation characteristics. Avoiding the Midas touch problem and improving the accuracy of interaction are among the main goals of the research and development of eye-control systems. This study reviews the methods and limitations of research on avoiding the Midas touch problem. For typical control clicking operations with low fault tolerance, such as mode switching and state selection in an eye-control system, this study proposes Magilock, a more reliable control triggering method with a high success rate in multi-channel eye-control systems. Magilock adds a control pre-locked mechanism between the two interactive steps of eye-control channel positioning control and other interactive channel triggering controls in the multi-channel eye-control system. This effectively avoids incorrect control triggering caused by multi-channel coordination disorder and gaze-point drift. This study also conducted ergonomic experiments to explore the lock and unlock times of the control pre-locked mechanism in Magilock. Taking into account the experimental data and subjective evaluation of the participants, we recommend setting the lock time and the unlock time of Magilock to 200 ms.

Intelligent Eye-Controlled Electric Wheelchair Based on Estimating Visual Intentions Using One-Dimensional Convolutional Neural Network and Long Short-Term Memory.

When an electric wheelchair is operated using gaze motion, eye movements such as checking the environment and observing objects are also incorrectly recognized as input operations. This phenomenon is called the "Midas touch problem", and classifying visual intentions is extremely important. In this paper, we develop a deep learning model that estimates the user's visual intention in real time and an electric wheelchair control system that combines intention estimation and the gaze dwell time method. The proposed model consists of a 1DCNN-LSTM that estimates visual intention from feature vectors of 10 variables, such as eye movement, head movement, and distance to the fixation point. The evaluation experiments classifying four types of visual intentions show that the proposed model has the highest accuracy compared to other models. In addition, the results of the driving experiments of the electric wheelchair implementing the proposed model show that the user's efforts to operate the wheelchair are reduced and that the operability of the wheelchair is improved compared to the traditional method. From these results, we concluded that visual intentions could be more accurately estimated by learning time series patterns from eye and head movement data.

Pactolo Bar: An Approach to Mitigate the Midas Touch Problem in Non-Conventional Interaction.

New ways of interacting with computers is driving research, which is motivated mainly by the different types of user profiles. Referred to as non-conventional interactions, these are found with the use of hands, voice, head, mouth, and feet, etc. and these interactions occur in scenarios where the use of mouse and keyboard would be difficult. A constant challenge in the adoption of new forms of interaction, based on the movement of pointers and the selection of interface components, is the Midas Touch (MT) problem, defined as the involuntary action of selection by the user when interacting with the computer system, causing unwanted actions and harming the user experience during the usage process. Thus, this article aims to mitigate the TM problem in interaction with web pages using a solution centered on the Head Tracking (HT) technique. For this purpose, a component in the form of a Bar was developed and inserted on the left side of the web page, called the Pactolo Bar (PB), in order to enable or disable the clicking event during the interaction process. As a way of analyzing the effectiveness of PB in relation to TM, two stages of tests were carried out based on the collaboration of voluntary participants. The first step aims to find the data that would lead to the best configuration of the BP, while the second step aims to carry out a comparative analysis between the PB solution and the eViacam software, whose use is also focused on the HT technique. The results obtained from the use of PB were considered promising, since the analysis of quantitative data points to a significant prevention of involuntary clicks in the iteration interface and the analysis of qualitative data showed the development of a better user experience due to the ease of use, which can be noticed in elements such as the PB size, the triggering mechanism, and its positioning in the graphical interface. This study benefits in the context of the user experience, because, when using non-conventional interactions, basic items such as aspects of the graphic elements, and interaction events raise new studies that seek to mitigate the problem of the Midas Touch.

Toward Gaze-Based Map Interactions: Determining the Dwell Time and Buffer Size for the Gaze-Based Selection of Map Features

The modes of interaction (e.g., mouse and touch) between maps and users affect the effectiveness and efficiency of transmitting cartographic information. Recent advances in eye tracking technology have made eye trackers lighter, cheaper and more accurate, broadening the potential to interact with maps via gaze. In this study, we focused exclusively on using gaze to choose map features (i.e., points, polylines and polygons) via the select operation, a fundamental action preceding other operations in map interactions. We adopted an approach based on the dwell time and buffer size to address the low spatial accuracy and Midas touch problem in gaze-based interactions and to determine the most suitable dwell time and buffer size for the gaze-based selection of map features. We conducted an experiment in which 38 participants completed a series of map feature selection tasks via gaze. We compared the participants’ performance (efficiency and accuracy) between different combinations of dwell times (200 ms, 600 ms and 1000 ms) and buffer sizes (point: 1°, 1.5°, and 2°; polyline: 0.5°, 0.7° and 1°). The results confirmed that a larger buffer size raised efficiency but reduced accuracy, whereas a longer dwell time lowered efficiency but enhanced accuracy. Specifically, we found that a 600 ms dwell time was more efficient in selecting map features than 200 ms and 1000 ms but was less accurate than 1000 ms. However, 600 ms was considered to be more appropriate than 1000 ms because a longer dwell time has a higher risk of causing visual fatigue. Therefore, 600 ms supports a better balance between accuracy and efficiency. Additionally, we found that buffer sizes of 1.5° and 0.7° were more efficient and more accurate than other sizes for selecting points and polylines, respectively. Our results provide important empirical evidence for choosing the most appropriate dwell times and buffer sizes for gaze-based map interactions.

Building EEG-based CAD object selection intention discrimination model using convolutional neural network (CNN)

Currently, building natural interaction systems based on physiological signals has become an crucial requirement for the development of Computer Aided Design (CAD). As the first step of model operation in CAD, object selection is essential and the efficiency of selecting has a great impact on the experience of users. In the research community, gaze-based interaction for object selection has been well-established. However, this interactive mode is still imperfect due to Midas touch problem. In this work, a selection intention discrimination (SID) model is implemented to decode electroencephalogram (EEG) signals generated during object selection process. Common Spatial Pattern (CSP) is applied to extract spatial features from EEG in four frequency bands. Then these features are learned by a Convolutional Neural Network (CNN) equipped with an adaptive weights training module to realize the SID. To verify the decoding feasibility of this model, a cognitive experiment related to object selection is conducted. The empirical result shows that the performance of this model is good. It turns out that EEG-based object selection is feasible, which can be a intuitive and natural interaction mode for CAD.

Development of an Eye-Gaze Input System With High Speed and Accuracy through Target Prediction Based on Homing Eye Movements

In this study, a method to predict a target on the basis of the trajectory of eye movements and to increase the pointing speed while maintaining high predictive accuracy is proposed. First, a predictive method based on ballistic (fast) eye movements (Approach 1) was evaluated in terms of pointing speed and predictive accuracy. In Approach 1, the so-called Midas touch problem (pointing to an unintended target) occurred, particularly when a small number of samples was used to predict a target. Therefore, to overcome the poor predictive accuracy of Approach 1, we developed a new predictive method (Approach 2) using homing (slow) eye movements rather than ballistic (fast) eye movements. Approach 2 overcame the disadvantage (inaccurate prediction) of Approach 1 by shortening the pointing time while maintaining high predictive accuracy.

A New Approach of Minimizing Midas Touch Problem for a Tracer-Free Tongue-Controlled Assistive Technology

Tongue-controlled assistive technologies (ATs) have a great potential to improve the quality of life for people with severe disabilities. Current tongue-controlled ATs, however, require either a tracer to be attached on the tongue or a complicated gesture/mechanism to avoid issuing commands unintendedly during routine intraoral events (e.g., tongue resting, swallowing, and speaking), which poses a high entry barrier to potential users. In this study, we propose a tracer-free tongue-controlled AT using four optical distance sensors and one ambient light sensor with a multi-sensor data fusion algorithm to distinguish the intended tongue contacts from the unintended ones. Eleven healthy human subjects participated in the study to evaluate the performance using the proposed technology in masking unintendedly-issued commands and navigating a mouse cursor on a computer. The proposed multi-sensor data fusion algorithm could mask 100%, 86.99%, and 99.50% of unintendedly-issued commands during tongue resting, swallowing, and speaking, respectively. The efficacy of the tongue commands in mouse navigation was evaluated by command-issuing task and maze-navigation task. The participants could issue tongue commands in a random order with 97.16±0.94% of accuracy. The completion time and the sum of deviation of the maze-navigation task were 31.36±2.74 s and 9.38±0.85 pixel <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , respectively. The subjects' performance on computer-access tasks, using the proposed tracer-free tongue-controlled technology, is at the level of the performance measured with existing tracer-based tongue-controlled technologies, while securing robustness in masking commands, unintendedly issued during routine intraoral events. Results of this preliminary study suggest that the tracer-free implementation is feasible option for the tongue-controlled ATs.

Optimization of Tongue Gesture Processing Algorithm for Standalone Multimodal Tongue Drive System

Multimodal tongue drive system (mTDS) is an assistive technology, which uses speech, tongue, and head movements of people with tetraplegia to help them control devices such as wheelchairs, PCs, or smartphones. In order to process tongue gestures, it is required to solve the magnetic dipole equation, which is computationally expensive to execute in a wearable embedded hardware. In this paper, a support vector machine with linear kernel-based algorithm is proposed, which uses an additional magnetometer that has 96.1% accuracy with a 2.2% sensitivity and 1.8% specificity among 15 participants, is computationally efficient for mTDS, generates high-performance classification results despite small changes of magnet positions (−5 to 5 mm) and orientations (−10° to 10°) from a four-dimensional robot emulated tongue movement data compared to five other algorithms. This algorithm generates more accurate results to control devices independently despite the Midas-touch problem between speech and tongue movements.

A Gaze-Based Virtual Keyboard Using a Mouth Switch for Command Selection.

Portable eye-trackers provide an efficient way to access the point of gaze from a user on a computer screen. Thanks to eyetracking, gaze-based virtual keyboard can be developed by taking into account constraints related to the gaze detection accuracy. In this paper, we propose a new gaze-based virtual keyboard where all the letters can be accessed directly through a single command. In addition, we propose a USB mouth switch that is directly connected through a computer mouse, with the mouse switch replacing the left click button. This approach is considered to tackle the Midas touch problem with eye-tracking for people who are severely disabled. The performance is evaluated on 10 participants by comparing the following three conditions: gaze detection with mouth switch, gaze detection with dwell time by considering the distance to the closest command, and the gaze detection within the surface of the command box. Finally, a workload using NASA-TLX test was conducted on the different conditions. The results revealed that the proposed approach with the mouth switch provides a better performance in terms of typing speed (36.6 ± 8.4 letters/minute) compared to the other conditions, and a high acceptance as an input device.

A Robust Noninvasive Eye Control Approach For Disabled People Based on Kinect 2.0 Sensor

A noninvasive eye control approach based on Kinect 2.0 was proposed to improve life quality for people with upper limb disabilities. First, Kinect 2.0 was used to track the critical points of the canthus and eyelid. Second, the greyscale differential algorithm was used to position the iris. Finally, a novel eye control model based on the canthus coordinates, central coordinates of pupils, and eyelid distance was established for cursor control. To reduce the calibration difficulty and the interference from eye jitter on the control, incremental indirect control was performed to ensure smooth and gradual cursor motion to the fixation point. To solve the Midas touch problem, an eye switch was proposed to reduce excessive unconscious cursor movement. Benefitting from the Kinect 2.0 advantages of high resolution, high efficiency, and good stability, the control speed reached 19.9fps in MATLAB, which meets the requirement for real-time control. The method also offers good adaptability to various noise sources, such as rotation, occlusion, and scaling. The experimental results suggest the availability and robustness of the new approach.

A multimodal interface to resolve the Midas-Touch problem in gaze controlled wheelchair.

Human-computer interaction (HCI) research has been playing an essential role in the field of rehabilitation. The usability of the gaze controlled powered wheelchair is limited due to Midas-Touch problem. In this work, we propose a multimodal graphical user interface (GUI) to control a powered wheelchair that aims to help upper-limb mobility impaired people in daily living activities. The GUI was designed to include a portable and low-cost eye-tracker and a soft-switch wherein the wheelchair can be controlled in three different ways: 1) with a touchpad 2) with an eye-tracker only, and 3) eye-tracker with soft-switch. The interface includes nine different commands (eight directions and stop) and integrated within a powered wheelchair system. We evaluated the performance of the multimodal interface in terms of lap-completion time, the number of commands, and the information transfer rate (ITR) with eight healthy participants. The analysis of the results showed that the eye-tracker with soft-switch provides superior performance with an ITR of 37.77 bits/min among the three different conditions (p<;0.05). Thus, the proposed system provides an effective and economical solution to the Midas-Touch problem and extended usability for the large population of disabled users.

Enhancing an Eye-Tracker based Human-Computer Interface with Multi-modal Accessibility Applied for Text Entry

In natural course, human beings usually make use of multisensory modalities for effective communication or efficiently executing day-to-day tasks. For instance, during verbal conversations we make use of voice, eyes, and various body gestures. Also effective human-computer interaction involves hands, eyes, and voice, if available. Therefore by combining multi-sensory modalities, we can make the whole process more natural and ensure enhanced performance even for the disabled users. Towards this end, we have developed a multimodal human-computer interface (HCI) by combining an eyetracker with a soft-switch which may be considered as typically representing another modality. This multi-modal HCI is applied for text entry using a virtual keyboard appropriately designed in-house, facilitating enhanced performance. Our experimental results demonstrate that using multi-modalities for text entry through the virtual keyboard is more efficient and less strenuous than single modality system and also solves the Midas-touch problem, which is inherent in an eye-tracker based HCI system where only dwell time is used for selecting a character.

Dynamic and Meta-Context Switching for Gaze-Based Interaction

In this paper we investigate the performance of a gaze-based interaction system that combines Dynamic Context Switching and Meta-Context Switching. These methods are extensions of the Context Switching interaction paradigm. The original context switching idea uses fixed-size contexts. Each context carries the same information, so the user can browse freely within a context without worrying about the Midas touch problem. A saccade to the other context triggers the selection of the item under focus. Dynamic context switching dynamically adjusts the size of a context to improve its useful area, where the context that has the user focus is displayed in full size and the other is minimized. Meta-context switching uses meta-keys to allow the user to escape from the current task and select other contexts or change the operation mode. We have designed and conducted two user experiments to evaluate these new gaze interaction techniques and compare them with selection by dwell time in a search task. The task required browsing through several pages using meta-keys. The experimental results show that dynamic context switching improves user performance when compared to fixed-size context switching and do not cause disorientation. The error rate was significantly higher for dwell time due to the Midas touch problem, although the time spent to complete the task was similar for dwell time and dynamic context switching.

A visual attention-based method to address the midas touch problem existing in gesture-based interaction

The "Midas Touch" problem has long been a difficult problem existing in gesture-based interaction. This paper proposes a visual attention-based method to address this problem from the perspective of cognitive psychology. There are three main contributions in this paper: (1) a visual attention-based parallel perception model is constructed by combining top-down and bottom-up attention, (2) a framework is proposed for dynamic gesture spotting and recognition simultaneously, and (3) a gesture toolkit is created to facilitate gesture design and development. Experimental results show that the proposed method has a good performance for both isolated and continuous gesture recognition tasks. Finally, we highlight the implications of this work for the design and development of all gesture-based applications.

New Solution to the Midas Touch Problem: Identification of Visual Commands Via Extraction of Focal Fixations

Reliable identification of intentional visual commands is a major problem in the development of eye-movements based user interfaces. This work suggests that the presence of focal visual fixations is indicative of visual commands. Two experiments are described which assessed the effectiveness of this approach in a simple gaze-control interface. Identification accuracy was shown to match that of the commonly used dwell time method. Using focal fixations led to less visual fatigue and higher speed of work. Perspectives of using focal fixations for identification of visual commands in various kinds of eye-movements based interfaces are discussed.

Use of saccadic latency for visual inspection system

In some industrial inspection processes, multiprocess-handling workers are required to enter the results of visual inspection tests without using their hands because of efficiency or hygiene reasons. We have developed a hands-free visual inspection system by using saccadic latency, a temporal characteristic of saccadic eye movements. The proposed system is free from the Midas touch problem (i.e., the difficulty in developing an eye-typing interface owing to the difficulty in differentiating between intentional blinks and gazes and natural ones). For verifying the system, an experiment was conducted in which ten subjects performed a visual inspection task. The average defect detection rates were 99.4%, and no Midas touch–related errors were observed. Results of the error analysis showed that redesigning the system interface would lead to a further enhancement of the system performance. © 2011 Wiley Periodicals, Inc. © 2012 Wiley Periodicals, Inc.

Using Unconstrained Tongue Motion as an Alternative Control Mechanism for Wheeled Mobility

Tongue drive system (TDS) is a tongue-operated, minimally invasive, unobtrusive, noncontact, and wireless assistive technology that infers users' intentions by detecting and classifying their voluntary tongue motions, and translating them to user-defined commands. We have developed customized interface circuitry between an external TDS (eTDS) prototype and a commercial powered wheelchair (PWC) as well as three control strategies to evaluate the tongue motion as an alternative control input for wheeled mobility. We tested the eTDS performance in driving PWCs on 12 able-bodied human subjects, of which 11 were novice. The results showed that all subjects could complete navigation tasks by operating the PWC using their tongue motions. Despite little prior experience, the average time using the eTDS and the tongue was only approximately three times longer than using a joystick and the fingers. Navigation time was strongly dependant on the number of issued commands, which reduced by gaining experience. Particularly, the unintended issued commands (the Midas touch problem) were rare, demonstrating the effectiveness of the tongue tracking and external magnetic field cancellation algorithms as well as the safety of the TDS for wheeled mobility.

Influences of dwell time and cursor control on the performance in gaze driven typing

In gaze controlled computer interfaces the dwell time is often used as selection criterion. But this solution comes along with several problems, especially in the temporal domain: Eye movement studies on scene perception could demonstrate that fixations of different durations serve different purposes and should therefore be differentiated. The use of dwell time for selection implies the need to distinguish intentional selections from merely per-ceptual processes, described as the Midas touch problem. Moreover, the feedback of the actual own eye position has not yet been addressed to systematic studies in the context of usability in gaze based computer interaction. We present research on the usability of a simple eye typing set up. Different dwell time and eye position feedback configurations were tested. Our results indicate that smoothing raw eye position and temporal delays in visual feedback enhance the system's functionality and usability. Best overall performance was obtained with a dwell time of 500 ms.