Touchless Gestures Research Articles

Compatibility effects with touchless gestures

Human actions are suspect to various compatibility phenomena. For example, responding is faster to the side where a stimulus appears than to the opposite side, referred to as stimulus–response (S–R) compatibility. This is even true, if the response is given to a different stimulus feature, while location itself is irrelevant (Simon compatibility). In addition, responses typically produce perceivable effects on the environment. If they do so in a predictable way, responses are faster if they produce a (e.g., spatially) compatible effect on the same side than on the other side. That it, a left response is produced faster if it results predictably in a left effect than in a right effect. This effect is called response-effect (R–E) compatibility. Finally, compatibility could also exist between stimuli and the effects, which is accordingly called stimulus-effect (S–E) compatibility. Such compatibility phenomena are also relevant for applied purposes, be it in laparoscopic surgery or aviation. The present study investigates Simon and R–E compatibility for touchless gesture interactions. In line with a recent study, no effect of R–E compatibility was observed, yet irrelevant stimulus location yielded a large Simon effect. Touchless gestures thus seem to behave differently with regard to compatibility phenomena than interactions via (other) tools such as levers.

Mid-Air Gesture Control of Multiple Home Devices in Spatial Augmented Reality Prototype

Touchless, mid-air gesture-based interactions with remote devices have been investigated as alternative or complementary to interactions based on remote controls and smartphones. Related studies focus on user elicitation of a gesture vocabulary for one or a few home devices and explore recommendations of respective gesture vocabularies without validating them by empirical testing with interactive prototypes. We have developed an interactive prototype based on spatial Augmented Reality (AR) of seven home devices. Each device responds to touchless gestures (identified from a previous elicitation study) via the MS Kinect sensor. Nineteen users participated in a two-phase test (with and without help provided by a virtual assistant) according to a scenario that required from each user to apply 41 gestural commands (19 unique). We report on main usability indicators: task success, task time, errors (false negative/positives), memorability, perceived usability, and user experience. The main conclusion is that mid-air interaction with multiple home devices is feasible, fairly easy to learn and apply, and enjoyable. The contributions of this paper are (a) validation of a previously elicited gesture set; (b) development of a spatial AR prototype for testing of mid-air gestures, and (c) extensive assessment of gestures and evidence in favor of mid-air interaction in smart environments.

Stimulus-Response and Response-Effect Compatibility With Touchless Gestures and Moving Action Effects

To determine whether response-effect (R-E) compatibility or stimulus-response (S-R) compatibility is more critical for touchless gesture responses. Content on displays can be moved in the same direction (S-R incompatible but R-E compatible) or opposite direction (S-R compatible but R-E incompatible) as the touchless gesture that produces the movement. Previous studies suggested that it is easier to produce a button-press response when it is R-E compatible (and S-R incompatible). However, whether this R-E compatibility effect also occurs for touchless gesture responses is unknown. Experiments 1 and 2 employed an R-E compatibility manipulation in which participants made responses with an upward or downward touchless gesture that resulted in the display content moving in the same (compatible) or opposite (incompatible) direction. Experiment 3 employed an S-R compatibility manipulation in which the stimulus occurred at the upper or lower location on the screen. Overall, only negligible influences of R-E compatibility on performing the touchless gestures were observed (in contrast to button-press responses), whereas S-R compatibility heavily affected the gestural responses. The R-E compatibility obtained in many previous studies with various types of responses appears not to hold for touchless gestures as responses. The results suggest that in the design of touchless interfaces, unique factors may contribute to determining which mappings of gesture and display movements are preferred by users.

User experience comparison among touchless, haptic and voice Head-Up Displays interfaces in automobiles

This paper evaluates driving experiences when using a Head-Up Display (HUD) system by different interaction methods, since HUDs in automotive market do not follow standard interaction methods (Betancur et al. in Int J Interact Des Manuf 12(1):199–214, [3]), and therefore it is difficult to identify what types of these methods are the most appropriated in terms of usability for being implemented in HUDs. This paper focuses on comparing the mental workload that a driver could experience while interacting with a specific HUD visual interface by touchless gestures, haptic or voice methods. Then, test subjects (n = 15) performed conjointly a driving activity and a set of in-vehicle tasks by using these interaction methods throughout a HUD. The experiments show that the haptic method was better accepted than the touchless gestures and voice methods. Nevertheless, the touchless gestures method was explored under some specific usability configurations, in order to demonstrate that it was not significantly different from the haptic method.

Drivers’ Interaction Preferences in Autonomous Vehicle Multimodal Interactive Systems

Within the scope of a broader project (which aimed to develop and implement a method for designing driver environments of autonomous vehicles), we investigated driver-vehicle interaction preferences within a multi-interactive, multi-display truck. Fifteen professional drivers explored different combinations of input (Buttons, Touch, Touchless gestures) and display (Instrument Cluster [ICL], Tablet, and Head Up Display [HUD]) and responded on the appropriateness, acceptance, and usability of the pairing. Results indicated that (1) Touchless input was not suitable for any display; (2) Buttons are best paired with the ICL and HUD; and (3) Touch was best paired with the Tablet. Future work should evaluate if the interactions with the system are easy to perform (i.e. interaction optimization); and driver understanding of system actions.

Motor-Intuitive Interactions Based on Image Schemas: Aligning Touchless Interaction Primitives with Human Sensorimotor Abilities

Elicitation and evaluation studies investigated intuitiveness of touchless gestures but did not operationalizeintuitiveness.Forexample,studiesfoundthatusersfailtomakeaccurate3Dstrokesas interactioncommands.Butthisphenomenonremainsunexplained.Inthispaper,wefirstexplainhow making accurate 3D strokes is generally unintuitive, because it exceeds our sensorimotor knowledge. Wethenintroducemotor-intuitive,touchlessinteractionthatusessensorimotorknowledgebyrelying on image schemas. Specifically, we propose an interaction primitive—mid-air, directional strokes— based on space schemas up‐down and left‐right. In a controlled study with large displays, we found that biomechanical factors affected directional strokes. Strokes were efficient (0.2s) and effective (12.5 ◦ angular error), but affected by directions and length. Our work operationalized intuitive touchless interaction using the continuum of knowledge in intuitive interaction, and demonstrated how user performance of a motor-intuitive, touchless primitive based on sensorimotor knowledge (image schemas) is affected by biomechanical factors.

Interacting with Touchless Gestures: Taxonomy and Requirements

Objective: The aim of this study is to make the taxonomy for classifying diverse touchless gestures and establish the design requirements that should be considered in determining suitable gestures during gesture-based interaction design. Background: Recently, the applicability of touchless gestures is more and more increasing as relevant technologies are being advanced. However, before touchless gestures are widely applied to various devices or systems, the understanding on human gestures' natures and their standardization should be prerequisite. Method: In this study, diverse gesture types in various literatures were collected and, based on those, a new taxonomy for classifying touchless gestures was proposed. And many gesture-based interaction design cases and studies were analyzed. Results: The proposed taxonomy consisted of two dimensions: shape (deictic, manipulative, semantic, or descriptive) and motion(static or dynamic). The case analysis based on the taxonomy showed that manipulative and dynamic gestures were widely applied. Conclusion: Four core requirements for valuable touchless gestures were intuitiveness, learnability, convenience and discriminability. Application: The gesture taxonomy can be applied to produce alternatives of applicable touchless gestures, and four design requirements can be used as the criteria for evaluating the alternatives.