Haptic User Research Articles

A provably secure and practical end-to-end authentication scheme for tactile Industrial Internet of Things

In the Industrial Internet of Things (IIoT), haptic control of machines or robots can be managed remotely. However, with the emergence of Tactile Industrial Internet of Things (TIIoT), the transmission of haptic data over public channels has raised security and privacy concerns. In such an environment, mutual authentication between haptic users and remotely controlled entities is crucial to prevent illegal control by adversaries. Therefore, we propose an end-to-end authentication scheme, SecTIIoT, to establish secure communication between haptic users and remote IoT devices. The scheme addresses security issues by using lightweight hash cryptographic primitives and employs a useful piggyback strategy to improve authentication efficiency. We demonstrate that SecTIIoT is resilient to various known attacks with formal security proofs and informal security analysis. Furthermore, our detailed performance analysis shows that SecTIIoT outperforms existing lightweight authentication schemes as it provides more security features while reducing computation and communication costs.

Exodex Adam-A Reconfigurable Dexterous Haptic User Interface for the Whole Hand.

Applications for dexterous robot teleoperation and immersive virtual reality are growing. Haptic user input devices need to allow the user to intuitively command and seamlessly “feel” the environment they work in, whether virtual or a remote site through an avatar. We introduce the DLR Exodex Adam, a reconfigurable, dexterous, whole-hand haptic input device. The device comprises multiple modular, three degrees of freedom (3-DOF) robotic fingers, whose placement on the device can be adjusted to optimize manipulability for different user hand sizes. Additionally, the device is mounted on a 7-DOF robot arm to increase the user’s workspace. Exodex Adam uses a front-facing interface, with robotic fingers coupled to two of the user’s fingertips, the thumb, and two points on the palm. Including the palm, as opposed to only the fingertips as is common in existing devices, enables accurate tracking of the whole hand without additional sensors such as a data glove or motion capture. By providing “whole-hand” interaction with omnidirectional force-feedback at the attachment points, we enable the user to experience the environment with the complete hand instead of only the fingertips, thus realizing deeper immersion. Interaction using Exodex Adam can range from palpation of objects and surfaces to manipulation using both power and precision grasps, all while receiving haptic feedback. This article details the concept and design of the Exodex Adam, as well as use cases where it is deployed with different command modalities. These include mixed-media interaction in a virtual environment, gesture-based telemanipulation, and robotic hand–arm teleoperation using adaptive model-mediated teleoperation. Finally, we share the insights gained during our development process and use case deployments.

Haptic User Interface of a Cable-Driven Input Device to Control the End Effector of a Surgical Telemanipulation System

Abstract In robotic telemanipulation for minimally-invasive surgery, lack of haptic sensation and non-congruent movement of input device and manipulator are major drawbacks. Input devices based on cable-driven parallel mechanisms have the potential to be a stiff alternative to input devices based on rigid parallel or serial kinematics by offering low inertia and a scalable workspace. In this paper, the haptic user interface of a cable-driven input device and its technical specifications are presented and assessed. The haptic user interface allows to intuitively control the gripping movement of the manipulator’s end effector by providing a two-finger precision grasp. By design, the interface allows to command input angles between 0° and 45°. Furthermore, interaction forces from the manipulator’s end effector can be displayed to the user’s twofinger grasp in a range from 0 N to 6 N with a frequency bandwidth of 17 Hz.

Haptic Responses to Angry and Happy Faces

ABSTRACT The present aim was to investigate if angry, neutral, and happy facial expressions have different effects on haptic responses. Two experiments were conducted. In Experiment 1, participants (N = 24) were to respond as fast as they could to images of angry, neutral, and happy faces using a haptic device that measured the amount of applied force in newton units and the duration of touch in milliseconds. Experiment 2 was identical to Experiment 1, except that ratings of emotion-related experiences evoked by the stimuli were also collected in terms of valence and arousal. The results from both studies showed that the force of touch in response to angry expressions was significantly stronger than in response to happy expressions. The ratings of valence showed that angry, neutral, and happy faces were experienced statistically significantly different from each other along the dimension of valence. Ratings of arousal showed that angry faces were experienced as significantly more arousing than neutral and happy faces. The results suggest a relationship between the processing of emotion-related facial expressions and haptic response systems. The force of touch could be used as one of the measures conveying information about the user’s state of mind when interacting with haptic user interfaces.

Mid-Air Haptic Rendering of 2D Geometric Shapes With a Dynamic Tactile Pointer.

An important challenge that affects ultrasonic mid-air haptics, in contrast to physical touch, is that we lose certain exploratory procedures such as contour following. This makes the task of perceiving geometric properties and shape identification more difficult. Meanwhile, the growing interest in mid-air haptics and their application to various new areas requires an improved understanding of how we perceive specific haptic stimuli, such as icons and control dials in mid-air. We address this challenge by investigating static and dynamic methods of displaying 2D geometric shapes in mid-air. We display a circle, a square, and a triangle, in either a static or dynamic condition, using ultrasonic mid-air haptics. In the static condition, the shapes are presented as a full outline in mid-air, while in the dynamic condition, a tactile pointer is moved around the perimeter of the shapes. We measure participants' accuracy and confidence of identifying shapes in two controlled experiments ( n1 = 34, n2 = 25). Results reveal that in the dynamic condition people recognise shapes significantly more accurately, and with higher confidence. We also find that representing polygons as a set of individually drawn haptic strokes, with a short pause at the corners, drastically enhances shape recognition accuracy. Our research supports the design of mid-air haptic user interfaces in application scenarios such as in-car interactions or assistive technology in education.

Correction to “Evaluation of Presence in Virtual Environments: Haptic Vest and User’s Haptic Skills”

In the above article [1] , the affiliation of all authors should read “Centre for Automation and Robotics (UPM-CSIC), Universidad Politecnica de Madrid, 28006 Madrid, Spain.”

Getting Your Hands Dirty Outside the Lab: A Practical Primer for Conducting Wearable Vibrotactile Haptics Research.

As haptics have become an ingrained part of our wearable experience, particularly through phones, smartwatches, and fitness trackers, significant research effort has been conducted to find new ways of using wearable haptics to convey information, especially while we are on-the-go. In this paper, instead of focusing on aspects of haptic information design, such as tacton encoding methods, actuators, and technical fabrication of devices, we address the more general recurring issues and "gotchas" that arise when moving from core haptic perceptual studies and in-lab wearable experiments to real world testing of wearable vibrotactile haptic systems. We summarize key issues for practitioners to take into account when designing and carrying out in-the-wild wearable haptic user studies, as well as for user studies in a lab environment that seek to simulate real-world conditions. We include not only examples from published work and commercial sources, but also hard-won illustrative examples derived from issues and failures from our own haptic studies. By providing a broad-based, accessible overview of recurring issues, we expect that both novice and experienced haptic researchers will find suggestions that will improve their own mobile wearable haptic studies.

Application of a Redundant Haptic Interface in Enhancing Soft-Tissue Stiffness Discrimination

Haptic-enabled teleoperated surgical systems have the potential to enhance the accuracy and performance of surgical interventions. The user interface of such a system can provide haptic feedback to the surgeon to more intuitively perform surgical tasks. In this letter, we study the added benefits of redundant manipulators as haptic interfaces for teleoperated surgical systems. First, we introduce the intrinsic benefits of employing a redundant haptic interface, namely, reduced apparent inertia and increased manipulability (one result of which is reduced friction forces). Next, we demonstrate that the haptic interface redundancy can further reduce its apparent inertia and friction via appropriately manipulating the extra degrees of freedom of the interface. This will consequently enhance the haptic feedback resolution (sensitivity) for the user. Finally, a psychophysical experiment is performed to validate the improved force perception for the user in a virtual soft-tissue palpation task. We conduct a set of perceptual experiments to evaluate how a redundant and non-redundant user interface affects the perception of the virtual stiffness. Experimental results demonstrate that the redundancy in the haptic user interface helps to enhance tissue stiffness discrimination ability of the user by reducing the distortions caused by the kinematics and dynamics of the user interface.

Improving User Performance in Haptics-Based Rehabilitation Exercises by Colocation of User's Visual and Motor Axes via a Three-Dimensional Augmented-Reality Display

Serious games are recently becoming a common sight in rehabilitation settings to provide motivation for patients undergoing therapy to regain upper limb function after disability. These are often presented using a two-dimensional (2-D) monitor to the patient who uses a robotic device (haptic user interface) as the game controller. In this letter, we develop a 3-D spatial augmented reality (AR) display to colocate visual and haptic feedback to the user in three rehabilitative games. The same games are also displayed in a 2-D nonimmersive virtual reality (VR) and are compared against their AR counterpart in terms of user task performance to evaluate the benefit of the 3-D AR system. To simulate a rehabilitation scenario, able-bodied participants are put under cognitive load (CL) for simulating disability-induced cognitive deficiencies when performing the tasks. A within-subjects analysis of ten participants was carried out for the rehabilitative games. The results show that AR leads to the best user performance with or without cognitive loading. This result is most evident in dynamic exercises where the participants are required to have quick reaction times and fast movement. Furthermore, even while AR had a significant difference over VR, one of the tasks showed that the performance in AR between non-CL and CL cases was similar, thereby showing how AR can alleviate the negative effects of CL.

Understanding the Effect of Handedness on Both-Handed Task Performance: An Experimental Study based on a Haptic-Controlled, Simulation-Based Surgical Skill Training Scenario

ABSTRACTUnderstanding the performance on both-handed tasks, such as endoscopic surgery, is critical to better organize and develop appropriate instructional systems to improve the necessary skills of surgeons. However, in the literature, only a limited number of studies have investigated the effect of handedness on both-handed task performance. This study aimed to provide an understanding of the participants’ performance differences while performing both-handed tasks through a haptic user interface in a simulated virtual environment specifically developed for surgical training purposes. Twenty-four surgeons attending a medical school in Turkey voluntarily participated in the study. The duration, accuracy, and collision measures were automatically recorded by software. The results revealed that the left-handed participants performed the both-handed tasks (camera: nondominant hand, tool: dominant-hand) in a significantly shorter time than the right-handed participants. This study also showed that haptic-controlled simulation-based surgical skill training systems can potentially provide measures for better understanding the individual behaviors and deliver alternative training environments specific to individual requirements.

Virtual haptic system for intuitive planning of bone fixation plate placement

Placement of pre-contoured fixation plate is a common treatment for bone fracture. Fitting of fixation plates on fractured bone can be preoperatively planned and evaluated in 3D virtual environment using virtual reality technology. However, conventional systems usually employ 2D mouse and virtual trackball as the user interface, which makes the process inconvenient and inefficient. In the paper, a preoperative planning system equipped with 3D haptic user interface is proposed to allow users to manipulate the virtual fixation plate intuitively to determine the optimal position for placement on distal medial tibia. The system provides interactive feedback forces and visual guidance based on the geometric requirements. Creation of 3D models from medical imaging data, collision detection, dynamics simulation and haptic rendering are discussed. The system was evaluated by 22 subjects. Results show that the time to achieve optimal placement using the proposed system was shorter than that by using 2D mouse and virtual trackball, and the satisfaction rating was also higher. The system shows potential to facilitate the process of fitting fixation plates on fractured bones as well as interactive fixation plate design.

A novel tri-axial capacitive-type skin sensor

This paper introduces a novel tri-axial capacitive force sensor. The sensor can measure the force vector, is embedded in soft 7 mm-thick silicone skin, enables temperature sensitivity compensation and has digital output. To measure the force vector, tilted capacitive sensor elements are used which are facing in different directions to differentiate the tangential forces. The sensor is intended for distributed contact sensing in a robotic skin, but could be also used for other applications such as novel haptic user interfaces in wearable devices. A series of experiments was performed and showed good sensor characteristics. The concept of the tilted force transducers has been proven to have the capability of detecting the force vector acting on the local sensor surface.

Haptic User Interface for a Telerobotic System – Design based on a Network Model

Loss of haptic feedback is one major drawback of telerobotic surgery systems. For this reason tasks of passive input devices can be taken over by haptic user interfaces. The main objective of this paper is the development of a haptic user interface with a kinematic structure matching the slave robot's kinematic structure. In addition, we focus on the detailed design and characterization of a haptic input device concerning the grasping degree of freedom. Requirements of the user interface are derived by analyzing the medical application and the structure of the slave robot. Since the user interface and the input device are supposed to provide realistic haptic feedback, static and dynamic behavior of the device is of interest. Electromechanical analogies help to design the dynamic behavior of the haptic input device. A characterization of static and dynamic behavior validates our design process. The implementation of the input device in the overall user interface is shown.The designed user interface, allows controlling the slave robot in three dimensions, rotation along the axis of the main kinematic chain and grasping. The user interfaces kinematic structure is taking the kinematic constrains of the slave kinematic into account. The input device can be handled similar as a forceps with three point precision grip, similar to a common surgical procedure without robotic assistance. The user interface provides haptic feedback in all directions of the kinematic structure.

Scenario-Based Observation Approach for Eliciting User Requirements for Haptic User Interfaces

As tactual information processing of visually impaired users has not been investigated sufficiently and there are few guidelines on the development of haptic user interfaces, development of haptic assistive system can pose many challenges. Despite the breadth and variety of tools available for elicitation of user requirements, no single tool adequately provides the needed information to develop such a specialized system for a unique user population where the understanding of user behaviors is limited. This article explores the state-of-art of requirements engineering, discusses the challenges in developing a haptic assistive system, and proposes a methodology of combining a controlled observation in a naturalistic setting with scenario-based design for effective and efficient user requirements elicitation. A case study of developing a haptically enhanced, collaborative learning-by-feeling science education system for visually impaired students was conducted to show validity and effectiveness of the proposed methodology. The case study showed that the methodology has a variety of benefits including the reduced uncertainty and a better fit to the natural behaviors of the user population.

Accessible haptic user interface design approach for users with visual impairments

With the number of people with visual impairments (e.g., low vision and blind) continuing to increase, vision loss has become one of the most challenging disabilities. Today, haptic technology, usi...

The moderating role of sensation seeking tendency in robotic haptic interfaces

This study examined the interaction effects between haptic force feedback and users’ sensation seeking tendency (i.e. need for sensations) on users’ feelings of presence (i.e. the state in which users experience virtual objects and virtual environments as if they were actual) in robotic haptic interfaces. Users with low sensation seeking tendency felt stronger physical presence and spatial presence in response to force feedback haptic stimuli (versus no force feedback), whereas users with high sensation seeking tendency did not show any difference between the two conditions, thus confirming the moderating role of the users’ sensation seeking tendency in the robotic haptic interface. Theoretical implications for human–computer interaction (HCI) research and managerial implications for the interactive media market are discussed.

Elicitation of Haptic User Interface Needs of People with Low Vision

Various assistive technologies such as haptic technology are used to help people with visual impairments comprehend complex information. Yet there is likely to be a misconception that users with the same disability category share the same user interface needs; furthermore, the majority of the literature has been oriented toward total blindness rather than low vision, possibly leading to dissatisfaction with assistive technologies and discontinuation of its use by those with low vision. The aim of this article is to advance the understanding of the needs of those with low vision especially in relation to haptic-incorporated multimodal user interfaces. A scenario-based, participatory design approach was used to explore their needs. A total of 19 user needs were systematically documented under three categories: audition (n = 5), touch (n = 11), and vision (n = 3). This article focuses on qualitatively exploring their needs and theoretically interpreting the needs in the light of previous studies.

Haptic perception of users with low vision and their needs in haptic-incorporated user interfaces

Purpose: This paper reports on research aimed at advancing understanding haptic capability and needs of users with low vision. The objective is to apply this understanding to the design of haptic-incorporated user interfaces. Method: Study 1 investigated the haptic perception between sighted participants and those with low vision through the magnitude estimation technique, and Study 2 explored the degree to which similar user interface needs were observed among the two vision groups. Results: Overall, our findings indicate there was no significant difference between the two vision groups in terms of haptic perception and user interface needs. A few differences in user interface preference did exist, however, and designers should take these into account. Conclusions: Participants with low vision were a group who relied on their vision in everyday life instead of touch. Thus, their haptic capability was less likely to be enhanced via brain plasticity, which probably contributed to no significant difference in haptic-incorporated user interface needs.Implications for RehabilitationNo significant different haptic capability and haptic user interface (UI) needs exists between cited participants and those with low vision.UI designers should take into consideration that a certain range of magnitude/type of haptic feedback is available to accommodate preferences of both vision groups, which would ultimately increase the likelihood of successfully developing universal designs.

Haptic User Interfaces for the Visually Impaired: Implications for Haptically Enhanced Science Learning Systems

The overall quality of haptic user interfaces designed to support visually impaired students' science learning through sensorial feedback was systematically studied to investigate task performance and user behavior. Fourteen 6th- to 11th-grade students with visual impairments recruited from a state-funded blind school were asked to perform three main tasks (i.e., menu selection, structure exploration, and force recognition) using haptic user interfaces and a haptic device. This study used several dependent measures that are categorized into three types of variables: (a) task performance including success rate, workload, and task completion time; (b) user behavior defined as cursor movements proportionately represented from the user's cursor positional data; and (c) user preference. Results showed that interface type has significant effects on task performance, user behavior, and user preference, with varying degree of impact to participants with severe visual impairments performing the tasks. The results of this study as well as a set of refined design guidelines and principles should provide insights to the future research of haptic user interfaces that can be used when developing haptically enhanced science learning systems for the visually impaired.

触覚•聴覚フィードバックを通じて,視覚障害者の感覚を拡張するコンピュータ操作支援システム

触覚•聴覚フィードバックを通じて,視覚障害者の感覚を拡張するコンピュータ操作支援システム