Haptic Guidance Research Articles

Midair Haptic Pursuit.

In human vision, smooth pursuit eye movement is the basic ability to visually follow a moving object by keeping it at the sight center. In this study, we validate that a human hand has a similar ability to track a midair haptic stimulus, i.e., a human palm exposed to a point vibration by a noncontact ultrasound tactile display can follow the continuous movement of the stimulation point. The experimental results show that the trackable velocity limit is 10 cm/s for motion parallel to the palm, when the initial velocity is zero. This ability of motion tracking by hand can be applied to haptic guidance for visually impaired people or for evacuation navigation, where no devices are needed to be equipped by users.

Relationship Between Gaze Behavior and Steering Performance for Driver–Automation Shared Control: A Driving Simulator Study

The driver steering performance is strongly related to gaze behavior in natural driving. The investigation of gaze behavior is crucial for understanding driver response to driver–automation shared control, which was developed to reduce the workload of drivers while keeping them in the control loop. This study focused on driver gaze behavior and its relationship to the steering performance when driving with different levels of shared control. Shared control was achieved through a haptic guidance steering system, which provided active assistant torques on the steering wheel. A driving simulator experiment was conducted with 15 participants, who drove five trials, each with a different level of shared control: first, no haptic guidance, second, normal haptic guidance, third, strong haptic guidance, fourth, having a driver's hands hovering over the steering wheel as a standby during automated driving, and, fifth, having a driver's hands free for a long time during automated driving. The results show a tendency that the correlation between gaze and steering movements decreased when the automation authority of shared control increased. A reduction in the lead time of gaze over steering movement was found in the condition of strong haptic guidance compared to normal haptic guidance and manual driving. This study suggests the potential of using the relationship between gaze behavior and the steering performance to analyze driver interaction with automation.

Haptic Error Modulation Outperforms Visual Error Amplification When Learning a Modified Gait Pattern.

Robotic algorithms that augment movement errors have been proposed as promising training strategies to enhance motor learning and neurorehabilitation. However, most research effort has focused on rehabilitation of upper limbs, probably because large movement errors are especially dangerous during gait training, as they might result in stumbling and falling. Furthermore, systematic large movement errors might limit the participants’ motivation during training. In this study, we investigated the effect of training with novel error modulating strategies, which guarantee a safe training environment, on motivation and learning of a modified asymmetric gait pattern. Thirty healthy young participants walked in the exoskeletal robotic system Lokomat while performing a foot target-tracking task, which required an increased hip and knee flexion in the dominant leg. Learning the asymmetric gait pattern with three different strategies was evaluated: (i) No disturbance: no robot disturbance/guidance was applied, (ii) haptic error amplification: unsafe and discouraging large errors were limited with haptic guidance, while haptic error amplification enhanced awareness of small errors relevant for learning, and (iii) visual error amplification: visually observed errors were amplified in a virtual reality environment. We also evaluated whether increasing the movement variability during training by adding randomly varying haptic disturbances on top of the other training strategies further enhances learning. We analyzed participants’ motor performance and self-reported intrinsic motivation before, during and after training. We found that training with the novel haptic error amplification strategy did not hamper motor adaptation and enhanced transfer of the practiced asymmetric gait pattern to free walking. Training with visual error amplification, on the other hand, increased errors during training and hampered motor learning. Participants who trained with visual error amplification also reported a reduced perceived competence. Adding haptic disturbance increased the movement variability during training, but did not have a significant effect on motor adaptation, probably because training with haptic disturbance on top of visual and haptic error amplification decreased the participants’ feelings of competence. The proposed novel haptic error modulating controller that amplifies small task-relevant errors while limiting large errors outperformed visual error augmentation and might provide a promising framework to improve robotic gait training outcomes in neurological patients.

Evaluation of Different Modes of Haptic Guidance for Robotic Surgery

Abstract The evolution of surgery has resulted in a plethora of systems varying in their application field, size, as well as degree of autonomy. Especially systems which combine principles of so-called synergistic robotic systems and master-slave-telemanipulators are interesting concerning cooperative aspects of surgeon and robotic system. While the former provide haptic guidance information (“virtual fixtures”), the latter provide haptic feedback from position or force sensor information of the slave device. During the design of these combined systems particular attention has to be paid during the allocation of information to the haptic information channel as superimposing of haptic guidance information and haptic sensor feedback can lead to concealment of essential feedback information. This paper reports on an experimental usability evaluation of different haptic guidance modes varying in their degree of autonomy as well as degree of freedom (DOF) with respect to three surgical scenarios, namely reaching a predefined position and orientation, tracking a predefined 3D trajectory, and applying a defined force (such as during e.g. a 3D-bone milling task). The goal was to evaluate whether some DOF of haptic guidance information can be left free for force sensor information feedback. General findings indicate that haptic guidance does not have to be augmented on three DOF to improve usability. Therefore, a combination of haptic sensor information feedback and haptic guidance information divided between individual DOF seems to be a potential solution.

SU139GENOME-WIDE ASSOCIATION ANALYSIS OF LIFETIME CANNABIS USE (N=184,765) IDENTIFIES NEW RISK LOCI, GENETIC OVERLAP WITH MENTAL HEALTH, AND A CAUSAL INFLUENCE OF SCHIZOPHRENIA ON CANNABIS USE

The motor system’s natural tendency is to move the limbs over equal amplitudes, for example in walking. However, in many situations in which people must perform complex movements, a certain degree of amplitude differentiation of the limbs is required. Visual and haptic feedback have recently been shown to facilitate such independence of limb movements. However, it is unknown whether motor expertise moderates the extent to which individuals are able to differentiate the amplitudes of their limb-movements while being supported with visual and haptic feedback. To answer this question 14 pre-professional dancers were compared to 14 non-dancers on simultaneously generating a small displacement with one foot, and a larger one with the other foot, in four different feedback conditions. In two conditions, haptic guidance was offered, either in a passive or active mode. In the other two conditions, veridical and enhanced visual feedback were provided. Surprisingly, no group differences were found regarding the degree to which the visual or haptic feedback assisted the generation of the different target amplitudes of the feet (mean amplitude difference between the feet). The correlation between the displacements of the feet and the standard deviation of the continuous relative phase between the feet, reflecting the degree of independence of the feet movements, also failed to show between-group differences. Sample entropy measures, indicating the predictability of the foot movements, did show a group difference. In the haptically-assisted conditions, the dancers demonstrated more predictable coordination patterns than the non-dancers as reflected by lower sample entropy values whereas the reverse was true in the visual-feedback conditions. The results demonstrate that motor expertise does not moderate the extent to which haptic tracking facilitates the differentiation of the amplitudes of the lower limb movements in an asymmetrical bipedal coordination task.

Multimodal Sensorimotor Integration for Expert-in-the-Loop Telerobotic Surgical Training

This paper presents a novel multimodal training platform integrated with hand-over-hand (HOH) haptic guidance for dual-console surgical robotic systems such as the da Vinci Si system. The expert-in-the-loop (EIL) framework incorporates a fuzzy interface system in order to provide a trainee with adaptive authority over the procedure as well as hand-over-hand haptic guidance adjusted in real time based on the proficiency level of the trainee. The EIL expertise-oriented framework enables performance of a surgical procedure by an expert surgeon on a patient, while simultaneously providing a trainee at any stage of the motor-skills development with multimodal training without jeopardizing patient safety. Closed-loop stability of the system is investigated using the circle criterion and it is shown that the proposed architecture is unconditionally stable. Experimental evaluations are presented in support of the proposed platform through the implementation of a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA, USA) and the dV-Trainer master console (Mimic Technology, Inc., Seattle, WA, USA). To the best of our knowledge, the implemented setup is the first research platform for dual-console studies involving the classic da Vinci surgical system.

Touching and traveling on 3D augmented tangible maps for learning geography

PurposeTangible physical maps which are enhanced by new digital forms of interaction can become an invaluable asset for learning geography in an embodied way. The purpose of this work is to evaluate an interactive augmented three-dimensional (3D) tangible map on which students interact and travel with their fingers.Design/methodology/approachA total of 58 fourth-grade students from eight elementary schools participated in the study. The participants played with the FingerTrips environment in 24 sessions and in groups of two or three. Each session lasted for about 20-25 min. After completing the interactive game, the students answered a questionnaire concerning their attitudes toward the tangible environment and participated in a short interview.FindingsStudents’ responses revealed that FingerTrips managed to transform the experience of meeting new places, understanding spatial relations and learning geography. Students supported that such an approach is closer to their interactive experiences and expectations, and exploits embodied learning affordances to achieve enjoyable learning. Students identified their finger-based trips as an effective and intriguing static haptic guidance that helped them learn more effectively.Originality/valueThe specific approach has two distinctive characteristics. First, a new interaction style on the map is suggested, the use of trips with fingers. Students have to follow predefined engraved paths on the 3D terrain to sense distances and changes in altitude and “touch” the topology asked to understand and explore. Second, it is examined whether a low fidelity interactive 3D terrain, which can be easily reconstructed and reprogrammed by primary school students, can become a useful canvas for learning geography.

Evaluation of Haptic Feedback on Bimanually Teleoperated Laparoscopy for Endometriosis Surgery.

Robotic minimal invasive surgery is gaining acceptance in surgical care. In contrast with the appreciated three-dimensional vision and enhanced dexterity, haptic feedback is not offered. For this reason, robotics is not considered beneficial for delicate interventions such as the endometriosis. Overall, haptic feedback remains debatable and yet unproven except for some simple scenarios such as fundamentals of laparoscopic surgery exercises. Objective: This work investigates the benefits of haptic feedback on more complex surgical gestures, manipulating delicate tissue through coordination between multiple instruments. Methods: A new training exercise, “endometriosis surgery exercise” (ESE) has been devised approximating the setting for monocular robotic endometriosis treatment. A bimanual bilateral teleoperation setup was designed for laparoscopic laser surgery. Haptic guidance and haptic feedback are, respectively, offered to the operator. User experiments have been conducted to assess the validity of ESE and examine possible advantages of haptic technology during execution of bimanual surgery. Results: Content and face validity of ESE were established by participating surgeons. Surgeons suggested ESE also as a mean to train lasering skills, and interaction forces on endometriotic tissue were found to be significantly lower when a bilateral controller is used. Collisions between instruments and the environment were less frequent and so were situations marked as potentially dangerous. Conclusion: This study provides some promising results suggesting that haptics may offer a distinct advantage in complex robotic interventions were fragile tissue is manipulated. Significance: Patients need to know whether it should be incorporated. Improved understanding of the value of haptics is important as current commercial surgical robots are widely used but do not offer haptics.

Helping older pedestrians navigate in the city: comparisons of visual, auditory and haptic guidance instructions in a virtual environment

ABSTRACTPreserving older pedestrians’ navigation skills in urban environments is a challenge for maintaining their quality of life. However, existing aids do not take into account older people’s perceptual and cognitive declines nor their user experience, and they call upon sensory modalities that are already used during walking. The present study was aimed at comparing different guidance instructions using visual, auditory, and haptic feedback in order to identify the most efficient and best accepted one(s). Sixteen middle-age (non-retired) adults, 21 younger-old (young-retired) adults, and 21 older-old (old-retired) adults performed a navigation task in a virtual environment. The task was performed with visual feedback (directional arrows superimposed on the visual scenes), auditory feedback (sounds in the left/right ear), haptic feedback (vibrotactile information delivered by a wristband), combinations of different types of sensory feedback, or a paper map. The results showed that older people benefited from the sensory guidance instructions, as compared to the paper map. Visual and auditory feedbacks were associated with better performance and user experience than haptic feedback or the paper map, and the benefits were the greatest among the older-old participants, even though the paper-map familiarity was appreciated. Several recommendations for designing pedestrian navigation aids are proposed.

Haptic Guidance Based on All-Optical Ultrasound Distance Sensing for Safer Minimally Invasive Fetal Surgery.

By intervening during the early stage of gestation, fetal surgeons aim to correct or minimize the effects of congenital disorders. As compared to postnatal treatment of these disorders, such early interventions can often actually save the life of the fetus and also improve the quality of life of the newborn. However, fetal surgery is considered one of the most challenging disciplines within Minimally Invasive Surgery (MIS), owing to factors such as the fragility of the anatomic features, poor visibility, limited manoeuvrability, and extreme requirements in terms of instrument handling with precise positioning. This work is centered on a fetal laser surgery procedure treating placental disorders. It proposes the use of haptic guidance to enhance the overall safety of this procedure and to simplify instrument handling. A method is described that provides effective guidance by installing a forbidden region virtual fixture over the placenta, thereby safeguarding adequate clearance between the instrument tip and the placenta. With a novel application of all-optical ultrasound distance sensing in which transmission and reception are performed with fibre optics, this method can be used with a sole reliance on intraoperatively acquired data. The added value of the guidance approach, in terms of safety and performance, is demonstrated in a series of experiments with a robotic platform.

Collaborative Haptic Exploration of Dynamic Remote Environments

Haptic perception is an important modality in reinforcing the presence of virtual or remote targets, but providing haptic feedback of dynamic environments still remains a challenging task. We address the issue of haptic telepresence systems by improving and integrating two independent approaches: real-time haptic rendering of unstructured spatial data and collaborative interaction with a remote partner. Contact with physical constraints is directly estimated from streaming point-cloud data without surface reconstruction, and haptic guidance cues that restrict or promote the users motion can be provided by both predefined triggers and gesture-based input from a helper. Through a user study with a proof-of-concept prototype, the authors show that the proposed approach significantly improves the performance of remote exploration tasks while enabling stable haptic interaction with real-world spatial data.

Neural circuits activated by error amplification and haptic guidance training techniques during performance of a timing-based motor task by healthy individuals

To promote motor learning, robotic devices have been used to improve subjects’ performance by guiding desired movements (haptic guidance—HG) or by artificially increasing movement errors to foster a more rapid learning (error amplification—EA). To better understand the neurophysiological basis of motor learning, a few studies have evaluated brain regions activated during EA/HG, but none has compared both approaches. The goal of this study was to investigate using fMRI which brain networks were activated during a single training session of HG/EA in healthy adults learning to play a computerized pinball-like timing task. Subjects had to trigger a robotic device by flexing their wrist at the correct timing to activate a virtual flipper and hit a falling ball towards randomly positioned targets. During training with HG/EA, subjects’ timing errors were decreased/increased, respectively, by the robotic device to delay or accelerate their wrist movement. The results showed that at the beginning of the training period with HG/EA, an error-detection network, including cerebellum and angular gyrus, was activated, consistent with subjects recognizing discrepancies between their intended actions and the actual movement timing. At the end of the training period, an error-detection network was still present for EA, while a memory consolidation/automatization network (caudate head and parahippocampal gyrus) was activated for HG. The results indicate that training movement with various kinds of robotic input relies on different brain networks. Better understanding the neurophysiological underpinnings of brain processes during HG/EA could prove useful for optimizing rehabilitative movement training for people with different patterns of brain damage.

Helping older pedestrians navigate unknown environments through vibrotactile guidance instructions

Many studies have consistently found evidence of aging-related navigation difficulties that lead to disorientation behaviors and/or reduced travel. Despite technological advances, older pedestrians mainly use paper maps to find their way in unknown environments while they have a lot of difficulties in using and interacting with them efficiently. In this context, the present study was aimed at assessing the effectiveness and acceptability and study older pedestrian’s behavior when wearing a vibrotactile instructions to help them finding their way in an unknown city as compared with visual instructions provided by a paper map. Fifty-eight participants (20 young adults ages 21–45, 20 younger-old adults ages 61–70, and 18 older-old adults ages 71–80) took part in a simulated navigation task where participants had to go from point A to point B in a virtual city. The task was performed either with a vibrotactile wristband delivering directional messages or with a standard paper map showing the visual instructions to follow. The data showed that vibrotactile guidance instructions improved the correct numbers of turns taken (left/right) at simple intersections. They also reduced travel time as compared with visual guidance instructions provided by a paper map. These benefits were greater among the older-old participants, who had trouble using the paper map and thus benefited greatly from the assistance provided by the vibrotactile guidance instructions. These findings suggest that such an assistance device using simple haptic guidance messages is able to improve older pedestrians’ mobility.

Haptic-Based Shared-Control Methods for a Dual-Arm System

We propose novel haptic guidance methods for a dual-arm telerobotic manipulation system, which are able to deal with several different constraints, such as collisions, joint limits, and singularities. We combine the haptic guidance with shared-control algorithms for autonomous orientation control and collision avoidance meant to further simplify the execution of grasping tasks. The stability of the overall system in various control modalities is presented and analyzed via passivity arguments. In addition, a human subject study is carried out to assess the effectiveness and applicability of the proposed control approaches both in simulated and real scenarios. Results show that the proposed haptic-enabled shared-control methods significantly improve the performance of grasping tasks with respect to the use of classic teleoperation with neither haptic guidance nor shared control.

Interpersonal interactions for haptic guidance during balance exercises

BackgroundCaregiver–patient interaction relies on interpersonal coordination during support provided by a therapist to a patient with impaired control of body balance. Research questionThe purpose of this study was to investigate in a therapeutic context active and passive participant involvement during interpersonal support in balancing tasks of increasing sensorimotor difficulty. MethodsTen older adults stood in semi-tandem stance and received support from a physical therapist (PT) in two support conditions: 1) physical support provided by the PT to the participant’s back via an instrumented handle affixed to a harness worn by the participant (“passive” interpersonal touch; IPT) or 2) support by PT and participant jointly holding a handle instrumented with a force-torque transducer while facing each other (“active” IPT). The postural stability of both support conditions was measured using the root-mean-square (RMS) of the Centre-of-Pressure velocity (RMS dCOP) in the antero-posterior (AP) and medio-lateral (ML) directions. Interpersonal postural coordination (IPC) was characterized in terms of cross-correlations between both individuals’ sway fluctuations as well as the measured interaction forces. ResultsActive involvement of the participant decreased the participant’s postural variability to a greater extent, especially under challenging stance conditions, than receiving support passively. In the passive support condition, however, stronger in-phase IPC between both partners was observed in the antero-posterior direction, possibly caused by a more critical (visual or tactile) observation of participants’ body sway dynamics by the therapist. In-phase cross-correlation time lags indicated that the therapist tended to respond to participants’ body sway fluctuations in a reactive follower mode, which could indicate visual dominance affecting the therapist during the provision of haptic support. SignificanceOur paradigm implies that in balance rehabilitation more partnership-based methods promote greater postural steadiness. The implications of this finding with regard to motor learning and rehabilitation need to be investigated.

Combining Full and Partial Haptic Guidance Improves Handwriting Skills Development.

It has been shown in previous studies that haptic guidance improves the learning outcomes of handwriting motor skills. Full and partial haptic guidance are developed and evaluated in the literature. In this paper, we present two experimental studies to examine whether combining full and partial haptic guidance is more effective for improving handwriting skills than merely full or partial guidance methods. Experiment I, with 22 participants, compares the effectiveness of merely full and partial haptic guidance methods towards improving learning outcomes of Arabic handwriting. Even though haptic guidance in general is found to be effective and pleasant by all participants, experiment I concludes that there are no statistically significant differences in the learning outcomes between full and partial haptic guidance. Experiment II investigates whether a combination of full and partial haptic guidance could further improve the learning outcomes, compared to merely full or partial haptic guidance. The learning outcomes and quality of experience are measured to evaluate each group's performance. Results from experiment II demonstrate that the combination of full and partial haptic guidance results in statistically significant improvements in the quality of handwriting, compared to mere full or partial haptic guidance. In particular, starting with partial haptic guidance at early stage of learning and then using full guidance at intermediate/advanced learning stages seemed to be the most effective. This implies that partial haptic guidance is more effective to learn the gross shape of handwriting skills (at early stages of the learning process) whereas full haptic guidance is more effective to learn the fine details of the handwriting skills (at intermediate or advanced stage of learning). Therefore, partial-then-full haptic guidance seems to be the most effective to improve learning outcomes.

Does motor expertise facilitate amplitude differentiation of lower limb-movements in an asymmetrical bipedal coordination task?

The motor system’s natural tendency is to move the limbs over equal amplitudes, for example in walking. However, in many situations in which people must perform complex movements, a certain degree of amplitude differentiation of the limbs is required. Visual and haptic feedback have recently been shown to facilitate such independence of limb movements. However, it is unknown whether motor expertise moderates the extent to which individuals are able to differentiate the amplitudes of their limb-movements while being supported with visual and haptic feedback. To answer this question 14 pre-professional dancers were compared to 14 non-dancers on simultaneously generating a small displacement with one foot, and a larger one with the other foot, in four different feedback conditions. In two conditions, haptic guidance was offered, either in a passive or active mode. In the other two conditions, veridical and enhanced visual feedback were provided. Surprisingly, no group differences were found regarding the degree to which the visual or haptic feedback assisted the generation of the different target amplitudes of the feet (mean amplitude difference between the feet). The correlation between the displacements of the feet and the standard deviation of the continuous relative phase between the feet, reflecting the degree of independence of the feet movements, also failed to show between-group differences. Sample entropy measures, indicating the predictability of the foot movements, did show a group difference. In the haptically-assisted conditions, the dancers demonstrated more predictable coordination patterns than the non-dancers as reflected by lower sample entropy values whereas the reverse was true in the visual-feedback conditions. The results demonstrate that motor expertise does not moderate the extent to which haptic tracking facilitates the differentiation of the amplitudes of the lower limb movements in an asymmetrical bipedal coordination task.

Haptic Guidance on Demand: A Grip-Force Based Scheduling of Guidance Forces.

In haptic shared control systems (HSC), a fixed strength of guidance force equates to a fixed level of control authority, which can be insufficient for complex tasks. An adaptable control authority based on operator input can allow the HSC system to better assist the operator under varied conditions. In this paper, we experimentally investigate () an adaptable authority HSC system that provides the operator with a direct way to adjust the control authority based on applied grip force. This system can serve as an intuitive 'manual override' function in case of HSC system malfunction. In a position tracking task, we explore two opposite approaches to adapt the control authority: increasing versus decreasing guidance strength with operator grip. These approaches were compared with unassisted control and two levels of fixed-level haptic guidance. Results show that the grip-adaptable approach allowed the operators to increase performance over unassisted control and over a weak guidance. At the same time, the approach substantially reduced the operator physical control effort required to cope with HSC system disturbances. Predictions based on the formalized model of the complete human-in-the-loop system corresponded to the experimental results, implying that such validated formalization can be used for model-based analysis and design of guidance systems.

Influence of haptic guidance on driving behaviour under degraded visual feedback conditions

Drivers always suffer varying degrees of performance decrements under insufficient visual feedback (VF) conditions. Nowadays, haptic guidance (HG) is a developing assistance technology to enhance steering performance; however, driver reactions to HG under degraded VF conditions are still unclear. Therefore, this study focuses on the influence of HG on driving behaviour when part of the road ahead is occluded. The experimental conditions combined three levels of HG, namely none, weak, and strong torques, with four scenarios of VF: whole, near, mid, and far segments. The driving experiment was conducted using a high-fidelity driving simulator with 12 participants. By analysing the standard deviation of lane position and time-to-lane crossing, it was shown that the lane keeping performance became worse without the HG for the degraded VF of near and far segments compared to that of whole and mid-segments. Furthermore, it indicates that the performance decrement in the worse cases was compensated by the implementation of HG, and the strong torque was significantly more effective than the weak torque. Additionally, the use of HG always resulted in an improved turning manoeuvre while approaching curves in the degraded VF of near and far segments.

Haptic Guidance in Dynamic Environments Using Optimal Reciprocal Collision Avoidance

Human guidance in situations where the users cannot rely on their main sensory modalities, such as assistive or search-and-rescue scenarios, is a challenging task. In this letter, we address the problem of guiding users along collision-free paths in dynamic environments, assuming that they cannot rely on their main sensory modalities. In order to safely guide the subjects, we adapt the optimal reciprocal collision avoidance to our specific problem. The proposed algorithm takes into account the stimuli which can be displayed to the users and the motion uncertainty of the users when reacting to the provided stimuli. The proposed algorithm was evaluated in three different dynamic scenarios. A total of 18 blindfolded human subjects were asked to follow haptic cues in order to reach a target area while avoiding real static obstacles and moving users. Three metrics such as time to reach the goal, length of the trajectories, and minimal distance from the obstacles are considered to compare results obtained using this approach and experiments performed without visual impairments. Experimental results reveal that blindfolded subjects are successfully able to avoid collisions and safely reach the targets in all the performed trials. Although in this letter we display directional cues via haptic stimuli, we believe that the proposed approach can be general and tuned to work with different haptic interfaces and/or feedback modalities.