Haptic Algorithm Research Articles

Music, Art Installations and Haptic Technology

This paper presents some directions on the design, development and creative use of haptic systems for musical composition, performance and digital art creation. This research has been conducted both from an artistic and a technical point of view and its ambition, over the last decade, apart from the artistic outcome, was to introduce the field of haptics to artistic communities based on an open, do it yourself—DIY ethos. The five directions presented here are not in any sense exhaustive and are based principally on a series of collaborative works and more personal open-ended explorations with the medium of haptics and, more specifically, force-feedback interaction. They will be highlighted along with information about the interaction models and their application to artistic works created by the author and other colleagues. Those directions are (i) Haptic Algorithms and Systems; (ii) Performers Intercoupling; (iii) Haptic Interfaces as Part of the Artistic Practice; (iv) Electromechanical Sound Generation; and (v) Media Art and Art Installations. The interdisciplinary field of musical haptics still has a relatively minor position in the sound and music computing research agendas and, more importantly, its artistic dimension is very rarely discussed. The findings of this research aim to indicate and clarify potential research pathways and offer some results on the use of haptics and force-feedback systems in an artistic context.

Haptic control of steer-by-wire systems using parameter estimation of rack system lateral load model

This paper describes a haptic control of steer-by-wire systems for rendering the conventional steering system under various road conditions using parameter estimation of rack system lateral load model. To design the conventional steering system model, the dynamic model of a rack system has been developed as a mass-spring-damper system with a friction model. An online parameter estimation of the rack system was designed to consider the various road conditions in the haptic rendering. In a steering wheel system, which is a haptic device, a limit cycle, and instable behavior can occur due to the sampling rate and quantization. To prevent the limit cycle and instable behavior, a passivity analysis was conducted and constraint conditions of the rendering coefficient was derived. The haptic control algorithm is designed to render the conventional steering system without limit cycles using passivity conditions. The performance of the proposed controller was evaluated via both computer simulations and vehicle tests under various steering conditions. The results demonstrate that the proposed algorithm ensures haptic rendering performance on dry and wet asphalt conditions.

Evaluation of Direct Haptic 4D Volume Rendering of Partially Segmented Data for Liver Puncture Simulation

This work presents an evaluation study using a force feedback evaluation framework for a novel direct needle force volume rendering concept in the context of liver puncture simulation. PTC/PTCD puncture interventions targeting the bile ducts have been selected to illustrate this concept. The haptic algorithms of the simulator system are based on (1) partially segmented patient image data and (2) a non-linear spring model effective at organ borders. The primary aim is to quantitatively evaluate force errors caused by our patient modeling approach, in comparison to haptic force output obtained from using gold-standard, completely manually-segmented data. The evaluation of the force algorithms compared to a force output from fully manually segmented gold-standard patient models, yields a low mean of 0.12 N root mean squared force error and up to 1.6 N for systematic maximum absolute errors. Force errors were evaluated on 31,222 preplanned test paths from 10 patients. Only twelve percent of the emitted forces along these paths were affected by errors. This is the first study evaluating haptic algorithms with deformable virtual patients in silico. We prove haptic rendering plausibility on a very high number of test paths. Important errors are below just noticeable differences for the hand-arm system.

Haptic rendering of isosurfaces directly from medical images.

Virtual environments for surgical training, planning and rehearsal have the potential to significantly enhance patient treatment and diagnosis. Haptic feedback devices provide forces to the physician through a manipulator, simulating palpation, scalpel cuts, or retraction of tissue. While haptics have been studied in other fields, medical applications of haptics remain in their infancy. We propose a method of haptically rendering isosurfaces (representing hard structures) directly from anatomical datasets rather than through traditional intermediate graphical representations such as polygons. Our algorithm determines an implicit surface representation of a volumetric isosurface on the fly, and renders the structure using standard haptic algorithms to calculate the forces felt by the user. This approach has the advantage of providing easy access to the rich volume dataset containing the actual anatomy. By relating Hounsfield units to density, haptic rendering has the ability to provide different resistances based on the tissue type being rendered. We developed and tested our algorithm using a quadric implicit surface, a common primitive in computer graphics, and have applied it to a variety of anatomical image datasets. Our paper describes the algorithm in sufficient detail to facilitate reproduction by others.

Effects of Haptic and 3D Audio Feedback on Operator Performance and Workload for Quadrotor UAVs in Indoor Environments

<div class=""abs_img""><img src=""[disp_template_path]/JRM/abst-image/00260005/06.jpg"" width=""300"" />Haptic and audio 3D feedback</div> Unmanned aerial vehicles (UAVs) have many potential applications in indoor environments. However, limited visual feedback makes it difficult to pilot UAVs in cluttered and enclosed spaces. Haptic feedback combined with visual feedback has been shown to reduce the number of collisions of UAVs in indoor environments, but has generally resulted in an increase in the mental workload of the operator. This paper investigates the potential of combining novel haptic and 3D audio feedback to provide additional information to operators of UAVs to improve performance and reduce workload. Two haptic feedback and two 3D audio feedback algorithms are presented and tested in a simulation-based human subject experiment. Operator workload is quantified using standard measures and a novel application of behavioral entropy. Experimental results indicate that 3D haptic feedback improved UAV pilot performance. Pilot workload was also improved for one of the haptic algorithms in one of the control directions (lateral). The 3D audio feedback algorithms investigated in this study neither improved nor degraded pilot performance. </span>

A new user-adapted search haptic algorithm to navigate along filiform structures.

One of the mayor research challenges of this century is the understanding of the human brain. Regarding this field line, simulation based research is gaining importance. A large amount of money is being spent in huge international projects such as The Human Brain Project [1] and The Blue Brain [2]. The behavior of the brain and, therefore, the behavior of brain simulations depend to a large extend on the neural topology. Neural elements are organized in a connected, dense, complex network of thread-like (i.e., filiform) structures. The analysis of a computer-based simulation using just the visual modality is a highly complex task due to the complexity of the neural topology and the large amounts of multi-variable and multi-modal data generated by computer simulations. In this paper, we propose the use of haptic devices to aid in the navigation along these neural structures, helping neurobiologists in the analysis of neural network topologies. However, haptic navigation constrained to complex filiform networks entails problems when these structures have high frequency features, noise and/or complex branching nodes. We address these issues by presenting a new user-adapted search haptic method that uses the forces exerted by the users to infer their intentions. In addition, we propose a specific calibration technique to adapt the haptic navigation to the user's skills and to the data. We validate this approach through a perceptual study. Finally, we show in this paper the application of our method to the analysis of dense and complex filiform structures in the neurobiology context. Additionally, our technique could be applied to other problems such as electronic circuits and graph exploration.

Tech to the Future: Making a "Kinection" with Haptic Interaction

The Kinect camera has been a great success in gaming but more importantly it has opened up new fields for researchers. The data streamed from the Kinect in real time requires new haptic algorithms. By developing a novel haptic rendering algorithm we have shown feasibility for using the Kinect in telerobotic applications such as robotic surgery. Providing the surgeon with a sense of touch could improve telerobotic surgery. One way to obtain data for such a haptic rendering is from sensors in the robot or on end-effectors (i.e., the surgical tools). But the latter raises issues of calibration as well as sterilization. An alternative approach is to use depth camera information for haptic rendering. RGB-D cameras like Kinect present a possible way to do this, if the device can be suitably miniaturized. Developing this algorithm into a practical tool for robotic surgeons requires a number of improvements and extensions. One concern is the presence of noise and shadows in depth data especially since shadows will be caused by surgical tools positioned between the Kinect and tissue being acted upon by the tools. A second challenge is to enhance the resolution of the depth data since surgical operations are at a relatively small size scale. A third concern is the need to register the virtual 3-D environment with the real environment. Finally, there is a need to capture haptic properties of tissues such that they can be rendered realistically with the appropriate stiffness.

Haptic Interaction with Elastic Volumetric Structures

Considerable efforts have been done to produce realistic results when simulating interaction with elastic materials. Many applications such as surgery planning, medical training, or virtual sculpting would benefit from a plausible simulation scenario. However, even though many works have proposed very satisfactory results, realistic simulation of deformable bodies is still an open issue. One of the challenges when designing a realistic elastic body simulation is the huge amount of data that needs to be processed. For the inner properties of the material are crucial when it comes to reproduce the elastic problem, the simulation naturally calls for volumetric information. In this paper the authors propose a technique to interactively deform 3D images, such as those acquired by a CT scanner. While producing a physically plausible haptic feedback, deformation and visualization algorithms produce an efficient and natural feeling. Using a free form deformation structure as a wrapper, it is possible to deform complex structures at high frame rates, independently of the size of the volume.

Force and Contact Location Shading Methods for Use Within Two- and Three-Dimensional Polygonal Environments

Current state-of-the-art haptic interfaces only provide kinesthetic (force) feedback, yet studies have shown that providing tactile feedback in concert with kinesthetic information can dramatically improve a person's ability to dexterously interact with and explore virtual environments. In this research, tactile feedback was provided by a device, called a contact location display (CLD), which is capable of rendering the center of contact to a user. The chief goal of the present work was to develop algorithms that allow the CLD to be used with polygonal geometric models, and to do this without the resulting contact location feedback being overwhelmed by the perception of polygonal edges and vertices. Two haptic shading algorithms were developed to address this issue and successfully extend the use of the CLD to 2D and 3D polygonal environments. Two experiments were run to evaluate these haptic shading algorithms. The first measured perception thresholds for rendering faceted objects as smooth objects. It was found that the addition of contact location feedback significantly increased user sensitivity to edges and that the use of shading algorithms was able to significantly reduce the number of polygons needed for objects to feel smooth. The second experiment explored the CLD device's ability to facilitate exploration and shape recognition within a 3D environment. While this study provided a validation of our 3D algorithm, as people were able to identify the rendered objects with reasonable accuracy, this study underscored the need for improvements in the CLD device design in order to be effectively used in general 3D environments.

Multirate Energy-Bounding Algorithm for High-Fidelity Stable Haptic Interaction Control

This brief describes a multirate energy-bounding algorithm that significantly increases the displayable stiffness while guaranteeing the overall haptic interaction stability. Unlike haptic interaction control algorithms that are based on the multirate simulation of virtual environments, the proposed multirate approach separates the low-level control thread from the haptic thread. The haptic thread updates the haptic interaction computation, or the collision detection and response and dynamic or deformation simulation, of a virtual environment at slow rate while the low-level control thread updates the actuator command force determined by the energy-bounding algorithm to guarantee the interaction stability at a high update rate. An analytical derivation study shows that the displayable stiffness of virtual environments can be significantly increased by an order of N, which is the ratio of the fast actuator command signal update to the slow haptic simulation update. Experiments using a commercial haptic device demonstrate that the displayed stiffness significantly increases while guaranteeing stability even when the haptic rendering rate is slow.

The influence of haptic support algorithm dynamics on the efficacy of motor learning

Background: Repetitive task training, delivered either by a therapist or haptic robot is the core of modern rehabilitation of movement. In the current rehabilitation, robotics-based movement training the level of haptic support assisting the movement is rather stationary and may remain the same for periods of days. The aim of this paper was to investigate the influence of haptic support algorithms (HSA) dynamics on the outcome of motor learning. Methods: Twenty-seven neurologically intact participants, divided into three groups supported by dynamically different HSA, played a rather demanding two degrees of freedom motor task (virtual reality based table football) to learn wrist movements with their inferior arm. The evaluation before training without robotic support was followed by the training session and concluded with evaluation after training without robotic support. Results: The results showed significant improvement in all three groups, but the statistical analysis reveals the difference within groups. The selection of the HSA that is appropriate for the given motor task had a significant influence on the level of acquired motor skills after the training period. Conclusions: The results of this study suggest that for every motor task or equivalently for every motor ability of a particular subject such a HSA scheme exists and should be implemented that maximizes training effects in a limited number of training attempts.

Stability enhancement of haptic interaction by frequency-dependent damping and its application to scaled teleoperation

This paper presents a frequency-dependent damping element, called an analog input shaper (AIS), to improve the stability of haptic interaction with virtual or real environments. High frequency inputs to a haptic interface, which usually occur in the collision with very stiff objects, can bring limit cycle oscillations and instabilities. In order to reduce these high frequency inputs and thus to improve the stability of haptic interaction, the AIS is added to the haptic system. Moreover, the AIS is applied to a passivity-based haptic stability control algorithm, called an energy-bounding algorithm (EBA), to increase the displayed impedance range by the EBA. Through scaled teleoperation experiments using an atomic force microscope (AFM), we show that the AIS can enhance stability during haptic interaction and consequently increase impedance range that a haptic interface can stably display. Also, we show that the AIS can alleviate conservativeness of the passivity-based haptic control algorithms.

Real-time Haptic Modeling and Simulation for Prosthetic Insertion

In this work a surgical simulator is produced which enables a training otologist to conduct a virtual, real-time prosthetic insertion. The simulator provides the Ear, Nose and Throat surgeon with real-time visual and haptic responses during virtual cochlear implantation into a 3D model of the human Scala Tympani (ST). The parametric model is derived from measured data as published in the literature and accounts for human morphological variance, such as differences in cochlear shape, enabling patient-specific preoperative assessment. Haptic modeling techniques use real physical data and insertion force measurements, to develop a force model which mimics the physical behavior of an implant as it collides with the ST walls during an insertion. Output force profiles are acquired from the insertion studies conducted in the work, to validate the haptic model. The simulator provides the user with real-time, quantitative insertion force information and associated electrode position as user inserts the virtual implant into the ST model. The information provided by this study may also be of use to implant manufacturers for design enhancements as well as for training specialists in optimal force administration, using the simulator. The paper reports on the methods for anatomical modeling and haptic algorithm development, with focus on simulator design, development, optimization and validation. The techniques may be transferrable to other medical applications that involve prosthetic device insertions where user vision is obstructed.

Six Degrees-of-Freedom Haptic Interaction with Fluids

We often interact with fluids in our daily life, either through tools such as when holding a glass of water or directly with our body when we swim or we wash our hands. Multimodal interactions with virtual fluids would greatly improve the simulations realism, particularly through haptic interaction. However, achieving realistic, stable, and real-time force feedback from fluids is particularly challenging. In this work, we propose a novel approach that allows real-time six Degrees of Freedom (DoF) haptic interaction with fluids of variable viscosity. Our haptic rendering technique, based on a Smoothed-Particle Hydrodynamics physical model, provides a realistic haptic feedback through physically based forces. 6DoF haptic interaction with fluids is made possible thanks to a new coupling scheme and a unified particle model, allowing the use of arbitrary-shaped rigid bodies. Particularly, fluid containers can be created to hold fluid and hence transmit to the user force feedback coming from fluid stirring, pouring, shaking, and scooping, to name a few. Moreover, we adapted an existing visual rendering algorithm to meet the frame rate requirements of the haptic algorithms. We evaluate and illustrate the main features of our approach through different scenarios, highlighting the 6DoF haptic feedback and the use of containers.

幾何学的力覚提示アルゴリズムの力学的解釈とインピーダンス型およびアドミッタンス型の実装法

Geometric haptic rendering algorithms, in which virtual proxy positions are determined according to input device positions, are widely used for impedance-type haptic rendering. Such methods are intuitively simple because they reduce mechanistic problems of determining reaction forces into geometric problems of updating proxy positions. Aiming to extend the scope of application of geometric haptic algorithms, this paper presents a mechanistic interpretation of geometric algorithms and proposes two techniques. One is a technique to combine geometric algorithms and viscous damping in impedance-type haptic rendering. The other is a technique to utilize geometric algorithms in admittance-type haptic rendering. The results of implementation experiments are presented.