Realistic Sensation Research Articles

Influence of habituation on pseudo-haptic weight perception of virtual objects

As the demand for realistic sensations in virtual reality (VR) environments escalates, interfaces that exploit cross-modal interactions between vision and haptics are gaining prominence. Pseudo-haptics, offering a tactile illusion without mechanical feedback devices, has emerged as a viable solution to enhance user immersion by simulating various sensations such as texture, fluid resistance, and weight. However, a potential issue in the long-term effectiveness of these illusions, particularly habituation, where the illusion diminishes after prolonged exposure, remains a concern. This study investigates whether habituation to pseudo-haptic weight illusions occurs with extended use. We conducted quantitative measurements of pseudo-haptic weight perception before and after prolonged exposure. The findings indicate a diminishing effect of pseudo-haptic weight illusion with repeated exposure, particularly among participants with high accuracy in weight perception. This study contributes important insights into the design of VR experiences, highlighting the need to consider the habituation effects when implementing pseudo-haptic illusions for weight perception.

A Novel Ungrounded Haptic Device for Generation and Orientation of Force and Torque Feedbacks.

To provide deeper immersion for the user in the virtual environments, both force and torque feedbacks are required rather than the mere use of visual and auditory ones. In this paper, we develop a novel propeller-based Ungrounded Handheld Haptic Device (UHHD) that delivers both force and torque feedbacks in one device to help the user experience a realistic sensation of immersion in a three-dimensional (3D) space. The proposed UHHD uses only a pair of propellers and a set of sliders to continuously generate the desired force and torque feedbacks up to 15N and 1N.m in magnitude in less than 370ms, respectively. The produced force and torque feedbacks are oriented in a desired direction using a gimbal mechanism where the propellers are mounted inside in such a way that a simple structure is obtained. These features facilitate the control of the proposed UHHD and enhance its practicality in various applications. To prove the capability of the system, we model it and elaborate on the force and torque analyses. Next, we develop a robust parallel force/position controller to tackle the structured and unstructured uncertainties. Finally, a measurement setup is manufactured to experimentally evaluate the performance of the UHHD and the controller. The implementation of the controller on the developed UHHD prototype shows that a satisfactory control performance is achievable in terms of offering the desired force and torque feedbacks.

Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario

Background: Stethoscopes and mannequins are basic tools for medical practitioners to diagnose various problems. Restricted by space, resources, and time, these tools and common video training are sometimes inaccessible. These obstacles may be solved by immersive virtual reality (VR) that improves interaction and blurs the line between virtual and reality. Therefore, this study combines virtual and real interaction technologies to investigate the impact of multi-sensory interaction on learning outcomes in chest auscultation.Methods: This study integrated VIVE Tracker technology to provide synchronized tactile stimulation in VR. University students in medical-related disciplines participated in the study. Interviews were conducted to understand how students perceived the effectiveness and usefulness of the thoracic auscultation VR.Conclusion: Most students expressed that tactile stimulation gave realistic sensations and effectively reinforced the immersive effect. Simultaneously, it increased their interest and motivation in learning. The integration of synchronized tactile stimulation is a novel approach that expands the realm of interactivity beyond handheld controller manipulation and creates a platform to simulate medical-related scenarios to learn complex knowledgeand skills.

Perceived Realism of Virtual Textures Rendered by a Vibrotactile Wearable Ring Display.

Wearable haptic displays that relocate feedback away from the fingertip provide a much-needed sense of touch to interactions in virtual reality, while also leaving the fingertip free from occlusion for augmented reality tasks. However, the impact of relocation on perceptual sensitivity to dynamic changes in actuation during active movement remains unclear. In this work, we investigate the perceived realism of virtual textures rendered via vibrations relocated to the base of the index finger and compare three different methods of modulating vibrations with active finger speed. For the first two methods, changing finger speed induced proportional changes in either frequency or amplitude of vibration, and for the third method did not modulate vibration. In psychophysical experiments, participants compared different types of modulation to each other, as well as to real 3D-printed textured surfaces. Results suggest that frequency modulation results in more realistic sensations for coarser textures, whereas participants were less discerning of modulation type for finer textures. Additionally, we presented virtual textures either fully virtually in midair or under augmented reality in which the finger contacted a flat surface; while we found no difference in experimental performance, participants were divided by a strong preference for either the contact or non-contact condition.

Revolution of hands-on model for interventional EUS: How to make a training model.

EUS interventions have an increasing role in the treatment for hepatobiliary-pancreatic diseases. However, the procedure itself is not frequently performed, needs expertise, and carries a high risk of complications. With these limitations, the hands-on practice model is very important for the endoscopist in training for EUS intervention. There have been various hands-on models for EUS interventions, ranging from in vivo living pig model to all-synthetic model. Although a living model provides realistic sensation, the preparation is complex and increases concerns for zoonotic issues. All-synthetic models are easier to prepare and store but not realistic and still need the room for improvement. Hybrid ex vivo model is more widely available and provides various training procedures but still needs special preparation for the porcine tissue.

Perceived realism of haptic rendering methods for bimanual high force tasks: original and replication study

Realistic haptic feedback is a key for virtual reality applications in order to transition from solely procedural training to motor-skill training. Currently, haptic feedback is mostly used in low-force medical procedures in dentistry, laparoscopy, arthroscopy and alike. However, joint replacement procedures at hip, knee or shoulder, require the simulation of high-forces in order to enable motor-skill training. In this work a prototype of a haptic device capable of delivering double the force (35 N to 70 N) of state-of-the-art devices is used to examine the four most common haptic rendering methods (penalty-, impulse-, constraint-, rigid body-based haptic rendering) in three bimanual tasks (contact, rotation, uniaxial transition with increasing forces from 30 to 60 N) regarding their capabilities to provide a realistic haptic feedback. In order to provide baseline data, a worst-case scenario of a steel/steel interaction was chosen. The participants needed to compare a real steel/steel interaction with a simulated one. In order to substantiate our results, we replicated the study using the same study protocol and experimental setup at another laboratory. The results of the original study and the replication study deliver almost identical results. We found that certain investigated haptic rendering method are likely able to deliver a realistic sensation for bone-cartilage/steel contact but not for steel/steel contact. Whilst no clear best haptic rendering method emerged, penalty-based haptic rendering performed worst. For simulating high force bimanual tasks, we recommend a mixed implementation approach of using impulse-based haptic rendering for simulating contacts and combine it with constraint or rigid body-based haptic rendering for rotational and translational movements.

A Novel Decoupled Model Predictive Control-Based Motion Cueing Algorithm for Driving Simulators

The motion cueing algorithm (MCA) is used to reproduce the driving/flying motion sensation of the real land or air vehicles for the users of motion simulator. Highly accurate MCAs are required for the motion simulators to create realistic sensation for the simulator users, otherwise, they might cause motion sickness and user discomfort. Model predictive control (MPC) is a popular technique in development of the high-fidelity MCAs as it is able to consider the motion sensation as well as the workspace constraints of the simulator platform. Tilt coordination in MCA is in charge of creating sustained linear acceleration feeling through tilting the platform cabin and exploiting gravitational acceleration under human motion threshold, to not allow the user to perceive this as a rotational motion. However, the existing MPC-based MCAs have not considered the rate limit to constrain the tilt motion under human threshold which causes violations of the rotational sensation threshold during generating sustained acceleration feeling. Instead, some researchers increase the penalty weights of rotational motions in order to slow down rotational motions. Using this strategy slows down all the simulator rotational motions not only the ones coming from tilt coordination, to generate sustained acceleration feeling, but also from rotational channel (due to real vehicle rotation) which causes motion sensation error and consequently motion sickness. Therefore, as the existing MPC-based MCAs cannot differentiate the rotations of tilt coordination channel (due to sustained acceleration feeling generation), from rotational channel (due to vehicle rotational motions), they are not able to produce accurate rotational motion sensations because of insufficient rotational motions. In this research, a novel decoupled MPC-based MCA is developed to systematically address the issues related to the existing MPC-based MCAs, in terms of inaccurate motion generation, by redesigning the MPC-based MCA using a series of vestibular system-based MPC models and considering tilt rate limit in tilt coordination. The simulation study using MATLAB software is used to verify and validate the proposed MCA compared to the existing MCAs. The proposed decoupled MPC-based MCA is able to generate accurate motion cues compared to those of the existing MCAs.

Haptics and VR: Technology and Applications

Although the most recently developed virtual reality systems convey photorealistic visual information and 3D audio information to users, it is not easy to create immersive and realistic sensations for users to perceive the real world [...]

Exploring Biomimetic Stiffness Modulation and Wearable Finger Haptics for Improving Myoelectric Control of Virtual Hand.

The embodiment of virtual hand (VH) by the user is generally deemed to be important for virtual reality (VR) based hand rehabilitation applications, which may help to engage the user and promote motor skill relearning. In particular, it requires that the VH should produce task-dependent interaction behaviors from rigid to soft. While such a capability is inherent to humans via hand stiffness regulation and haptic interactions, yet it have not been successfully imitated by VH in existing studies. In this paper, we present a work which integrates biomimetic stiffness regulation and wearable finger force feedback in VR scenarios involving myoelectric control of VH. On one hand, the biomimetic stiffness modulation intuitively enables VH to imitate the stiffness profile of the user's hand in real time. On the other hand, the wearable finger force-feedback device elicits a natural and realistic sensation of external force on the fingertip, which provides the user a proper understanding of the environment for enhancing his/her stiffness regulation. The benefits of the proposed integrated system were evaluated with eight healthy subjects that performed two tasks with opposite stiffness requirements. The achieved performance is compared with reduced versions of the integrated system, where either biomimetic impedance control or wearable force feedback is excluded. The results suggest that the proposed integrated system enables the stiffness of VH to be adaptively regulated by the user through the perception of interaction torques and vision, resulting in task-dependent behaviors from rigid to soft for VH.

Soft Pneumatic Fingertip Actuator Incorporating a Dual Air Chamber to Generate Multi-Mode Simultaneous Tactile Feedback

A complete set of diverse haptic feedbacks is essential for a highly realistic and immersive virtual environment. In this sense, a multi-mode haptic interface that simultaneously generates multiple kinds of haptic signals is highly desirable. In this paper, we propose a new silicone-made pneumatically controlled fingertip actuator to render pressure and vibrotactile feedback concurrently to offer a realistic and effective haptic sensation. A new silicone-based stacked dual-layer air chamber was designed. The volume of the chambers is controlled by pneumatic valves with compressed air tanks. The top/upper air chamber renders vibration feedback, whereas the bottom/lower air chamber renders pressure feedback. The proposed silicone-made fingertip actuator is designed so that it can be easily worn at the fingertips. To demonstrate the potential of the system, a virtual environment for rendering three different types of haptic textures was implemented. Extensive performance evaluation and user studies were carried out to demonstrate the proposed actuator’s effectiveness compared to an actuator with single vibrotactile feedback.

Evaluation of Realistic Sensation Using Biological Reaction Measurements for Food Videos Presented by KMMD.

We have prototyped a KANSEI multimedia display (KMMD) that is able to release scent through the screen in order to realize collaboration between images and scents. Two types of “sukiyaki” food videos were presented to subjects using this device, and a method for objectively evaluating the realistic sensation of the food videos was examined using biological reaction measurements. The sukiyaki scent was added to one type of video to improve appetite. Viewers’ saliva flow rate, line of sight, pupil diameter, autonomic nerve activity, and cerebral blood flow were measured at the same time, and changes in these measured values were analyzed. As a result, the scent was effective in improving the sensation, as if the food was present in front of the eyes and increasing the saliva flow rate. Additionally, in a realistic scene, it was found that the line of sight follows the performer’s eating behavior as if the viewers themselves are eating. The sympathetic nervous system temporarily increases, mydriasis occurs, and the frontal lobe is activated. Furthermore, the possibility of objective evaluation of realistic sensations was demonstrated by the correlation between appetite, accompanied by salivary sensation, and the biological reaction measurement results.

Presence of Three-Dimensional Sound Field Facilitates Listeners' Mood, Felt Emotion, and Respiration Rate When Listening to Music.

Many studies have investigated the effects of music listening from the viewpoint of music features such as tempo or key by measuring psychological or psychophysiological responses. In addition, technologies for three-dimensional sound field (3D-SF) reproduction and binaural recording have been developed to induce a realistic sensation of sound. However, it is still unclear whether music listened to in the presence of 3D-SF is more impressive than in the absence of it. We hypothesized that the presence of a 3D-SF when listening to music facilitates listeners’ moods, emotions for music, and physiological activities such as respiration rate. Here, we examined this hypothesis by evaluating differences between a reproduction condition with headphones (HD condition) and one with a 3D-SF reproduction system (3D-SF condition). We used a 3D-SF reproduction system based on the boundary surface control principle (BoSC system) to reproduce a sound field of music in the 3D-SF condition. Music in the 3D-SF condition was binaurally recorded through a dummy head in the BoSC reproduction room and reproduced with headphones in the HD condition. Therefore, music in the HD condition was auditorily as rich in information as that in the 3D-SF condition, but the 3D-sound field surrounding listeners was absent. We measured the respiration rate and heart rate of participants listening to acousmonium and pipe organ music. The participants rated their felt moods before and after they listened to music, and after they listened, they also rated their felt emotion. We found that the increase in respiration rate, the degree of decrease in well-being, and unpleasantness for both pieces in the 3D-SF condition were greater than in the HD condition. These results suggest that the presence of 3D-SF enhances changes in mood, felt emotion for music, and respiration rate when listening to music.

Perception of Vestibular Sensation During Turning Operation of Construction Machine

A major challenge in remote control is the reduction in work efficiency compared with on-board operation. The factors of reduction in work efficiency include a lack of information (information such as perspective, realistic sensation, vibration, and sound) compared to on-board operations. One of the factors is the lack of vestibular/somatosensory information regarding rotation. To clarify the effect of the presence of input of vestibular/somatosensory information regarding rotation on the worker’s operation, we conducted a basic laboratory experiment of a horizontal turning operation. The experimental results indicate that a response appropriate for the input of information regarding rotation can be made only with visual information; however, the reaction is delayed in the case without the input of information regarding rotation in comparison with a case with the input of information regarding rotation.

Psychomotor skills development for Veress needle placement using a virtual reality and haptics-based simulator

PurposeVeress needle (VN) insertion, if not correctly performed, could cause severe injuries to intra-abdominal organs and vessels. Therefore, cognitive and psychomotor skills training is needed. Virtual reality (VR) and haptic technologies have the potential to offer realistic simulations.MethodsWe developed a novel VR and haptic surgical simulator for VN insertion to teach trainees how to correctly puncture the abdominal wall, experiencing realistic tactile sensations throughout the simulation. The simulator allows for both procedural and realistic training. We released two different versions: the first using the OpenHaptics^{text {TM}} (OH) Toolkit and the second exploiting CHAI3D. We evaluated the learning effect using different performance indexes (time to perform the procedure, error in insertion angle, number of undesired contacts with organs) in an insertion task for both experienced urologists and students.ResultsA general improvement of the chosen performance indexes was registered in the second repetition of the task for both groups. From the questionnaires, the simulator leveraging OH provides the trainee with a more precise haptic feedback, whereas the one exploiting CHAI3D allows them to perform the procedure more easily thanks to the better visualization of the virtual environment. The results proved that the participants appreciated both implementations, and the System Usability Scale (SUS) test resulted in a “good” usability.ConclusionThe haptics-based and VR simulator has shown the potential to be an important resource for the basic urological training in obtaining the pneumoperitoneum and improving the acquisition of the necessary psychomotor skills, allowing for extended and more effective training without compromising patient safety.

A Comparison Between Separated Electrodes and Concentric Electrodes for Electrotactile Stimulation

Equipping prosthetic devices with sensory feedback capability is a continuing research challenge to increase their control and embodiment, and to decrease their rejection by their users. Electrotactile stimulation is one of the non-invasive sensory feedback techniques with a low power, light-weight and relatively easy to control stimulation mechanism to provide the feedback. Currently, there are two electrode setups used in the literature, concentric electrodes and separated electrodes, to apply the electrotactile stimulation to deliver tactile feedback. There is minimal data available in the literature to compare the two electrode setups. This study compares the impact of the electrode arrangements on the dynamic range, just noticeable difference (JND), comfort level, degree of localisation, intensity and type of sensation induced, and the resulting EMG interference from the electrotactile stimulation. The experimental results presented suggest that the concentric electrodes induce a more comfortable and realistic sensation with a reduction in the probability of inducing undesired feeling of pain, pinch or pinprick sensations. Further, the concentric electrodes result in an increased localisation of the induced sensation whilst maintaining a similar dynamic range and JND. EMG interference was shown to be lower for the concentric electrodes.

Crowd Navigation in VR: Exploring Haptic Rendering of Collisions.

Virtual reality (VR) is a valuable experimental tool for studying human movement, including the analysis of interactions during locomotion tasks for developing crowd simulation algorithms. However, these studies are generally limited to distant interactions in crowds, due to the difficulty of rendering realistic sensations of collisions in VR. In this article, we explore the use of wearable haptics to render contacts during virtual crowd navigation. We focus on the behavioral changes occurring with or without haptic rendering during a navigation task in a dense crowd, as well as on potential after-effects introduced by the use haptic rendering. Our objective is to provide recommendations for designing VR setup to study crowd navigation behavior. To the end, we designed an experiment (N=23) where participants navigated in a crowded virtual train station without, then with, and then again without haptic feedback of their collisions with virtual characters. Results show that providing haptic feedback improved the overall realism of the interaction, as participants more actively avoided collisions. We also noticed a significant after-effect in the users' behavior when haptic rendering was once again disabled in the third part of the experiment. Nonetheless, haptic feedback did not have any significant impact on the users' sense of presence and embodiment.

Magnetorheological Fluid Haptic Shoes for Walking in VR.

In this article, present RealWalk, a pair of haptic shoes for HMD-based VR, designed to create realistic sensations of ground surface deformation, and texture using Magnetorheological fluid (MR fluid). RealWalk offers a novel interaction scheme through the physical interaction between the shoes, and the ground surfaces while walking in VR. Each shoe consists of two MR fluid actuators, an insole pressure sensor, and a foot position tracker. The MR fluid actuators are designed in the form of multi-stacked disc structure with a long flow path to maximize the flow resistance. With changing the magnetic field intensity in MR fluid actuators based on the ground material in the virtual scene, the viscosity of MR fluid is changed accordingly. When a user steps on the ground with the shoes, the two MR fluid actuators are pressed down, creating a variety of ground material deformation such as snow, mud, and dry sand. We built an interactive VR application, and compared RealWalk with vibrotactile-based haptic shoes in four different VR scenes: grass, sand, mud, and snow. We report that, compared to vibrotactile-haptic shoes, RealWalk provides higher ratings in all scenes for discrimination, realism, and satisfaction. We also report qualitative user feedback for their experiences.

Production of ERCP training model using a 3D printing technique (with video)

BackgroundERCP training models are very different in terms of anatomical differences, ethical issues, storage problems, realistic tactile sensation, durability and portability. There is no easy way to select an optimized model for ERCP training. If the ERCP training model could be made as a soft silicone model using 3D printing technique, it would have numerous advantages over the models presented so far. The purpose of this study was to develop an optimized ERCP training model using a 3D printing technique and to try to find ways for implementing various practical techniques.MethodsAll organ parts of this model were fabricated using silicone molding techniques with 3D printing. Especially, various anatomy of the ampulla of Vater and common bile duct (CBD) were creatively designed for different diagnostic and therapeutic procedures. In order to manufacture each of the designed organ parts with silicone, a negative part had to be newly designed to produce the molder. The negative molders were 3D printed and then injection molding was applied to obtain organ parts in silicone material. The eight different types of ampulla and CBD were repeatedly utilized and replaced to the main system as a module-type.ResultsERCP training silicone model using 3D technique was semi-permanently used to repeat various ERCP procedures. All ERCP procedures using this model could be observed by real-time fluoroscopic examination as well as endoscopic examination simultaneously. Using different ampulla and CBD modules, basic biliary cannulation, difficult cannulation, stone extraction, mechanical lithotripsy, metal stent insertion, plastic stent insertion, and balloon dilation were successfully and repeatedly achieved. Endoscopic sphincterotomy was also performed on a specialized ampulla using a Vienna sausage. After repeat procedures and trainings, all parts of organs including the ampulla and CBD modules were not markedly damaged or deformed.ConclusionsWe made a specialized ERCP training silicon model with 3D printing technique. This model is durable, relatively cheap and easy to make, and thus allows the users to perform various specialized ERCP techniques, which increases its chances of being a good ERCP training model.

Virtual Reality to Foster Social Integration by Allowing Wheelchair Users to Tour Complex Archaeological Sites Realistically

People with disabilities encounter numerous barriers when dealing with the simplest and most usual things in their daily lives. This is even more remarkable when they are faced with archaeological heritage buildings or environments. People with reduced mobility come too often upon architectural barriers that stop them from enjoying their visits to sites and monuments. This paper introduces a virtual reality (VR) experience developed to provide people in wheelchairs with the most realistic sensations while virtually touring some archaeological sites. To this end, the remote sensing of the site enables the production of a realistic 3D model leading to the creation of a virtual world that the user will explore. This VR application has been developed to traverse one of the most important monumental buildings in Spanish Protohistory, the site of Cancho Roano (Zalamea de la Serena, Spain).

An Instrument for Remote Kissing and Engineering Measurement of Its Communication Effects Including Modified Turing Test

Various communication systems have been developed to integrate the haptic channel in digital communication. Future directions of such haptic technologies are moving towards realistic virtual reality applications and human-robot social interaction. With the digitisation of touch, robots equipped with touch sensors and actuators can communicate with humans on a more emotional and intimate level, such as sharing a hug or kiss just like humans do. This paper presents the design guideline, implementation and evaluations of a novel haptic kissing machine for smart phones - the Kissenger machine. The key novelties and contributions of the paper are: (i) A novel haptic kissing device for mobile phones, which uses dynamic perpendicular force stimulation to transmit realistic sensations of kissing in order to enhance intimacy and emotional connection of digital communication; (ii) Extensive evaluations of the Kissenger machine, including a lab experiment that compares mediated kissing with Kissenger to real kissing, a unique haptic Turing test that involves the first academic study of human-machine kiss, and a field study of the effects of Kissenger on long distance relationships. The first experiment showed that mediated kissing with Kissenger elicited similar ratings for pleasure, arousal and user experience as real kissing. Experiment 2 confirmed our hypothesis that interrogators have a higher chance of winning the Imitation Game (Turing test) when Kissenger is used during the game. Results from experiment 3 showed that long relationship couples who used Kissenger for a week experienced increased relationship satisfaction and decreased perceived stress.