Multisensory Feedback Research Articles

The multisensory control of sequential actions

Many motor tasks are comprised of sequentially linked action phases, as when reaching for, lifting, transporting, and replacing a cup of coffee. During such tasks, discrete visual, auditory and/or haptic feedback are typically associated with mechanical events at the completion of each action phase, as when breaking and subsequently making contact between the cup and the table. An emerging concept is that important sensorimotor control operations, that affect subsequent action phases, are centred on these discrete multisensory events. By predicting sensory feedback at the completion of action phases, and comparing with the actual feedback that arises, task performance can be continuously monitored. If errors are detected, the sensorimotor system can quickly respond with task-protective corrective actions. The aim of this study was to investigate how discrete multisensory feedback at the completion of action phases are used in these control operations. To investigate this question, 42 healthy human participants (both male and female) performed a visually guided sequential reaching task where auxiliary discrete visual, auditory and/or haptic feedback was associated with the completion of action phases. Occasionally however, this feedback was removed in one or two modalities. The results show that although the task was visually guided, its control was critically influenced by discrete auditory and haptic feedback. Multisensory integration effects occurred, that enhanced the corrective actions, when auditory feedback was unexpectedly removed along with haptic or visual feedback. This multisensory enhancement may facilitate the ability to detect errors during sequential actions and amplify task-protective corrective actions.

Light touch alters vestibular-evoked balance responses: Insights into dynamics of sensorimotor reweighting.

Integrated multisensory feedback plays a crucial role in balance control. Minimal fingertip contact with a surface (light-touch), reduces center of pressure (CoP) by adding sensory information about postural orientation and balance state. Electrical vestibular stimulation (EVS) can increase sway by adding erroneous vestibular cues. This juxtaposition of conflicting sensory cues can be exploited to explore the dynamics of sensorimotor adaptations. We used continuous stochastic EVS (0-25Hz; ±4mA; 200-300s) to evoke balance responses in CoP (Exp-1, Exp-2). Systems analyses (coherence, gain) quantified coupling and size of balance responses to EVS. We had participants either touch (TOUCH; <2N) or not touch (NO-TOUCH) a load cell during EVS (Exp-1, Exp-2), or we intermittently removed the touch surface (Exp-2) to measure the effects of light touch on vestibular-evoked balance responses. We hypothesized that coherence and gain between EVS and CoP would decrease, consistent with the CNS down-weighting vestibular cues that conflict with light touch. Light touch reduced CoP displacement, but increased variation in the CoP signal explained by EVS input. Significant coherence between EVS and CoP was observed up to ~30Hz in both conditions but was significantly greater in the TOUCH condition from 12-28.5-Hz. Conversely, EVS-CoP gain was 63% lower in TOUCH, compared to NO-TOUCH. Our findings show that light touch can reduce the size of vestibular-evoked responses but also increase high frequency vestibular contributions for sway. This suggests that the CNS can use discrete changes in sensory inputs to alter balance behavior but cannot fully suppress responses to a potent cue.

The Application of VR Technology in the Field of Gaming

With the rapid advancement of Virtual Reality (VR) technology, its application in the gaming sector has introduced an unprecedented level of immersion for players. This paper aims to explore the integration of VR technology in game design and assess its potential and challenges in enhancing player experiences. The motivation for this research stems from the fundamental differences between VR and traditional games in terms of immersion, interactivity, and multisensory feedback. The paper provides an overview of the basic concepts and principles of VR technology, compares it with traditional gaming technologies, and discusses its role in delivering immersive gaming experiences, enhancing player interaction, and providing multisensory feedback, particularly in multiplayer online gaming contexts. Despite the rapid growth of the VR gaming market, high hardware costs, lack of content, and technical challenges remain significant barriers to widespread adoption. The conclusion of this paper underscores the significance and potential for future development of VR technology in the gaming field, while also identifying key areas for improvement necessary for mainstream acceptance.

Shaping corticospinal pathways in virtual reality: effects of task complexity and sensory feedback during mirror therapy in neurologically intact individuals

BackgroundRestoration of limb function for individuals with unilateral weakness typically requires volitional muscle control, which is often not present for individuals with severe impairment. Mirror therapy—interventions using a mirror box to reflect the less-impaired limb onto the more-impaired limb—can facilitate corticospinal excitability, leading to enhanced recovery in severely impaired clinical populations. However, the mirror box applies limitations on mirror therapy, namely that all movements appear bilateral and are confined to a small area, impeding integration of complex activities and multisensory feedback (e.g., visuo-tactile stimulation). These limitations can be addressed with virtual reality, but the resulting effect on corticospinal excitability is unclear.ObjectiveExamine how virtual reality-based unilateral mirroring, complex activities during mirroring, and visuo-tactile stimulation prior to mirroring affect corticospinal excitability.Materials and methodsParticipants with no known neurological conditions (n = 17) donned a virtual reality system (NeuRRoVR) that displayed a first-person perspective of a virtual avatar that matched their motions. Transcranial magnetic stimulation-induced motor evoked potentials in the nondominant hand muscles were used to evaluate corticospinal excitability in four conditions: resting, mirroring, mirroring with prior visuo-tactile stimulation (mirroring + TACT), and control. During mirroring, the movements of each participant’s dominant limb were reflected onto the nondominant limb of the virtual avatar, and the avatar’s dominant limb was kept immobile (i.e., unilateral mirroring). The mirroring + TACT condition was the same as the mirroring condition, except that mirroring was preceded by visuo-tactile stimulation of the nondominant limb. During the control condition, unilateral mirroring was disabled. During all conditions, participants performed simple (flex/extend fingers) and complex (stack virtual blocks) activities.ResultsWe found that unilateral mirroring increased corticospinal excitability compared to no mirroring (p < 0.001), complex activities increased excitability compared to simple activities during mirroring (p < 0.001), and visuo-tactile stimulation prior to mirroring decreased excitability (p = 0.032). We also found that these features did not interact with each other.DiscussionsThe findings of this study shed light onto the neurological mechanisms of mirror therapy and demonstrate the unique ways in which virtual reality can augment mirror therapy. The findings have important implications for rehabilitation for design of virtual reality systems for clinical populations.

The Technology to Enhance Patient Motivation in Virtual Reality Rehabilitation: A Review.

Virtual reality (VR) technology has experienced a steady rise and has been widely applied in the field of rehabilitation. The integration of VR technology in rehabilitation has shown promising results in enhancing their motivation for treatment, thereby enabling patients to actively engage in rehab training. Despite the advancement, there is a dearth of comprehensive summary and analysis on the use of VR technology to enhance patient motivation in rehabilitation. Thus, this narrative review aims to evaluate the potential of VR technology in enhancing patient motivation during motor rehabilitation training. This review commences with an explanation of how enhancing motivation through the VR rehabilitation system could improve the efficiency and effectiveness of rehabilitation training. Then, the technology was analyzed to improve patient motivation in the present VR rehabilitation system in detail. Furthermore, these technologies are classified and summarized to provide a comprehensive overview of the state-of-the-art approaches for enhancing patient motivation in VR rehabilitation. Findings showed VR rehabilitation training utilizes game-like exercises to enhance the engagement and enjoyment of rehabilitation training. By immersing patients in a simulated environment with multisensory feedback, VR systems offer a unique approach to rehabilitation that can lead to improved patient motivation. Both ultimately lead to improved patient outcomes, which is not typically achievable with traditional rehabilitation methods. The review concludes that VR rehabilitation presents an opportunity to improve patient motivation and adherence to long-term rehabilitation training. However, to further enhance patient self-efficacy, VR rehabilitation should integrate psychology and incorporate methods. Moreover, it is necessary to build a game design theory for rehabilitation games, and the latest VR feedback technology should also be introduced.

Might patients with cerebellar ataxia benefit from the Computer Assisted Rehabilitation ENvironment (CAREN)? A pilot study focusing on gait and balance.

Introduction: Ataxia is a neurological symptom that causes decreased balance, loss of coordination, and gait alterations. Innovative rehabilitation devices like virtual reality (VR) systems can provide task-oriented, repetitive and intensive training with multisensorial feedback, thus promoting neuroplastic processes. Among these VR technologies, the Computer Assisted Rehabilitation ENvironment (CAREN) associates a split belt treadmill on a 6-degrees of freedom platform with a 180° VR screen and a Vicon motion capture system to monitor patients' movements during training sessions. Methods: Eight patients affected by cerebellar ataxia were enrolled and received 20 sessions of CAREN training in addition to standard rehabilitation treatment. Each patient was evaluated at the beginning and at the end of the study with 3D gait analysis and clinical scales to assess balance, gait function and risk of falls. Results: We found improvements in kinematic, kinetic, and electromyographic parameters (as per pre-post- CAREN training), as well as in clinical outcomes, such as balance and risk of falls in ataxic patients. In addition, we found that trunk rotation improved, after CAREN intervention, approximating to the normative values. Discussion: Our results suggested that CAREN might be useful to improve specific biomechanical parameters of gait in ataxic patients.

Self-Guided DMT: Exploring a Novel Paradigm of Dance Movement Therapy in Mixed Reality for Children with ASD.

Children diagnosed with Autism Spectrum Disorder (ASD) often exhibit motor disorders. Dance Movement Therapy (DMT) has shown great potential for improving the motor control ability of children with ASD. However, traditional DMT methods often lack vividness and are difficult to implement effectively. To address this issue, we propose a Mixed Reality DMT approach, utilizing interactive virtual agents. This approach offers immersive training content and multi-sensory feedback. To improve the training performance of children with ASD, we introduce a novel training paradigm featuring a self-guided mode. This paradigm enables the rapid creation of a virtual twin agent of the child with ASD using a single photo to embody oneself, which can then guide oneself during training. We conducted an experiment with the participation of 24 children diagnosed with ASD (or ASD propensity), recording their training performance under various experimental conditions. Through expert rating, behavior coding of training sessions, and statistical analysis, our findings revealed that the use of the twin agent for self-guidance resulted in noticeable improvements in the training performance of children with ASD. These improvements were particularly evident in terms of enhancing movement quality and refining overall target-related responses. Our study holds clinical potential in the field of medical treatment and rehabilitation for children with ASD.

Optimization Approach for Multisensory Feedback in Robot-Assisted Pouring Task

Optimization Approach for Multisensory Feedback in Robot-Assisted Pouring Task

Effects of Congruent Multisensory Feedback on the Perception and Performance of Virtual Reality Hand-Retargeted Interaction

Effects of Congruent Multisensory Feedback on the Perception and Performance of Virtual Reality Hand-Retargeted Interaction

Effects of Error Modulation-Based Visual and Haptic Feedback Fusion Strategies on Motor Learning and Motivation

Visual or haptic feedback based on error modulation has been used to improve the effect of robot-assisted rehabilitation training. However, there are several investigations on the effects of error modulation-based visual and haptic feedback fusion strategies on motor learning and motivation. To observe the influence of different feedback fusion strategies on motor learning and motivation, a parallel controlled study was conducted, dividing 30 healthy subjects into three groups with similar skill levels. The no error modulation group received visual and haptic feedback without error modulation; the visual amplification haptic reduction group received visual error amplification combined with haptic error reduction, and the visual reduction haptic amplification (VRHA) group received visual error reduction combined with haptic error amplification. Each subject implemented a trajectory-tracking task with an upper limb rehabilitation robot. They went through baseline, training, assessment, and generalization tests and completed 340 consecutive tracking movements. To evaluate motor learning and motivation, the average tracking error, the root mean square (RMS) of surface electromyography (sEMG) signals, and the intrinsic motivation inventory scale were all examined. In the assessment tests, the average tracking error was significantly decreased in all three groups. In particular, the VRHA group had a larger reduction in average tracking error in the generalization test, lower RMS of sEMG signals both in the assessment and generalization tests and higher perceived competence in the assessment tests. The VRHA fusion strategy significantly improved the subjects’ motor learning and transfer ability, decreased muscle activation, and increased motor learning motivation. These findings may provide some new insights for multisensory feedback fusion technology in the application of rehabilitation robots.

Feeling Connected: The Role of Haptic Feedback in VR Concerts and the Impact of Haptic Music Players on the Music Listening Experience

Today, some of the most widely attended concerts are in virtual reality (VR). For example, the videogame Fortnite recently attracted 12.3 million viewers sitting in homes all over the world to a VR Travis Scott rap concert. As such VR concerts become increasingly ubiquitous, we are presented with an opportunity to design more immersive virtual experiences by augmenting VR with other multisensory technologies. Given that sound is a multi-modal phenomenon that can be experienced sonically and vibrationally, we investigated the importance of haptic feedback to musical experiences using a combination of qualitative and empirical methodologies. Study 1 was a qualitative study demonstrating that, unlike their live counterparts, current VR concerts make it harder for audiences to form a connection with artists and their music. Furthermore, VR concerts lack multisensory feedback and are perceived as less authentic than live concert experiences. Participants also identified a variety of different kinds of touch that they receive at live concerts and suggested that ideal VR concerts would replicate physical touch and thermal feedback from the audience, emotional touch, and vibrations from the music. Specifically, users advocated for the use of haptic devices to increase the immersiveness of VR concert experiences. Study 2 isolated the role of touch in the music listening experience and empirically investigated the impact of haptic music players (HMPs) on the audio-only listening experience. An empirical, between-subjects study was run with participants either receiving vibrotactile feedback via an HMP (haptics condition) or no vibrotactile feedback (control) while listening to music. Results indicated that listening to music while receiving vibrotactile feedback increased participants’ sense of empathy, parasocial bond, and loyalty towards the artist, while also decreasing participants’ feelings of loneliness. The connection between haptics condition and these dependent variables was mediated by the feeling of social presence. Study 2 thus provides initial evidence that HMPs may be used to meet people’s need for connection, multisensory immersion, and complex forms of touch in VR concerts as identified in Study 1.

Design and Evaluation of CPR Emergency Equipment for Non-Professionals.

Sudden cardiac death is a sudden and highly fatal condition. Implementing high-quality emergency cardiopulmonary resuscitation (CPR) early on is an effective rescue method for this disease. However, the rescue steps of CPR are complicated and difficult to remember, and the quantitative indicators are difficult to control, which leads to a poor quality of CPR emergency actions outside the hospital setting. Therefore, we have developed CPR emergency equipment with a multisensory feedback function, aiming to guide rescuers in performing CPR through visual, auditory, and tactile interaction. This equipment consists of three components: first aid clothing, an audio-visual integrated terminal, and a vital sign detector. These three components are based on a micro-power WiFi-Mesh network, enabling the long-term wireless transmission of the multisensor data. To evaluate the impact of the multisensory feedback CPR emergency equipment on nonprofessionals, we conducted a controlled experiment involving 32 nonmedical subjects. Each subject was assigned to either the experimental group, which used the equipment, or the control group, which did not. The main evaluation criteria were the chest compression (CC) depth, the CC rate, the precise depth of the CC ratio (5-6 cm), and the precise rate of the CC ratio -(100-120 times/min). The results indicated that the average CC depth in the experimental group was 51.5 ± 1.3 mm, which was significantly better than that of the control group (50.2 ± 2.2 mm, p = 0.012). Moreover, the average CC rate in the experimental group (110.1 ± 6.2 times/min) was significantly higher than that of the control group (100.4 ± 6.6 times/min) (p < 0.001). Compared to the control group (66.37%), the experimental group showed a higher proportion of precise CC depth (82.11%), which is closer to the standard CPR rate of 100%. In addition, the CC ratio of the precise rate was 93.75% in the experimental group, which was significantly better than that of 56.52% in the control group (p = 0.024). Following the experiment, the revised System Availability Scale (SUS) was utilized to evaluate the equipment's usability. The average total SUS score was 78.594, indicating that the equipment's acceptability range was evaluated as 'acceptable', and the overall adjective rating was 'good'. In conclusion, the multisensory feedback CPR emergency equipment significantly enhances the CC performance (CC depth, CC rate, the precise depth of CC ratio, the precise rate of CC ratio) of nonprofessionals during CPR, and the majority of participants perceive the equipment as being easy to use.

Sonification as a reliable alternative to conventional visual surgical navigation

Despite the undeniable advantages of image-guided surgical assistance systems in terms of accuracy, such systems have not yet fully met surgeons’ needs or expectations regarding usability, time efficiency, and their integration into the surgical workflow. On the other hand, perceptual studies have shown that presenting independent but causally correlated information via multimodal feedback involving different sensory modalities can improve task performance. This article investigates an alternative method for computer-assisted surgical navigation, introduces a novel four-DOF sonification methodology for navigated pedicle screw placement, and discusses advanced solutions based on multisensory feedback. The proposed method comprises a novel four-DOF sonification solution for alignment tasks in four degrees of freedom based on frequency modulation synthesis. We compared the resulting accuracy and execution time of the proposed sonification method with visual navigation, which is currently considered the state of the art. We conducted a phantom study in which 17 surgeons executed the pedicle screw placement task in the lumbar spine, guided by either the proposed sonification-based or the traditional visual navigation method. The results demonstrated that the proposed method is as accurate as the state of the art while decreasing the surgeon’s need to focus on visual navigation displays instead of the natural focus on surgical tools and targeted anatomy during task execution.

Visuo-Tactile Feedback-Based Robot Manipulation for Object Packing

Robots are increasingly expected to manipulate objects, of which properties have high perceptual uncertainty from any single sensory modality. This directly impacts successful object manipulation. Object packing is one of the challenging tasks in robot manipulation. In this work, a new visuo-tactile feedback-based manipulation planning framework for object packing is proposed, which makes use of the on-the-fly multisensory feedback and an attention-guided deep affordance model as perceptual states as well as a deep reinforcement learning (DRL) pipeline. Significantly, multiple sensory modalities, vision and touch [tactile and force/torque (F/T)], are employed in predicting and indicating the manipulable regions of multiple affordances (i.e., graspability and pushability) for objects with similar appearances but different intrinsic properties (e.g., mass distribution). To improve the manipulation efficiency, the DRL algorithm is trained to select the optimal actions for successful object manipulation. The proposed method is evaluated on both an open dataset and our collected dataset and demonstrated in the use case of the object packing task. The results show that the proposed method outperforms the existing methods and achieves better accuracy with much higher efficiency.

High Speed Emulation in a Vehicle-in-the-Loop Driving Simulator

Rendering accurate multisensory feedback is critical to ensure natural user behavior in driving simulators. In this work, we present a virtual reality (VR)-based Vehicle-in-the-Loop (ViL) simulator that provides visual, vestibular, and haptic feedback to drivers in high speed driving conditions. Designing our simulator around a four-wheel steer-by-wire vehicle enables us to emulate the dynamics of a vehicle traveling significantly faster than the test vehicle and to transmit corresponding haptic steering feedback to the driver. By scaling the speed of the test vehicle through a combination of VR visuals, vehicle dynamics emulation, and steering wheel force feedback, we can safely and immersively run experiments up to highway speeds within a limited driving space. In double lane change and highway weaving experiments, our high speed emulation method tracks yaw motion within human perception limits and provides sensory feedback comparable to the same maneuvers driven manually.

Multisensory Cues for Gait Rehabilitation with Smart Glasses: Methodology, Design, and Results of a Preliminary Pilot.

Recent advances in mobile technology have shown that augmented unisensory feedback can be leveraged to improve gait using wearable systems, but less is known about the possible benefits and usability of multisensory (i.e., multimodal) feedback. This paper introduces the preliminary results of an innovative research project aiming to develop an mHealth system including Android smart glasses, and providing multisensory cues for gait rehabilitation of people affected by Parkinson's disease in and out of the medical context. In particular, the paper describes a preliminary pilot focusing on the design of visual, auditory, and haptic cues, and testing the design methodologies to be used in further developments of the project. Considered research questions were: Which kinds of images, sounds, and vibrations mostly influence gait speed, stride length, and cadence? Which are the ones stressing the user the least? Which ones induce the most immediate reaction? Thus, in this starting part of the research project, different typologies of sensory cues were designed, tested, and evaluated considering quantitative and qualitative parameters to properly answer the research questions.

Multi-sensory feedback improves spatially compatible sensori-motor responses

To interact with machines, from computers to cars, we need to monitor multiple sensory stimuli, and respond to them with specific motor actions. It has been shown that our ability to react to a sensory stimulus is dependent on both the stimulus modality, as well as the spatial compatibility of the stimulus and the required response. However, the compatibility effects have been examined for sensory modalities individually, and rarely for scenarios requiring individuals to choose from multiple actions. Here, we compared response time of participants when they had to choose one of several spatially distinct, but compatible, responses to visual, tactile or simultaneous visual and tactile stimuli. We observed that the presence of both tactile and visual stimuli consistently improved the response time relative to when either stimulus was presented alone. While we did not observe a difference in response times of visual and tactile stimuli, the spatial stimulus localization was observed to be faster for visual stimuli compared to tactile stimuli.

Neurophysiological mechanisms of gait disturbance in advanced Parkinson's disease patients

AbstractThis review considers pathophysiological mechanisms of posture‐gait disturbances in Parkinson's disease (PD). Clinical studies have shown that posture‐gait disturbance attributes to the dysfunction of the whole neuraxis in addition to the musculoskeletal system. The cerebral cortex, basal ganglia (BG), cerebellum, brainstem, and spinal cord temporally and spatially integrate and coordinate multisensory feedback and efferent copies of the motor command. Therefore, the extensive repertoire of voluntary movements can be coupled with anticipatory and reactive postural adjustments to provide the framework for supporting and stabilizing the goal‐directed gait activity. Redundancies in the system allow adaptation and compensation through reward‐oriented and error‐based learning processes implemented through the BG and cerebellar pathways, respectively. However, the impairment of these systems in PD may considerably compromise the capacity to adapt and lead to maladaptive changes impairing posture‐gait control. When these impairments occur, the risk of falls can significantly increase, and interventions are required to reduce morbidity. The damage in dopamine (DA) neurons is the primary cause of PD. Insufficient DA supply in the striatum disturbs the operation of intrinsic networks in the BG. This also pathologically increases GABAergic BG output to the cerebral cortex and brainstem, resulting in functional disconnection of other structures from the BG. The disconnection makes PD patients disable to achieve habitually acquired automatized gait control. Moreover, Lewy body degeneration in most brain areas, particularly in cholinergic and other monoaminergic systems vulnerable to neurodegeneration, further disturbs posture‐gait control and alters non‐motor symptoms of PD.

Vibrotactile enhancement in hand rehabilitation has a reinforcing effect on sensorimotor brain activities.

ObjectiveStroke patients often suffer from hand dysfunction or loss of tactile perception, which in turn interferes with hand rehabilitation. Tactile-enhanced multi-sensory feedback rehabilitation is an approach worth considering, but its effectiveness has not been well studied. By using functional near-infrared spectroscopy (fNIRS) to analyze the causal activity patterns in the sensorimotor cortex, the present study aims to investigate the cortical hemodynamic effects of hand rehabilitation training when tactile stimulation is applied, and to provide a basis for rehabilitation program development.MethodsA vibrotactile enhanced pneumatically actuated hand rehabilitation device was tested on the less-preferred hand of 14 healthy right-handed subjects. The training tasks consisted of move hand and observe video (MO), move hand and vibration stimulation (MV), move hand, observe video, and vibration stimulation (MOV), and a contrast resting task. Region of interest (ROI), a laterality index (LI), and causal brain network analysis methods were used to explore the brain’s cortical blood flow response to a multi-sensory feedback rehabilitation task from multiple perspectives.Results(1) A more pronounced contralateral activation in the right-brain region occurred under the MOV stimulation. Rehabilitation tasks containing vibrotactile enhancement (MV and MOV) had significantly more oxyhemoglobin than the MO task at 5 s after the task starts, indicating faster contralateral activation in sensorimotor brain regions. (2) Five significant lateralized channel connections were generated under the MV and MOV tasks (p < 0.05), one significant lateralized channel connection was generated by the MO task, and the Rest were not, showing that MV and MOV caused stronger lateralization activation. (3) We investigated all thresholds of granger causality (GC) resulting in consistent relative numbers of effect connections. MV elicited stronger causal interactions between the left and right cerebral hemispheres, and at the GC threshold of 0.4, there were 13 causal network connection pairs for MV, 7 for MO, and 9 for MOV.ConclusionVibrotactile cutaneous stimulation as a tactile enhancement can produce a stronger stimulation of the brain’s sensorimotor brain areas, promoting the establishment of neural pathways, and causing a richer effect between the left and right cerebral hemispheres. The combination of kinesthetic, vibrotactile, and visual stimulation can achieve a more prominent training efficiency from the perspective of functional cerebral hemodynamics.

Haptically enhanced VR surgical training system

Abstract This paper proposes a cost-effective VR surgical training system which computes haptic feedback forces when a VR surgical tool interacts with virtual tissues. A 3 degrees of freedom (DoF) reverse linear Delta mechanism is used to render computed force feedback data which are then received by the fingertip of the operator. Additionally, the moving plate allows rendering of surface properties and lateral forces occurring due to a tumor with different stiffness parameters below the skin surface. Controllers are designed and implemented to regulate the haptic feedback device’s end-effector position and applied force. The virtual surgical instruments are controlled by a 7DoF serial link manipulator which captures the operator’s movement by the utilization of various sensors. The controllers to regulate forces as well as the positions are evaluated with the proposed haptic feedback device. The mean RMSE of the force and mean error of the angular displacement are 0.0707 N and 1.95°, respectively. The presented system can provide multisensory feedback including visual, auditory and haptic feedback interactively depending on the operator’s input in the presented VR surgical training system.