Kinesthetic Force Research Articles

3D Hand Motion Generation for VR Interactions Using a Haptic Data Glove

Recently, VR-based training applications have become popular and promising, as they can simulate real-world situations in a safe, repeatable, and cost-effective way. For immersive simulations, various input devices have been designed and proposed to increase the effectiveness of training. In this study, we developed a novel device that generates 3D hand motion data and provides haptic force feedback for VR interactions. The proposed device can track 3D hand positions using a combination of the global position estimation of ultrasonic sensors and the hand pose estimation of inertial sensors in real time. For haptic feedback, shape–memory alloy (SMA) actuators were designed to provide kinesthetic forces and an efficient power control without an overheat problem. Our device improves upon the shortcomings of existing commercial devices in tracking and haptic capabilities such that it can track global 3D positions and estimate hand poses in a VR space without using an external suit or tracker. For better flexibility in handling and feeling physical objects compared to exoskeleton-based devices, we introduced an SMA-based actuator to control haptic forces. Overall, our device was designed and implemented as a lighter and less bulky glove which provides comparable accuracy and performance in generating 3D hand motion data for a VR training application (i.e., the use of a fire extinguisher), as demonstrated in the experimental results.

Challenges of controlling the rotation of virtual objects with variable grip using force-feedback gloves

Some virtual reality (VR) applications require true-to-life object manipulation, such as for training or teleoperation. We investigate an interaction technique that replicates the variable grip strength applied to a held object when using force-feedback gloves in VR. We map the exerted finger pressure to the rotational freedom of the virtual object. With a firm grip, the object’s orientation is fixed to the hand. With a loose grip, the user can allow the object to rotate freely within the hand. A user study (N = 21) showed how challenging it was for participants to control the object’s rotation with our prototype employing the SenseGlove DK1. Despite high action fidelity, the grip variability led to poorer performance and increased task load compared to the default fixed rotation. We suspect low haptic fidelity as an explanation as only kinesthetic forces but no cutaneous cues are rendered. We discuss the system design limitations and how to overcome them in future haptic interfaces for physics-based multi-finger object manipulation.

Variable Stiffness Devices Using Fiber Jamming for Application in Soft Robotics and Wearable Haptics.

Variable stiffness actuation has applications in a wide range of fields, including wearable haptics, soft robots, and minimally invasive surgical devices. There have been numerous design approaches to control and tune stiffness and rigidity; however, most have relatively low specific load-carrying capacities (especially for flexural loads) in the most rigid state that restricts their use in small or slender devices. In this article, we present an approach to the design of slender, high flexural stiffness modules based on the principle of fiber jamming. The proposed fiber jamming modules (FJMs) consist of axially packed fibers in an airtight envelope that transition from a flexible to a rigid beam when a vacuum is created inside the envelope. This FJM can provide the flexural stiffness of up to eight times that of a particle jamming module in the rigid state. Unlike layer jamming modules, the design of FJMs further allows them to control stiffness while bending in space. We present an analytical model to guide the parameter choices for the design of fiber jamming devices. Finally, we demonstrate applications of FJMs, including as a versatile tool, as part of a kinesthetic force feedback haptic glove and as a programmable structure.

Visual Feedback Weakens the Augmentation of Perceived Stiffness by Artificial Skin Stretch.

Tactile stimulation devices are gaining popularity in haptic science and technology-they are lightweight, low-cost, can be wearable, and do not suffer from instability during closed loop interactions with users. Applying tactile stimulation, by means of stretching the fingerpad skin concurrently with kinesthetic force feedback, has been shown to augment the perceived stiffness during interactions with elastic objects. However, to date, the perceptual augmentation due to artificial skin-stretch was studied in the absence of visual feedback. In this article, we tested whether this perceptual augmentation is robust when the stretch is applied in combination with visual displacement feedback. We used a forced-choice stiffness discrimination task with four conditions: force feedback, force feedback with skin-stretch, force and visual feedback, and force and visual feedback with skin-stretch. We found that the visual feedback weakens, but does not eliminate, the skin-stretch induced perceptual effect. Additionally, no effect of visual feedback on the discrimination precision was found.

Conditioned haptic perception for 3D localization of nodules in soft tissue palpation with a variable stiffness probe.

This paper provides a solution for fast haptic information gain during soft tissue palpation using a Variable Lever Mechanism (VLM) probe. More specifically, we investigate the impact of stiffness variation of the probe to condition likelihood functions of the kinesthetic force and tactile sensors measurements during a palpation task for two sweeping directions. Using knowledge obtained from past probing trials or Finite Element (FE) simulations, we implemented this likelihood conditioning in an autonomous palpation control strategy. Based on a recursive Bayesian inferencing framework, this new control strategy adapts the sweeping direction and the stiffness of the probe to detect abnormal stiff inclusions in soft tissues. This original control strategy for compliant palpation probes shows a sub-millimeter accuracy for the 3D localization of the nodules in a soft tissue phantom as well as a 100% reliability detecting the existence of nodules in a soft phantom.

A Modular Wearable Finger Interface for Cutaneous and Kinesthetic Interaction: Control and Evaluation

In this paper, we present a novel modular wearable interface for haptic interaction and robotic teleoperation. It is composed of a 3-degree-of-freedom (3-DoF) fingertip cutaneous device and a 1-DoF finger kinesthetic exoskeleton, which can be either used together as a single device or separately as two different devices. The 3-DoF fingertip device is composed of a static body and a mobile platform. The mobile platform is capable of making and breaking contact with the finger pulp and reangle to replicate contacts with arbitrarily oriented surfaces. The 1-DoF finger exoskeleton provides kinesthetic force to the proximal and distal interphalangeal finger articulations using one servo motor grounded on the proximal phalanx. This paper presents the wearable device as well as three different position, force, and compliance control schemes, together with their evaluations. We also present three human subjects experiments, enrolling a total of 40 different participants: the first experiment considered a curvature discrimination task, the second one a robot-assisted palpation task, and the third one an immersive experience in virtual reality. Results show that providing cutaneous and kinesthetic feedback through our device significantly improve the performance of all the considered tasks. Moreover, although cutaneous-only feedback shows promising performance, adding kinesthetic feedback improves most metrics. Finally, subjects rank our device as highly wearable, comfortable, and effective.

The Treadport: Natural Gait on a Treadmill

To evaluate the differences between walking on an advanced robotic locomotion interface called the Treadport and walking overground with healthy subjects. Previous studies have compared treadmill-based and overground walking in terms of gait parameters. The Treadport's unique features including self-selected speed capability, large belt, kinesthetic force feedback, and virtual reality environment distinguish it from other locomotion interfaces and could provide a natural walking experience for the users. Young, healthy subjects (N = 17) walked 10 meters 10 times each for both overground and the Treadport environments. Comparison between walking conditions used spatiotemporal and kinematic parameters. In addition, electromyographic data was collected for five of the 17 subjects to compare muscle activity between the two conditions. Gait on the Treadport was found to have no significant differences (p > .05) with overground walking in terms of hip and knee joint angles, cadence and stride length and stride speed, and muscle activation of the four muscle groups measured. Differences (p < .05) were observed in ankle dorsiflexion which was reduced by 2.47 ± 0.01 degrees on the Treadport. Walking overground and on the Treadport is highly correlated and not significantly different in 13 of 14 parameters. This study suggests that the Treadport creates an environment for natural walking experience, where natural gait of users is almost preserved, with great potential to be useful for other applications, such as gait rehabilitation of individuals with walking impairments.

Comparison Between Force-Controlled Skin Deformation Feedback and Hand-Grounded Kinesthetic Force Feedback for Sensory Substitution

Teleoperation and virtual reality systems benefit from force sensory substitution when kinesthetic force feedback devices are infeasible due to stability or workspace limitations. We compared the performance of sensory substitution when it is provided through a cutaneous method (skin deformation feedback) and a kinesthetic method (hand-grounded force feedback). For skin deformation feedback, we used a new force-controlled tactile sensory substitution device with the ability to provide tangential and normal force directly to the finger pad. Three-axis force control with 15 Hz bandwidth was achieved using a delta mechanism and three-axis force sensor. For hand-grounded force feedback, forces were grounded against the palm. As a control, world-grounded force feedback was provided using a three-degree-of-freedom kinesthetic force feedback device. Study participants were able to match a reference world-grounded force better with hand-grounded kinesthetic force feedback than with skin deformation feedback. Participants were also able to apply more accurate and precise forces with hand-grounded kinesthetic force feedback than with skin deformation feedback. Conversely, skin deformation feedback resulted in the lowest error during initial force adjustment. These experiments demonstrate relative advantages and disadvantages of skin deformation and hand-grounded kinesthetic force feedback for force sensory substitution.

Three-Dimensional Skin Deformation as Force Substitution: Wearable Device Design and Performance During Haptic Exploration of Virtual Environments.

Virtual reality systems would benefit from a compelling force sensory substitute when workspace or stability limitations prevent the use of kinesthetic force feedback systems. We present a wearable fingertip haptic device with the ability to make and break contact in addition to rendering both shear and normal skin deformation to the fingerpad. A delta mechanism with novel bias spring and tether actuator relocation method enables the use of high-end motors and encoders, allowing precise device control: 10 Hz bandwidth and 0.255 mm RMS tracking error were achieved during testing. In the first of two experiments, participants determined the orientation of a stiff region in a surrounding compliant virtual surface with an average angular error of 7.6 degree, similar to that found in previous studies using traditional force feedback. In the second experiment, we evaluated participants' ability to interpret differences in friction. The Just Noticeable Difference (JND) of surface friction coefficient discrimination using our skin deformation device was 0.20, corresponding with a reference friction coefficient of 0.5. While higher than that found using kinesthetic feedback, this demonstrates that users can perceive differences in surface friction without world-grounded kinesthetic forces. These experiments show that three DoF skin deformation enables both stiffness and friction discrimination capability in the absence of kinesthetic force feedback.

An autoclavable wireless palpation instrument for minimally invasive surgery.

Minimally invasive surgery prevents surgeons from manually palpating organs to locate subsurface tumors and other structures. One solution is to use ultrasound; however, it is not always reliable. Various minimally invasive surgery instruments that provide tactile feedback have been proposed to augment ultrasound sensing for tumor localization; however, current designs have limitations such as cumbersome wiring, difficulty in manipulation, lack of sterilizability and high cost. This paper presents a novel, autoclavable, wireless, hand-held palpation instrument that uses a custom, low-cost, disposable tactile sensor to provide tactile and kinesthetic force feedback. The use of a replaceable, disposable tactile sensor avoids deterioration in sensor performance due to repeated autoclaving. The instrument features a passive joint in the end effector that allows the sensor to self-align to the palpation surface in a wide range of orientations. All of the electronics are packaged in a removable module that allows the rest of the instrument to be easily cleaned and autoclaved. Two versions of the tactile sensor, using piezoresistive sensing and capacitive sensing respectively, have been designed for use with this instrument. The instrument is shown to be able to detect 6 mm diameter spherical tumors at a depth of 9-10 mm in ex vivo tissue samples.

Tactor-Induced Skin Stretch as a Sensory Substitution Method in Teleoperated Palpation

When we use a tool to explore or manipulate an object, friction between the surface of the tool and the fingerpads generates skin stretch cues that are related to the interaction forces between the tool and the object. In this study, we emulate these naturally occurring skin stretch cues in order to convey force direction and magnitude information to users during teleoperation. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. In this study, ten participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using five feedback conditions: skin stretch, force, reduced gain force, graphic, and vibration. When participants received skin stretch feedback, they localized the stiff region as well as with force feedback, with no increase in task completion time. Additionally, participants receiving skin-stretch feedback localized the stiff region statistically significantly more accurately than those using vibration feedback. Although participants using skin stretch exhibited higher interaction forces than when using force, vibration, and graphical feedback, skin stretch statistically significantly decreased interaction forces compared with reduced gain force feedback. Thus, skin-stretch feedback is a compelling substitute for force feedback and may be useful in scenarios where force feedback is reduced or infeasible.

Kinesthetic Force Feedback and Belt Control for the Treadport Locomotion Interface.

This paper describes an improved control system for the Treadport immersive locomotion interface, with results that generalize to any treadmill that utilizes an actuated tether to enable self-selected walking speed. A new belt controller is implemented to regulate the user's position; when combined with the user's own volition, this controller also enables the user to naturally self-select their walking speed as they would when walking over ground. A new kinesthetic-force-feedback controller is designed for the tether that applies forces to the user's torso. This new controller is derived based on maintaining the user's sense of balance during belt acceleration, rather than by rendering an inertial force as was done in our prior work. Based on the results of a human-subjects study, the improvements in both controllers significantly contribute to an improved perception of realistic walking on the Treadport. The improved control system uses intuitive dynamic-system and anatomical parameters and requires no ad hoc gain tuning. The control system simply requires three measurements to be made for a given user: the user's mass, the user's height, and the height of the tether attachment point on the user's torso.

Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback.

During tool-mediated interaction with everyday objects, we experience kinesthetic forces and tactile sensations in the form of vibration and skin deformation at the fingerpad. Fingerpad skin deformation is caused by forces applied tangentially and normally to the fingerpad skin, resulting in tangential and normal skin displacement. We designed a device to convey 3-degree-of-freedom (DoF) force information to the user via skin deformation, and conducted two experiments to determine the devices effectiveness for force-feedback substitution and augmentation. For sensory substitution, participants used 1-DoF and 3-DoF skin deformation feedback to locate a feature in a 3-DoF virtual environment. Participants showed improved precision and shorter completion time when using 3-DoF compared to 1-DoF skin deformation feedback. For sensory augmentation, participants traced a path in space from an initial to a target location, while under guidance from force and/or skin deformation feedback. When force feedback was augmented with skin deformation, participants reduced their path-following error over the cases when force or skin deformation feedback are used separately. We conclude that 3-DoF skin deformation feedback is effective in substituting or augmenting force feedback. Such substitution or augmentation could be used when force feedback is unattainable or attenuated due to device limitations or system instability.

The Subterranean Language of Jeanine Durning's inging and Monique Jenkinson's Instrument

Two recent choreographic works—Jeanine Durning's inging and Monique Jenkinson's Instrument—employ words not as content but as kinesthetic forces. In each, as audiences sit with their own embodied thoughts, words recede and silent questions arise to connect us in the space and time of performance.

Improving Transparency in Teleoperation by Means of Cutaneous Tactile Force Feedback

A study on the role of cutaneous and kinesthetic force feedback in teleoperation is presented. Cutaneous cues provide less transparency than kinesthetic force, but they do not affect the stability of the teleoperation system. On the other hand, kinesthesia provides a compelling illusion of telepresence but affects the stability of the haptic loop. However, when employing common grounded haptic interfaces, it is not possible to independently control the cutaneous and kinesthetic components of the interaction. For this reason, many control techniques ensure a stable interaction by scaling down both kinesthetic and cutaneous force feedback, even though acting on the cutaneous channel is not necessary.We discuss here the feasibility of a novel approach. It aims at improving the realism of the haptic rendering, while preserving its stability, by modulating cutaneous force to compensate for a lack of kinesthesia. We carried out two teleoperation experiments, evaluating (1) the role of cutaneous stimuli when reducing kinesthesia and (2) the extent to which an overactuation of the cutaneous channel can fully compensate for a lack of kinesthetic force feedback. Results showed that, to some extent, it is possible to compensate for a lack of kinesthesia with the aforementioned technique, without significant performance degradation. Moreover, users showed a high comfort level in using the proposed system.

MR 햅틱 큐를 이용한 차량 기어변속 보조장치의 성능평가

This paper proposes a driver supportive device with haptic cue function which can be applicable for vehicles adopting manual transmission system to transmit gear-shifting information to a driver by kinesthetic forces. This haptic cue function is implemented on accelerator pedal by utilizing magnetorheological(MR) fluid and clutch mechanism. In order to achieve this goal, an MR clutch mechanism is devised to be capable of rotary motion of accelerator pedal. The proposed MR clutch is then optimally designed and manufactured under consideration of spatial limitation of vehicles. After transmission torque is experimentally evaluated according to field intensity. The manufactured MR clutch is integrated with accelerator pedal and electric motor to establish the haptic cue device. Control performances are experimentally evaluated via a simple feed-forward control algorithm.

Virtual prototyping of mechanical systems with tool mediated haptic feedback

Haptic feedback usually involves two types of stimulation forces: forces that address the touch sense and forces that address the kinesthetic perception. Touch forces have a low intensity and a complex structure since they reflect contact phenomena where friction plays an important role. Therefore, they are quite difficult to simulate. Virtual prototyping with haptic feedback should ideally involve both types of forces, but the integration of the touch feeling makes the simulator very complex. In this paper, we present a novel concept for virtual prototyping in which the touch interaction is separated from the kinesthetic force feedback. This is possible using a prototype that has a real part undertaking the touch interaction and a virtual part that simulate feedback for the kinesthetic forces. In this way, a full haptic interaction with the virtual prototype is established by means of a device that provides a realistic simulation of the product. In order to illustrate the concept, several experiments have been carried out for the case of specific subsystems of a car, which are particularly involved in the driver---car interaction: steering system, clutch pedal and the gearshift. A user test is described in the last part as well as the conclusions of the research.

Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics.

A novel sensory substitution technique is presented. Kinesthetic and cutaneous force feedback are substituted by cutaneous feedback (CF) only, provided by two wearable devices able to apply forces to the index finger and the thumb, while holding a handle during a teleoperation task. The force pattern, fed back to the user while using the cutaneous devices, is similar, in terms of intensity and area of application, to the cutaneous force pattern applied to the finger pad while interacting with a haptic device providing both cutaneous and kinesthetic force feedback. The pattern generated using the cutaneous devices can be thought as a subtraction between the complete haptic feedback (HF) and the kinesthetic part of it. For this reason, we refer to this approach as sensory subtraction instead of sensory substitution. A needle insertion scenario is considered to validate the approach. The haptic device is connected to a virtual environment simulating a needle insertion task. Experiments show that the perception of inserting a needle using the cutaneous-only force feedback is nearly indistinguishable from the one felt by the user while using both cutaneous and kinesthetic feedback. As most of the sensory substitution approaches, the proposed sensory subtraction technique also has the advantage of not suffering from stability issues of teleoperation systems due, for instance, to communication delays. Moreover, experiments show that the sensory subtraction technique outperforms sensory substitution with more conventional visual feedback (VF).

Force and Contact Location Shading Methods for Use Within Two- and Three-Dimensional Polygonal Environments

Current state-of-the-art haptic interfaces only provide kinesthetic (force) feedback, yet studies have shown that providing tactile feedback in concert with kinesthetic information can dramatically improve a person's ability to dexterously interact with and explore virtual environments. In this research, tactile feedback was provided by a device, called a contact location display (CLD), which is capable of rendering the center of contact to a user. The chief goal of the present work was to develop algorithms that allow the CLD to be used with polygonal geometric models, and to do this without the resulting contact location feedback being overwhelmed by the perception of polygonal edges and vertices. Two haptic shading algorithms were developed to address this issue and successfully extend the use of the CLD to 2D and 3D polygonal environments. Two experiments were run to evaluate these haptic shading algorithms. The first measured perception thresholds for rendering faceted objects as smooth objects. It was found that the addition of contact location feedback significantly increased user sensitivity to edges and that the use of shading algorithms was able to significantly reduce the number of polygons needed for objects to feel smooth. The second experiment explored the CLD device's ability to facilitate exploration and shape recognition within a 3D environment. While this study provided a validation of our 3D algorithm, as people were able to identify the rendered objects with reasonable accuracy, this study underscored the need for improvements in the CLD device design in order to be effectively used in general 3D environments.

Physical Interaction with a Virtual Knee Joint—The 9 DOF Haptic Display of the Munich Knee Joint Simulator

In specific fields, medical education at many universities is rather theoretical and the amount of practical training is limited. A significant improvement can be achieved using virtual reality training stations with lifelike visual, acoustic, tactile, and kinesthetic feedback. Particularly, when simulating procedures that require direct contact with the patient body, a realistic haptic simulation addressing tactile and kinesthetic senses can be essential for the acceptance of virtual simulation stations. A purely passive phantom may provide realistic haptic feedback, but its properties cannot be changed over time. This paper presents the haptic display of the Munich Knee Joint Simulator, which was developed to improve training and education of physical knee joint examinations. The haptic interface comprises a combination of passive phantom segments providing realistic tactile sensations, and strong actuators generating highly dynamic kinesthetic force feedback. A 3 degree of freedom (DOF) manipulator was developed in this study to drive the thigh prosthesis and one 6 DOF industrial robot was used to actuate the shank prosthesis. Both manipulators are driven by hybrid admittance-impedance controllers capable of simulating the complex dynamics of the thigh and the shank. Both actuators are equipped with a 6 DOF force torque sensor and they are virtually coupled by an analytical knee joint model. The proposed setup is capable of simulating a mechanical stiffness as high as 80 kN/m in the translatory DOF and simultaneously allows free motion in the rotatory DOF. Experimental tests of the simulator with orthopedic physicians proved the usability of the proposed concept.