Haptic Feedback System Research Articles

Intelligent glove for visually impaired people using haptic feedback system

The primary objective is to bridge the gap between the visually impaired people and society. There are approximately 37 million people across the globe who are blind, over 15 million are from India. People with visual disabilities are often dependent on external assistance. Blind people use Braille language for reading, writing and for communication purpose. Blind people face problems such as reading SMS and replying for it in case of emergency. Braille language is used as the basis of the project. Therefore interfacing Braille pad with the GSM module so that the blind people can have the access to the SMS system is advantageous. Thus we were motivated to develop an experimental system for communication to assist people who are completely blind using Braille pad system. In which GSM module convert text inputs into Braille patterns through vibrations using DC cylindrical motors with PIC microcontroller as the master. Therefore the words corresponding vibration will be intimated through the gloves to the visually impaired people. It will result to overcome the disabilities of visually impaired to understand other’s expressions.

Design Consideration of the Haptic Feedback System for Robotic SuperLimbs

Design Consideration of the Haptic Feedback System for Robotic SuperLimbs

Development and Testing of a Wearable Vibrotactile Haptic Feedback System for Proprioceptive Rehabilitation

The human sense of touch is an integral part of daily life. For tasks involving grasping and manipulation of objects, force feedback is a key requirement. Most of the systems give contact point or complete grasping force feedback; for precision grasping and other physical interactions, finger awareness and force feedback from independent fingers is essential. In this study a novel, wearable proprioceptive rehabilitation system is designed which restores the ability of identifying and distinguishing between individual fingers of a prosthetic hand or an exoskeleton in a non-invasive manner. Moreover, it provides different levels of force feedback from every finger as well, which enables the user to distinguish and control force in precision grasping activities. For testing the system accuracy, classical psychophysical methods were used on a group of 14 voluntary disabled subjects. The tests were conducted in both, ideal and real-world conditions i.e. without and with distractions and accuracies were calculated accordingly. A p-test was also conducted to observe significance between the samples of with and without distraction datasets. The system performed with an overall accuracy of 82.04% which was well above the min. performance measure of 60%. Vi-HaB is standalone system and can be mounted on any upper limb rehabilitation (prosthesis, exoskeleton) system for finger awareness and force feedback.

Guiding and Navigation for the Blind using Deep Convolutional Neural Network Based Predictive Object Tracking

Indoor and outdoor Navigation is a tough task for a visually impaired person and they would most of the time require human assistance. Existing solutions to this problem are in the form of smart canes and wearable. Both of which use sensors like on - board proximity and obstacle detection, as well as a haptic or auditory feedback system to warn the user of stationary or incoming obstacles so that they do not collide with any of them as they move. This approach has many drawbacks as it is not yet a stand - alone reliable device for the user to trust when navigating, and when frequently triggered in crowded areas, the feedback system will confuse the user with too many requests resulting in loss of actual information. Our Goal here is to create a Personalized assistant to the user, which they can interact naturally with their voice to mimic the aid of an actual human assistance while they are on the move. It works by using its object detection module with a high reliability training accuracy to detect the boundaries of objects in motion per frame and once the bounding box crosses the threshold accuracy, recognize the object in the box and pass the information to the system core, where it verifies if the information needed to be passed onto the user or not, if yes it passes the converted speech information to the voice interaction model. The voice interaction model is consent-based, it would accept and respond to navigation queries from the user and will intelligently inform them about the obstacle which needs to be avoided. This ensures only the essential information in the form of voice requests is passed onto the user, which they can use to navigate and also interact with the assistant for more information.

Haptic simulation and synthesis of mechanisms

The design and development of hand-actuated mechanisms requires the consideration of ergonomics and human-centred design. Prototypes are used to create mechanisms with high-quality haptics and intuitive interaction. In order to enable fast and easy prototyping at an early development stage, virtual prototyping is desired. In similar tasks haptic feedback systems are used to display virtual systems haptically to the user and let him feel its behaviour. This paper shows the implementation of haptic simulation of mechanisms using haptic feedback systems. Subsequent, haptic synthesis is introduced. This haptic-interactive operation allows the user to modify the virtual prototype of a mechanism during simulation just by pushing and pulling. The concept of haptic synthesis is realised by a combination of real-time simulation of mechanisms, the derivation of desired modifications based on measured forces and an automated optimisation-based mechanism synthesis. For this purpose, the haptic feedback system RePlaLink is presented. Real-time simulation issues are discussed. Online parameter changes are introduced and the implementation of the haptic synthesis is shown.

Sole Mate: Safe Path-Finding by Obstacle Detection and Distance Estimation for the Blind

The world has increased its demand for assistive technology (AT). There are a lot of researches and developments going on with respect to AT. Among the AT devices which are being developed, the need for a reliable and less expensive device which serves as an assistance for a visually challenged person is in serious demand all around the world. We, therefore, intend to provide a solution for this by constructing a device that has the capability to detect the obstacles within a given range for a visually challenged person and alerting the person about the obstacles. This involves various components like a camera for image detection, an ultrasonic distance sensor for distance estimation and a vibration motor which works on the principle of Haptic feedback and rotates with varied intensities depending on how far the obstacle is from the user. This paper presents a model which is a part of the footwear of the user and hence, no additional device is required to hold onto for assistance. The model involves the use of a microcontroller, a camera, to dynamically perceive the obstacles and a haptic feedback system to alert the person about the same. The camera dynamically acquires the real time video footage which is further processed by the microcontroller to detect the obstacles. Simultaneously, one more algorithm is being executed to estimate the distance with the help of an ultrasonic distance sensor. Depending on the distance, the frequency of the vibration motor, which acts as the output for notifying the user about the obstacle, is varied (haptic feedback). With this system, a visually challenged person will be able to avoid the obstacles successfully without the use of any additional device.

Multi-Sensory Stimuli Improve Distinguishability of Cutaneous Haptic Cues.

Wearable haptic systems offer portable, private tactile communication to a human user. To date, advances in wearable haptic devices have typically focused on the optimization of haptic cue transmission using a single modality, or have combined two types of cutaneous feedbacks, each mapped to a particular parameter of the task. Alternatively, researchers have employed arrays of haptic tactile actuators to maximize information throughput to a user. However, when large cue sets are to be transmitted, such as those required to communicate language, perceptual interference between transmitted cues can decrease the efficacy of single-sensory systems, or require large footprints to ensure salient spatiotemporal cues are rendered to the user. In this paper, we present a wearable, multi-sensory haptic feedback system, MISSIVE (Multi-sensory Interface of Stretch, Squeeze, and Integrated Vibration Elements), that conveys multi-sensory haptic cues to the user's upper arm. We present experimental results that demonstrate that rendering haptic cues with multi-sensory components-specifically, lateral skin stretch, radial squeeze, and vibrotactile stimuli-improved perceptual distinguishability in comparison to similar cues with all-vibrotactile components. These results support the incorporation of diverse stimuli, both vibrotactile and nonvibrotactile, for applications requiring large haptic cue sets.

Vibrotactile Feedback Improves Foot Placement Perception on Stairs for Lower-Limb Prosthesis Users.

Lower-limb amputees demonstrate decreased performance in stair ambulation compared to their intact-limb counterparts. An estimated 21% of amputees can navigate stairs without a handrail; almost 33% do not use stairs at all. The absence of tactile sensation on the bottom of the foot, creating uncertainty in foot placement, may be overcome by integrating sensory feedback into prosthesis design. Here we describe the design and evaluation of a haptic feedback system worn on the thigh to provide vibrotactile cues of foot placement with respect to stair steps. Tactor discrimination and foot placement awareness tests were performed to analyze system efficacy. Control participants wearing ski boots (N=10) and below-knee amputees (N=2) could discriminate individual tactor vibrations with 95.4% and 90.1% accuracy, respectively. The use of vibrotactile feedback increased accuracy in reporting foot placement by 15% and 17.5%, respectively. These results suggest that using vibrotactile arrays for sensory feedback may improve stair descent performance in lower-limb amputees.

Biaxial sensing suture breakage warning system for robotic surgery.

The number of procedures performed with robotic surgery may exceed one million globally in 2018. The continual lack of haptic feedback, however, forces surgeons to rely on visual cues in order to avoid breaking sutures due to excessive applied force. To mitigate this problem, the authors developed and validated a novel grasper-integrated system with biaxial shear sensing and haptic feedback to warn the operator prior to anticipated suture breakage. Furthermore, the design enables facile suture manipulation without a degradation in efficacy, as determined via measured tightness of resulting suture knots. Biaxial shear sensors were integrated with a da Vinci robotic surgical system. Novice subjects (n = 17) were instructed to tighten 10 knots, five times with the Haptic Feedback System (HFS) enabled, five times with the system disabled. Seven suture failures occurred in trials with HFS enabled while seventeen occurred in trials without feedback. The biaxial shear sensing system reduced the incidence of suture failure by 59% (p = 0.0371). It also resulted in 25% lower average applied force in comparison to trials without feedback (p = 0.00034), which is relevant because average force was observed to play a role in suture breakage (p = 0.03925). An observed 55% decrease in standard deviation of knot quality when using the HFS also indicates an improvement in consistency when using the feedback system. These results suggest this system may improve outcomes related to knot tying tasks in robotic surgery and reduce instances of suture failure while not degrading the quality of knots produced.

Suture Breakage Warning System for Robotic Surgery.

As robotic surgery has increased in popularity, the lack of haptic feedback has become a growing issue due to the application of excessive forces that may lead to clinical problems such as intraoperative and postoperative suture breakage. Previous suture breakage warning systems have largely depended on visual and/or auditory feedback modalities, which have been shown to increase cognitive load and reduce operator performance. This work catalogues a new sensing technology and haptic feedback system (HFS) that can reduce instances of suture failure without negatively impacting performance outcomes including knot quality. Suture breakage is common in knot-tying as the pulling motion introduces prominent shear forces. A shear sensor mountable on the da Vinci robotic surgical system's Cadiere grasper detects forces that correlate to the suture's internal tension. HFS then provides vibration feedback to the operator as forces near a particular material's failure load. To validate the system, subjects tightened a total of four knots, two with the Haptic Feedback System (HFS) and two without feedback. The number of suture breakages were recorded and knot fidelity was evaluated by measuring knot slippage. Results showed that instances of suture failure were significantly reduced when HFS was enabled (p = 0.0078). Notably, knots tied with HFS also showed improved quality compared to those tied without feedback (p = 0.010). The results highlight the value of HFS in improving robotic procedure outcomes by reducing instances of suture failures, producing better knots, and reducing the need for corrective measures.

Real-Time Trajectory Generation for Haptic Feedback Manipulators in Virtual Cockpit Systems

To obtain real-time interactions in the virtual cockpit system (VCS), a real-time trajectory generation method based on dynamical nonlinear optimization and regression prediction for the haptic feedback manipulator (HFM) is presented in this paper. First, a haptic feedback system based on servoserial manipulator is constructed. Then, the trajectory planning problem for the HFM is formulated as a nonlinear optimization problem to balance the motion time and power consumption and ensure the safety of physical human–robot interactions (pHRI). Multiple optimization problems are solved to generate the optimal database off-line. Finally, the classified multivariate (CM) regression method is presented to learn the database and generate optimal trajectories with arbitrary initial and objective positions on-line. Results show that trajectories with rapidity, safety, and lower power consumption can be generated in real-time by this method, which lay a basis of haptic interactions in the VCS.

A Soft Robotic Wearable Wrist Device for Kinesthetic Haptic Feedback.

Advances in soft robotics provide a unique approach for delivering haptic feedback to a user by a soft wearable device. Such devices can apply forces directly on the human joints, while still maintaining the safety and flexibility necessary for use in close proximity to the human body. To take advantage of these properties, we present a new haptic wrist device using pressure-driven soft actuators called reverse pneumatic artificial muscles (rPAMs) mounted on four sides of the wrist. These actuators are originally pre-strained and release compressive stress under pressure, applying a safe torque around the wrist joints while being compact and portable, representing the first soft haptic device capable of real-time feedback. To demonstrate the functional utility of this device, we created a virtual path-following task, wherein the user employs the motion of their wrist to control their embodied agent. We used the haptic wrist device to assist the user in following the path and study their performance with and without haptic feedback in multiple scenarios. Our results quantify the effect of wearable soft robotic haptic feedback on user performance. Specifically, we observed that our haptic feedback system improved the performance of users following complicated paths in a statistically significant manner, but did not show improvement for simple linear paths. Based on our findings, we anticipate broader applications of wearable soft robotic haptic devices toward intuitive user interactions with robots, computers, and other users.

Workspace analysis for haptic feedback manipulator in virtual cockpit system

To obtain natural space experience of haptic interaction for users in virtual cockpit systems (VCS), a haptic feedback system and a workspace analysis framework for haptic feedback manipulator (HFM) are presented in this paper. Firstly, improving the classical three-dimensional workspace obtained by the Monte Carlo method, a novel workspace representation method, oriented workspace, is presented, which can indicate both the position and the orientation of the end-effector. Then, aimed at the characters of HFMs, the oriented workspace is divided into the effective workspace and the prohibited area by extracting the control panel area. At last, the effective workspace volume and the control panel area are calculated by the double-directed extremum method, with the accuracy improved by repeatedly adding and extracting boundary points. By simulation, the area in which interactions between the manipulator and users hand performed is determined and accordingly the effective workspace along with its boundary and volume are obtained in a relative high precision, which lay a basis for haptic interaction in VCS.

Sinyal Elektrik untuk Memperkaya Pengendalian Robot Jarak Jauh

A remote control operator enables a robot to perform in a hazard or area which can not be reached by humans. To control the robot, several researchers have equipped a robot with a system which can give others types of feedback information, besides of visual feedback from the robot. One of the feedback types is haptic feedback. The aim of this feedback is to make the operator become immersed with the robot. The existing researchers are using electro-mechanics system. However, these systems are complex, bulky, and hence prevent a seamless embodiment between an operator’s body. The objective of this research is to develop a haptic feedback system combined with stereo vision feedback which compact, versatile and easy to fit. This system is tested to accomplish the task using mobile robot and robot arm. The result shows that this system can help the operator to control robot better .

Micropipette aspiration of the Pacinian corpuscle.

The Pacinian corpuscle (PC) is a cutaneous mechanoreceptor sensitive to high-frequency vibrations (20-1000Hz). The PC is of importance due to its integral role in somatosensation and the critical need to understand PC function for haptic feedback system development. Previous theoretical and computational studies have modeled the physiological response of the PC to sustained or vibrating mechanical stimuli, but they have used estimates of the receptor's mechanical properties, which remain largely unmeasured. In this study, we used micropipette aspiration (MPA) to determine an apparent Young's modulus for PCs isolated from a cadaveric human hand. MPA was applied in increments of 5mm H2O (49Pa), and the change in protrusion length of the PC into the pipette was recorded. The protrusion length vs. suction pressure data were used to calculate the apparent Young's modulus. Using 10 PCs with long-axis lengths of 2.99±0.41mm and short-axis lengths of 1.45±0.22mm, we calculated a Young's modulus of 1.40±0.86kPa. Our measurement is on the same order of magnitude as those approximated in previous models, which estimated the PC to be on the same order of magnitude as skin or isolated cells, so we recommend that a modulus in the kPa range be used in future studies.

Identifying the effects of using integrated haptic feedback for gait rehabilitation of stroke patients.

This paper evaluates the prospects of using a novel Integrated Haptic Feedback (IHF) system. IHF can provide over-ground gait training regimens for post-stroke ambulatory subjects. IHF system combines the use of a portable cane for kinesthetic feedback and a wearable vibrotactor array for tactile feedback. Continuous somatosensory input is aiforded to the users at the handle of cane; it serves the purpose of balance assurance at higher gait speeds. Besides, restricted use of upper limb for weight-bearing inspires the users to involve the paretic lower limbs more actively. Furthermore, tactile feedback contributes in enhancing the gait symmetry through afferent signal of vibration. Six post-stroke ambulatory individuals participated in walking trials to identity the effects of IHF system. Results indicate that while walking faster patients' body sway was not disturbed. Statistically significant increase was observed in temporal stance symmetry (p-value=0.02) and in paretic muscle (vastus medialis obliquus and semitendinosus) activation during stance phase (p-value<0.01). The IHF system can be a valuable tool to assist physical therapist in gait rehabilitation of post-stroke individuals.

Development of a Robotic Colonoscopic Manipulation System, Using Haptic Feedback Algorithm.

PurposeColonoscopy is one of the most effective diagnostic and therapeutic tools for colorectal diseases. We aim to propose a master-slave robotic colonoscopy that is controllable in remote site using conventional colonoscopy.Materials and MethodsThe master and slave robot were developed to use conventional flexible colonoscopy. The robotic colonoscopic procedure was performed using a colonoscope training model by one expert endoscopist and two unexperienced engineers. To provide the haptic sensation, the insertion force and the rotating torque were measured and sent to the master robot.ResultsA slave robot was developed to hold the colonoscopy and its knob, and perform insertion, rotation, and two tilting motions of colonoscope. A master robot was designed to teach motions of the slave robot. These measured force and torque were scaled down by one tenth to provide the operator with some reflection force and torque at the haptic device. The haptic sensation and feedback system was successful and helpful to feel the constrained force or torque in colon. The insertion time using robotic system decreased with repeated procedures.ConclusionThis work proposed a robotic approach for colonoscopy using haptic feedback algorithm, and this robotic device would effectively perform colonoscopy with reduced burden and comparable safety for patients in remote site.

The Use of Vibrotactile Feedback to Restore Texture Recognition Capabilities, and the Effect of Subject Training.

This paper presents a vibrotactile haptic feedback system for use under dynamic conditions, verifies its functionality, and shows how results may be affected by the amount of training that subjects receive. We hope that by using vibrotactile feedback to distinguish between different textures, upper-limb amputees may be able to partially regain the sense of touch. During a previous experiment (Motamedi et al., 2015) we noticed a correlation between how familiar the subjects were with haptic systems, and how well they were able to use the haptic system to accurately identify textures. This observation lead us to conduct a second experiment, the results of which are the main focus of this paper. We began with a group of subjects who were completely unfamiliar with haptic systems, and tracked the improvements in their accuracy over a period of four weeks. Although the subjects showed a 16% improvement in their ability to recognize textures, going from a 64% success rate after the first week to 80% after the fourth, perfect accuracy was not attained. A subsequent experiment, however, shows that this result should not diminish our perception of the haptic system's effectiveness. When we asked the same subjects to identify the textures using only their fingertips, we found that even humans cannot distinguish between near-identical textures with complete accuracy. The subjects' overall success rate when using their own hands was 91%, demonstrating that the proposed haptic system is not far from achieving the same texture recognition capabilities as the human sense of touch.

Development and Real World Use of a Vibratory Haptic Feedback System for Upper-Limb Prosthetic Users

ABSTRACT Introduction Sensory feedback is largely unavailable for persons with upper-limb amputation with conventional prostheses. The current study created a portable vibratory haptic feedback system integrated into the prosthesis to test its usefulness in gripping objects during daily life. Materials and Methods Development involved optimizing a mapping algorithm between the force sensor and the feedback tactor vibration, developing a custom process for mounting a wireless force sensor onto the prosthetic thumb with reliable output, and developing an instrumented object for testing grip force accuracy. Clinical testing involved optimizing tactor placement, measuring grip force accuracy, measuring the ability to perform daily gripping tasks, and surveying prosthetic users' opinion of the system after using the vibratory haptic device at home. A total of six individuals with unilateral transradial amputation participated in this study. Results The results demonstrated optimal tactor placement more proximal versus distal on the forearm. Accuracy with haptic versus no haptic feedback demonstrated that haptic feedback improved the grip force accuracy by 129% (adjusted P = 0.041) for light grip force target (2 lb or 0.9 kg) in the nonportable system and 21% (adjusted P = 0.051) for medium grip force target (10 lb or 4.5 kg) in the portable system. Haptic feedback did not statistically improve grip accuracy at strong grip forces (20 lb or 9.1 kg). Haptic feedback improved the gripping technique during the Assessment of Capacity for Myoelectric Control (ACMC) by 1.22 points, although this was not statistically significant (P = 0.27, n = 4). The participants provided specific positive examples of how the vibratory haptic system was useful at home. The overall usability score was 3.6 where 4 indicated excellent and 0 indicated poor. Conclusions This kind of system has potential to improve the lives of upper-limb prosthetic users.

Dexterous Grasping Tasks Generated With an Add-on End Effector of a Haptic Feedback System

The simulation of grasping operations in virtual reality (VR) is required for many applications, especially in the domain of industrial product design, but it is very difficult to achieve without any haptic feedback. Force feedback on the fingers can be provided by a hand exoskeleton, but such a device is very complex, invasive, and costly. In this paper, we present a new device, called HaptiHand, which provides position and force input as well as haptic output for four fingers in a noninvasive way, and is mounted on a standard force-feedback arm. The device incorporates four independent modules, one for each finger, inside an ergonomic shape, allowing the user to generate a wide range of virtual hand configurations to grasp naturally an object. It is also possible to reconfigure the virtual finger positions when holding an object. The paper explains how the device is used to control a virtual hand in order to perform dexterous grasping operations. The structure of the HaptiHand is described through the major technical solutions required and tests of key functions serve as validation process for some key requirements. Also, an effective grasping task illustrates some capabilities of the HaptiHand.