Haptic System Research Articles

AI-Enabled Vibrotactile Feedback-Based Condition Monitoring Framework for Outdoor Mobile Robots

An automated Condition Monitoring (CM) and real-time controlling framework is essential for outdoor mobile robots to ensure the robot’s health and operational safety. This work presents a novel Artificial Intelligence (AI)-enabled CM and vibrotactile haptic-feedback-based real-time control framework suitable for deploying mobile robots in dynamic outdoor environments. It encompasses two sections: developing a 1D Convolutional Neural Network (1D CNN) model for predicting system degradation and terrain flaws threshold classes and a vibrotactile haptic feedback system design enabling a remote operator to control the robot as per predicted class feedback in real-time. As vibration is an indicator of failure, we identified and separated system- and terrain-induced vibration threshold levels suitable for CM of outdoor robots into nine classes, namely Safe, moderately safe system-generated, and moderately safe terrain-induced affected by left, right, and both wheels, as well as severe classes such as unsafe system-generated and unsafe terrain-induced affected by left, right, and both wheels. The vibration-indicated data for each class are modelled based on two sensor data: an Inertial Measurement Unit (IMU) sensor for the change in linear and angular motion and a current sensor for the change in current consumption at each wheel motor. A wearable novel vibrotactile haptic feedback device architecture is presented with left and right vibration modules configured with unique haptic feedback patterns corresponding to each abnormal vibration threshold class. The proposed haptic-feedback-based CM framework and real-time remote controlling are validated with three field case studies using an in-house-developed outdoor robot, resulting in a threshold class prediction accuracy of 91.1% and an effectiveness that, by minimising the traversal through undesired terrain features, is four times better than the usual practice.

A Haptic Feedback System for Spatial Orientation in the Visually Impaired: A Comprehensive Approach

A Haptic Feedback System for Spatial Orientation in the Visually Impaired: A Comprehensive Approach

Walker Integrated with IOT Based Fall Detection, Emergency Response System with Haptic Feedback System for Muscle Relief

In the last decade, the clinical reasoning in physical therapy has been to develop systems for physiotherapists to make clinical decisions rapidly, effectively and efficiently, in response to the increasingly complex needs of health and rehabilitation units. Some studies show the importance of walking aids during rehabilitation from some diseases, and after surgery for arthroplasty in the elderly population, and elderly patients with balance disorders, muscle weakness. Walkers are important devices that aid the rehabilitation process. The use of a walker is recommended for gait changes and imbalance due to various factors, such as surgery of the lower limbs or neurodegenerative changes, especially in the early recovery period. In this project we design a solution for the patient using walkers, this device helps them to maintain the proper balance. The system's foundation lies in a network of strategically placed sensors, including gyroscopes, accelerometers, and pressure sensors, meticulously capturing data on the user's body posture and gait patterns. Employing advanced algorithms, the gathered data is processed and analyzed in real-time to evaluate the user's alignment and identify any deviations from the ideal posture. Furthermore, the system facilitates seamless data recording and analysis, enabling healthcare professionals and physiotherapists to access valuable insights on International Journal of Scientific Research in Engineering and Management (IJSREM) Volume: 07 Issue: 08 | August - 2023 SJIF Rating: 8.176 ISSN: 2582-3930 © 2023, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM25086 | Page 2 the user's progress and performance. The accumulated data assists in developing personalized rehabilitation plans and tracking the user's improvements over time. Index Terms— Microcontroller, LCD, Movement Detection, Accelerometer sensor, ultrasonic sensor, GSM, GPS.

Haptic Feedback Device Using 3D-Printed Flexible, Multilayered Piezoelectric Coating for In-Car Touchscreen Interface.

This study focuses on the development of a piezoelectric device capable of generating feedback vibrations to the user who manipulates it. The objective here is to explore the possibility of developing a haptic system that can replace physical buttons on the tactile screen of in-car systems. The interaction between the user and the developed device allows completing the feedback loop, where the user's action generates an input signal that is translated and outputted by the device, and then detected and interpreted by the user's haptic sensors and brain. An FEM (finite element model) via ANSYS multiphysics software was implemented to optimize the haptic performance of the wafer structure consisting of a BaTiO3 multilayered piezocomposite coated on a PET transparent flexible substrate. Several parameters relating to the geometric and mechanical properties of the wafer, together with those of the electrodes, are demonstrated to have significant impact on the actuation ability of the haptic device. To achieve the desired vibration effect on the human skin, the haptic system must be able to drive displacement beyond the detection threshold (~2 µm) at a frequency range of 100-700 Hz. The most optimized actuation ability is obtained when the ratio of the dimension (radius and thickness) between the piezoelectric coating and the substrate layer is equal to ~0.6. Regarding the simulation results, it is revealed that the presence of the conductive electrodes provokes a decrease in the displacement by approximately 25-30%, as the wafer structure becomes stiffer. To ensure the minimum displacement generated by the haptic device above 2 µm, the piezoelectric coating is screen-printed by two stacked layers, electrically connected in parallel. This architecture is expected to boost the displacement amplitude under the same electric field (denoted E) subjected to the single-layered coating. Accordingly, multilayered design seems to be a good alternative to enhance the haptic performance while keeping moderate values of E so as to prevent any undesired electrical breakdown of the coating. Practical characterizations confirmed that E=20 V/μm is sufficient to generate feedback vibrations (under a maximum input load of 5 N) perceived by the fingertip. This result confirms the reliability of the proposed haptic device, despite discrepancies between the predicted theory and the real measurements. Lastly, a demonstrator comprising piezoelectric buttons together with electronic command and conditioning circuits are successfully developed, offering an efficient way to create multiple sensations for the user. On the basis of empirical data acquired from several trials conducted on 20 subjects, statistical analyses together with relevant numerical indicators were implemented to better assess the performance of the developed haptic device.

Music, Art Installations and Haptic Technology

This paper presents some directions on the design, development and creative use of haptic systems for musical composition, performance and digital art creation. This research has been conducted both from an artistic and a technical point of view and its ambition, over the last decade, apart from the artistic outcome, was to introduce the field of haptics to artistic communities based on an open, do it yourself—DIY ethos. The five directions presented here are not in any sense exhaustive and are based principally on a series of collaborative works and more personal open-ended explorations with the medium of haptics and, more specifically, force-feedback interaction. They will be highlighted along with information about the interaction models and their application to artistic works created by the author and other colleagues. Those directions are (i) Haptic Algorithms and Systems; (ii) Performers Intercoupling; (iii) Haptic Interfaces as Part of the Artistic Practice; (iv) Electromechanical Sound Generation; and (v) Media Art and Art Installations. The interdisciplinary field of musical haptics still has a relatively minor position in the sound and music computing research agendas and, more importantly, its artistic dimension is very rarely discussed. The findings of this research aim to indicate and clarify potential research pathways and offer some results on the use of haptics and force-feedback systems in an artistic context.

Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force

This study presents a novel method to realize an augmented reality (AR) and haptic feedback teleoperation ultrasound system, which can reduce the influence of time delay on the operator. The physician uses a haptic device to send commands to a remote site while observing the AR environment. In the virtual environment, the organ model is projected onto the corresponding position according to the human pose, and the UR robot model can respond to the physician's action in advance. Based on force data, the dynamic environmental model is updated by the recursive least-squares method, allowing real-time force feedback. The contact force is computed from the identified parameters together with the position and velocity of the master site. An experimental platform has been built, where experiments have been conducted to evaluate the overall architecture, highlighting the role of AR and contact-force prediction models, and including clinical validation. Results show that the system can correctly perform remote ultrasound scanning tasks and ensure authenticity and synchronization.

Task-Oriented Prediction and Communication Co-Design for Haptic Communications

Prediction has recently been considered as a promising approach to meet low-latency and high-reliability requirements in long-distance haptic communications. However, most of the existing methods did not take features of tasks and the relationship between prediction and communication into account. In this paper, we propose a task-oriented prediction and communication co-design framework, where the reliability of the system depends on prediction errors and packet losses in communications. The goal is to minimize the required radio resources subject to the low-latency and high-reliability requirements of various tasks. Specifically, we consider the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">just noticeable difference</i> (JND) as a performance metric for the haptic communication system. We collect experiment data from a real-world teleoperation testbed and use <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">time-series generative adversarial networks</i> (TimeGAN) to generate a large amount of synthetic data. This allows us to obtain the relationship between the JND threshold, prediction horizon, and the overall reliability including communication reliability and prediction reliability. We take 5G New Radio as an example to demonstrate the proposed framework and optimize bandwidth allocation and data rates of devices. Our numerical and experimental results show that the proposed framework can reduce wireless resource consumption up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$77.80\%$</tex-math></inline-formula> compared with a task-agnostic benchmark.

A dual robust control architecture with variable stiffness and damping parameters for switching task dominance in collaborative haptic systems

AbstractIn collaborative haptic training systems, a novice operator is interfaced with an expert operator and cooperatively performs some task on a real/virtual environment. Most control architectures for collaborative haptic training systems do not consider the switching task dominance together with investigating overall stability in the presence of nonlinear dynamics and uncertainty. In this paper, a theoretical framework is presented for switching task dominance in collaborative haptic training systems based on supervision and intervention of the expert operator. To that effect, the novice operator performs the operation with as little as possible interference haptic signals in the normal operational conditions. On the other hand, the expert operator is able to intervene the operation to guide the novice operator when it is necessary. The most challenging part of controller design for such systems is to provide the mentioned supervisory framework in a way that the stability of interaction between the operators and the system is ensured with acceptable task performance in various operational conditions. This work offers a variable‐gain dual robust control scheme to address the above problem. The key idea is that the tracking gain of each controller is adjusted in real‐time to switch the task authorities. It is verified that the input‐to‐state stability property is satisfied for each subsystem. Then, the overall stability is proved by leveraging the small gain theorem. Finally, the functionality and performance of the suggested control architecture is demonstrated through simulation and experimental studies.

Emulation Framework for Haptic Data Transmission Using Real-Time Transport Protocol

The Tactile Internet (TI) can be regarded as the next evolution in the world of communication. With its envisioned purpose and potential in shaping up the economy, industry and society, this paradigm aims to bring a new dimension to life by enabling humans to interact with machines remotely and in real-time with haptic and kinesthetic feedback. However, to translate this into reality, Tactile Internet will need to meet the stringent requirements of extremely low latency in conjunction with ultra-high reliability, availability, and security. This poses a challenge on the available communication systems to achieve a round-trip delay within 1 to 10 milliseconds time bound that enables the timely delivery of critical tactile and haptic sensations.&#x0D; This paper aims to evaluate the Real-Time Transport Protocol (RTP) through an emulation framework. It integrates containerization using Linux-based Docker Containers with NS-3 Network Simulator to conceptualize a haptic teleoperation system. The framework is then used to test the protocol’s feasibility for delivering texture haptic data between master and slave domains in accordance with the end-to-end delay requirements specified by IEEE 1918.1 standards. The results have shown that the timely provision of haptic data is achievable by obtaining an average round-trip delay of 17.8493 ms from the emulation experiment. As such, the results satisfy the expected IEEE 1918.1 standards constraints for medium-dynamic environment use cases.

A low-cost robotic hand prosthesis with apparent haptic sense controlled by electroencephalographic signals

One of the biggest problems faced by amputees is obtaining a suitable low-cost prosthesis. To address this problem, the design and implementation of a transradial prosthesis controlled by electroencephalographic (EEG) signals was carried out. This prosthesis is an alternative to prostheses using electromyographic (EMG) signals, which are very complex and exhausting for the patient to execute.We collected EEG signal data using the Emotiv Insight Headset, which were then processed to control the movement of the prosthesis, known as the Zero Arm. Additionally, we incorporated Machine Learning algorithms to classify different types of objects and shapes. The prosthesis also features a haptic feedback system, which simulates the function of mechanoreceptors in the skin, providing the user with a sense of touch when using the prosthesis.Our research has yielded a viable and cost-effective prosthetic limb. We utilized 3D printing and easily obtainable servomotors and controllers, making the prosthesis affordable and accessible.Performance tests of the Zero Arm prosthesis have yielded promising results. The prosthesis demonstrated an average success rate of 86.67% across various tasks, indicating its reliability and effectiveness. Additionally, the prosthesis has an average recognition rate of 70% for different types of objects, a noteworthy accomplishment.

Investigating the efficacy of a tactile feedback system to increase the gait speed of older adults

The objective of this study was to determine (1) if a novel haptic feedback system could increase the walking speed of older adults while it is being employed during overground walking and (2) whether the frequency at which this feedback was presented would have a differential impact on the ability of users to change walking speed while it was present. Given that peak thigh extension has been found to be a biomechanical surrogate for stride length, and consequently gait speed, vibrotactile haptic feedback was provided to the participants' thighs as a cue to increase peak thigh extension while the effect on gait speed was monitored. Ten healthy community-dwelling older adults (68.4 ± 4.1 years) participated. Participants' peak thigh extension, cadence, normalized stride length and velocity, along with their coefficients of variation (COV) were compared across baseline normal and fast walking (with no feedback) and three different frequency of feedback conditions. The findings indicated that, compared to self-selected normal and fast walking speeds, peak thigh extension was significantly increased when feedback was present and after it was withdrawn in a post-test. An increase in thigh extension led to an increase in stride length and, consequently, an increase in stride velocity compared to normal speed. There were no significant differences in the gait parameters as a function of feedback frequency during its application. In conclusion, while present, the haptic feedback system increased thigh extension and walking speed in older adults regardless of the feedback frequency and when the feedback was withdrawn, participants could maintain an increase in those parameters.

Identifying critical transitions and instability in haptic systems

Identifying critical transitions and instability in haptic systems

Unified Human Intention Recognition and Heuristic-Based Trajectory Generation for Haptic Teleoperation of Non-Holonomic Vehicles

In this paper, a novel bilateral shared control approach is proposed to address the issue of strong dependence on the human, and the resulting burden of manipulation, in classical haptic teleoperation systems for vehicle navigation. A Hidden Markov Model (HMM) is utilized to handle the Human Intention Recognition (HIR), according to the force input by the human—including the HMM solution, i.e., Baum–Welch algorithm, and HMM decoding, i.e., Viterbi algorithm—and the communication delay in teleoperation systems is added to generate a temporary goal. Afterwards, a heuristic and sampling method for online generation of splicing trajectory based on the goal is proposed innovatively, ensuring the vehicle can move feasibly after the change in human intention is detected. Once the trajectory is available, the vehicle velocity is then converted to joystick position information as the haptic cue of the human, which enhances the telepresence. The shared teleoperation control framework is verified in the simulation environment, where its excellent performance in the complex environment is evaluated, and its feasibility is confirmed. The experimental results show that the proposed method can achieve simple and efficient navigation in a complex environment, and can also give a certain situational awareness to the human.

Evaluating the effect of non-invasive force feedback on prosthetic grasp force modulation in participants with and without limb loss.

Grasping an object is one of the most common and complex actions performed by humans. The human brain can adapt and update the grasp dynamics through information received from sensory feedback. Prosthetic hands can assist with the mechanical performance of grasping, however currently commercially available prostheses do not address the disruption of the sensory feedback loop. Providing feedback about a prosthetic hand's grasp force magnitude is a top priority for those with limb loss. This study tested a wearable haptic system, i.e., the Clenching Upper-Limb Force Feedback device (CUFF), which was integrated with a novel robotic hand (The SoftHand Pro). The SoftHand Pro was controlled with myoelectrics of the forearm muscles. Five participants with limb loss and nineteen able-bodied participants completed a constrained grasping task (with and without feedback) which required modulation of the grasp to reach a target force. This task was performed while depriving participants of incidental sensory sources (vision and hearing were significantly limited with glasses and headphones). The data were analyzed with Functional Principal Component Analysis (fPCA). CUFF feedback improved grasp precision for participants with limb loss who typically use body-powered prostheses as well as a sub-set of able-bodied participants. Further testing, that is more functional and allows participants to use all sensory sources, is needed to determine if CUFF feedback can accelerate mastery of myoelectric control or would benefit specific patient sub-groups.

Promoting breast health awareness: Can a sensor-enabled training system for patient education help?

IntroductionAccording to a 2009 study published in the Journal of Clinical Oncology, 79% of women (N = 222) diagnosed with breast cancer reported that they identified their cancers through breast self-exam (BSE). However, the U.S. Preventative Services Task Force does not require clinicians to teach women how to perform BSE. MethodsTo address this grave challenge, our team at the Technology Enabled Clinical Improvement (TECI) Center has developed a mobile, sensor-enabled haptic training system to teach women proper BSE technique. To validate the efficacy of the training system, our team deployed a data collection at the 2019 Breast Cancer and African Americans (BCAA) event where survey, sensor, and anecdotal data were collected from 61 participants. The custom-built breast model used in this study had a single, hard mass embedded in it. ResultsParticipants at the BCAA event were able to successfully identify the mass 65% of the time and used an average force of 7.2 N. When looking at participants’ confidence in their abilities to perform BSE, only 10% of respondents answered “very confident” pre-training whereas post-training, the reporting for “very confident” jumped to 66% (p < 0.01). ConclusionBy comparison, our previous work revealed that practitioners who use less than 10 N of force are 70% more likely to miss a lesion. The integration of sensors into the BSE haptic training system allowed for objective, evidence-based assessment of hands-on skill. In addition to teaching women proper BSE technique, this training empowered women to be informed advocates in their breast health journey. Future community-based training/feedback sessions will allow for continuous advancement of the training system.

GroundFlow: Liquid-based Haptics for Simulating Fluid on the Ground in Virtual Reality.

Utilizing haptic devices to enhance the immersive experience is a direct approach in virtual reality (VR) applications. Various studies develop haptic feedback using force, wind, and thermal mechanisms. However, most haptic devices simulate feedback in dry environments such as the living room, prairie, or city. Water-related environments are thus less explored, for example, rivers, beaches, and swimming pools. In this paper we present GroundFlow, a liquid-based haptic floor system for simulating fluid on the ground in VR. We discuss design considerations and propose a system architecture and interaction design. We conduct two user studies to assist in the design of a multiple-flow feedback mechanism, develop three applications to explore the potential uses of the mechanism, and consider the limitations and challenges thereof to inform VR developers and haptic practitioners.

A Modular Haptic Agent System with Encountered-Type Active Interaction

Virtual agents are artificial intelligence systems that can interact with users in virtual reality (VR), providing users with companionship and entertainment. Virtual pets have become the most popular virtual agents due to their many benefits. However, haptic interaction with virtual pets involves two challenges: the rapid construction of various haptic proxies, and the design of agent-initiated active interaction. In this paper, we propose a modular haptic agent (MHA) prototype system, enabling the tactile simulation and encountered-type haptic interaction of common virtual pet agents through a modular design method and a haptic mapping method. Meanwhile, the MHA system with haptic interaction is actively initiated by the agents according to the user’s intention, which makes the virtual agents appear more autonomous and provides a better experience of human–agent interaction. Finally, we conduct three user studies to demonstrate that the MHA system has more advantages in terms of realism, interactivity, attraction, and raising user emotions. Overall, MHA is a system that can build multiple companion agents, provide active interaction and has the potential to quickly build diverse haptic agents for an intelligent and comfortable virtual world.

Haptic Systems: Trends and Lessons Learned for Haptics in Spacesuits

Haptic technology uses forces, vibrations, and movements to simulate a sense of touch. In the context of spacesuits, proposals to use haptic systems are scant despite evidence of their efficacy in other domains. Existing review studies have sought to summarize existing haptic system applications. Despite their contributions to the body of knowledge, existing studies have not assessed the applicability of existing haptic systems in spacesuit design to meet contemporary challenges. This study asks, “What can we learn from existing haptic technologies to create spacesuits?”. As such, we examine academic and commercial haptic systems to address this issue and draw insights for spacesuit design. The study shows that kinesthetic and tactile haptic systems have been effectively utilized in various domains, including healthcare, gaming, and education to improve the sense of touch and terrain and reduce sensory deprivation. Subjective and objective evaluation methods have been utilized to assess the efficacy and safety of haptic systems. Furthermore, this study discusses the usefulness, safety, and applicability of haptics in spacesuits and the implications for research into space haptics.

Peripheral focused ultrasound stimulation and its applications: From therapeutics to human-computer interaction.

Peripheral focused ultrasound stimulation (pFUS) has gained increasing attention in the past few decades, because it can be delivered to peripheral nerves, neural endings, or sub-organs. With different stimulation parameters, ultrasound stimulation could induce different modulation effects. Depending on the transmission medium, pFUS can be classified as body-coupled US stimulation, commonly used for therapeutics or neuromodulation, or as an air-coupled contactless US haptic system, which provides sensory inputs and allows distinct human-computer interaction paradigms. Despite growing interest in pFUS, the underlying working mechanisms remain only partially understood, and many applications are still in their infancy. This review focused on existing applications, working mechanisms, the latest progress, and future directions of pFUS. In terms of therapeutics, large-sample randomized clinical trials in humans are needed to translate these state of art techniques into treatments for specific diseases. The airborne US for human-computer interaction is still in its preliminary stage, but further efforts in task-oriented US applications might provide a promising interaction tool soon.

Programmable tactile feedback system for blindness assistance based on triboelectric nanogenerator and self-excited electrostatic actuator

Vibro-tactile devices are highly resistant to interference and perform excellently in noisy environments, rendering them the preferred method for blindness assistance. To construct an effective haptic feedback system, two key elements are required: a highly sensitive vibration actuator and a safe high-voltage power source. In this work, a self-powered tactile feedback system (STFS) is proposed on the basis of triboelectric nanogenerator (TENG) and self-excited electrostatic actuator (SEEA). The performances of the TENG and SEEA have been systematically investigated and optimized to ensure that the former can power the latter to produce sufficient vibration stimuli on the user’s skin. With the help of a programmed switch matrix, a series of practical applications have been demonstrated, including a refreshable Braille system, a haptic navigation interface, and tactile-assisted encrypted communication. The proposed STFS holds the merits of being portable, cost-effective, multifunctional, and high-accurate, demonstrating the great potential to substantially improve the quality of life for visually impaired individuals.