Palpation Task Research Articles

Skin-stretch Haptic Feedback Augmentation Improves Performance in a Simulated Laparoscopic Palpation Task.

Laparoscopic surgery brings substantial benefits to patients. However, it remains challenging for surgeons because of motion constraints and perception limitations. Notably, the perception of interactions with organs is largely compromised. This paper evaluates the effectiveness of a forearm-based skin-stretch haptic feedback system rendering surgical tool tip force. Twenty novice participants had to discern the stiffness of samples to investigate stiffness perception in a simulated laparoscopic task. The experimental protocol involved manipulating samples with three difficulty levels and testing three feedback conditions: no augmentation, visual feedback, and tactile feedback. The results demonstrate that feedback significantly enhances the success rate of laparoscopic palpation tasks. The proposed tactile feedback boosts confidence and task speed and reduces peak force and perceived workload. These benefits become even more pronounced when difficulty increases. These promising findings affirm the value of skin-stretch haptic feedback augmentation in improving performance for simulated laparoscopy tasks, paving the way for more integrated and deployable devices for the operating room.

Laparoscopic surgery augmentation through vibro-acoustic sensing of instrument-tissue interactions

Abstract This paper presents the use of vibro-acoustic sensing to augment laparoscopic surgery procedures by analyzing the signals produced during cutting and palpation tasks on various tissue samples. Vibro-acoustic signals were acquired during an experiment on a dedicated phantom covered in dense foam, where three trocars were inserted into the phantom to place the endoscopic camera and two laparoscopic instruments. The results of the signals analysis demonstrate the potential of this approach for making laparoscopic interactions audible, differentiating between tissue types, and detecting variations in tissue properties. Vibro-acoustic sensing could be a valuable tool for integrating sound into the current clinical workflow for enhancing endoscope video images.

Vibro-acoustic sensing of tissue-instrument-interactions allows a differentiation of biological tissue in computerised palpation

Background:The shift towards minimally invasive surgery is associated with a significant reduction of tactile information available to the surgeon, with compensation strategies ranging from vision-based techniques to the integration of sensing concepts into surgical instruments. Tactile information is vital for palpation tasks such as the differentiation of tissues or the characterisation of surfaces. This work investigates a new sensing approach to derive palpation-related information from vibration signals originating from instrument-tissue-interactions. Methods:We conducted a feasibility study to differentiate three non-animal and three animal tissue specimens based on palpation of the surface. A sensor configuration was mounted at the proximal end of a standard instrument opposite the tissue-interaction point. Vibro-acoustic signals of 1680 palpation events were acquired, and the time-varying spectrum was computed using Continuous-Wavelet-Transformation. For validation, nine spectral energy-related features were calculated for a subsequent classification using linear Support Vector Machine and k-Nearest-Neighbor. Results:Indicators derived from the vibration signal are highly stable in a set of palpations belonging to the same tissue specimen, regardless of the palpating subject. Differences in the surface texture of the tissue specimens reflect in those indicators and can serve as a basis for differentiation. The classification following a supervised learning approach shows an accuracy of >93.8% for the three-tissue classification tasks and decreases to 78.8% for a combination of all six tissues. Conclusions:Simple features derived from the vibro-acoustic signals facilitate the differentiation between biological tissues, showing the potential of the presented approach to provide information related to the interacting tissue. The results encourage further investigation of a yet little-exploited source of information in minimally invasive surgery.

Face mediated human–robot interaction for remote medical examination

Realtime visual feedback from consequences of actions is useful for future safety-critical human–robot interaction applications such as remote physical examination of patients. Given multiple formats to present visual feedback, using face as feedback for mediating human–robot interaction in remote examination remains understudied. Here we describe a face mediated human–robot interaction approach for remote palpation. It builds upon a robodoctor–robopatient platform where user can palpate on the robopatient to remotely control the robodoctor to diagnose a patient. A tactile sensor array mounted on the end effector of the robodoctor measures the haptic response of the patient under diagnosis and transfers it to the robopatient to render pain facial expressions in response to palpation forces. We compare this approach against a direct presentation of tactile sensor data in a visual tactile map. As feedback, the former has the advantage of recruiting advanced human capabilities to decode expressions on a human face whereas the later has the advantage of being able to present details such as intensity and spatial information of palpation. In a user study, we compare these two approaches in a teleoperated palpation task to find the hard nodule embedded in the remote abdominal phantom. We show that the face mediated human–robot interaction approach leads to statistically significant improvements in localizing the hard nodule without compromising the nodule position estimation time. We highlight the inherent power of facial expressions as communicative signals to enhance the utility and effectiveness of human–robot interaction in remote medical examinations.

Enhancing the Localization of Uterine Leiomyomas Through Cutaneous Softness Rendering for Robot-Assisted Surgical Palpation Applications.

Integrating tactile feedback for lump localization in Robot-assisted Minimally Invasive Surgery (RMIS) represents an open research issue, which is still far to be solved. Main reasons for this are related e.g. to the need for a transparent connection with the teleoperating console, and an intuitive decoding of the delivered information. In this article, we focus on the specific case of RMIS treatment of uterine leiomyomas or fibroids, where little has been done in haptics to improve the outcomes of robotics-enabled palpation tasks. In this article, we propose the usage of a wearable haptic interface for softness rendering as a lump display. The device was integrated in a teleoperation architecture that simulates a robot-assisted surgical palpation task of leiomyomas. This article moved from an ex-vivo sample characterization of uterine tissues to show the effectiveness of our interface in conveying meaningful softness information. We extensively tested our system with gynecologic surgeons in palpation tasks with silicone specimens, which replicated the characteristics of uterine tissues with embedded leyomiomas. Results show that our system enables a softness-based discrimination of the embedded fibroids comparable to the one that physicians would achieve using directly their fingers in palpation tasks. Furthermore, the feedback provided by the haptic interface was perceived as comfortable, intuitive, and highly useful for fibroid localization.

User involvement in early-stage design of medical training devices – case of a palpation task trainer prototype

Design of new medical training equipment is demanding as unavoidable complexity and ambiguity facing designers must be addressed in the early stages of the development. In this paper, a novel concept for abdominal palpation training is presented and used to exemplify challenges and approaches for designing new medical training equipment. Concluding the initial development of the palpation training concept, experienced medical personnel evaluated a conceptual prototype. A Likert scale questionnaire, a clinical assessment by participants, and recorded sensor data were used to evaluate prototype functionalities, perceived tactile- and visual realism, and usability of the concept in medical training. Obtained results are used to discuss insights for further development of the concept. Further, the paper discusses observations from user interactions and considerations regarding fidelity of medical training equipment prototypes. Moreover, it highlights the benefits of utilizing mixed-method research to identify areas of improvement for conceptual prototypes even before initiating industrial development efforts. By this, designers can ensure that user needs, product requirements, and sufficient fidelity are collectively captured in new medical training equipment in the early design stages.

Bio-Inspired Haptic Feedback for Artificial Palpation in Robotic Surgery.

Adding haptic feedback has been reported to improve the outcome of minimally invasive robotic surgery. In this study, we seek to determine whether an algorithm based on simulating responses of a cutaneous afferent population can be implemented to improve the performance of presenting haptic feedback for robot-assisted surgery. We propose a bio-inspired controlling model to present vibration and force feedback to help surgeons localize underlying structures in phantom tissue. A single pair of actuators was controlled by outputs of a model of a population of cutaneous afferents based on the pressure signal from a single sensor embedded in surgical forceps. We recruited 25 subjects including 10 expert surgeons to evaluate the performance of the bio-inspired controlling model in an artificial palpation task using the da Vinci surgical robot. Among the control methods tested, the bio-inspired system was unique in allowing both novices and experts to easily identify the locations of all classes of tumors and did so with reduced contact force and tumor contact time. This work demonstrates the utility of our bio-inspired multi-modal feedback system, which resulted in superior performance for both novice and professional users, in comparison to a traditional linear and the existing piecewise discrete algorithms of haptic feedback.

Task Dynamics of Prior Training Influence Visual Force Estimation Ability During Teleoperation

The lack of haptic feedback in teleoperation is a potential barrier to safe handling of soft materials, yet in Robot-assisted Minimally Invasive Surgery (RMIS), haptic feedback is often unavailable. Due to its availability in open and laparoscopic surgery, surgeons with such experience potentially possess learned models of tissue stiffness that might promote good force estimation abilities during RMIS. To test if prior haptic experience leads to improved force estimation ability in teleoperation, 33 naive participants were assigned to one of three training conditions: manual manipulation, teleoperation with force feedback, or teleoperation without force feedback, and learned to tension a silicone sample to a set of forces. They were then asked to perform the tension task, and a previously unencountered palpation task, to a different set of forces under teleoperation without force feedback. Compared to the teleoperation groups, the manual group had higher force error in the tension task outside the range of forces they had trained on, but showed better speed-accuracy functions in the palpation task at low force levels. This suggests that the dynamics of the training modality affect force estimation ability during teleoperation, with the prior haptic experience accessible if formed under the same dynamics as the task.

Conditioned haptic perception for 3D localization of nodules in soft tissue palpation with a variable stiffness probe.

This paper provides a solution for fast haptic information gain during soft tissue palpation using a Variable Lever Mechanism (VLM) probe. More specifically, we investigate the impact of stiffness variation of the probe to condition likelihood functions of the kinesthetic force and tactile sensors measurements during a palpation task for two sweeping directions. Using knowledge obtained from past probing trials or Finite Element (FE) simulations, we implemented this likelihood conditioning in an autonomous palpation control strategy. Based on a recursive Bayesian inferencing framework, this new control strategy adapts the sweeping direction and the stiffness of the probe to detect abnormal stiff inclusions in soft tissues. This original control strategy for compliant palpation probes shows a sub-millimeter accuracy for the 3D localization of the nodules in a soft tissue phantom as well as a 100% reliability detecting the existence of nodules in a soft phantom.

A technology-aided multi-modal training approach to assist abdominal palpation training and its assessment in medical education

Computer-assisted multimodal training is an effective way of learning complex motor skills in various applications. In particular disciplines (eg. healthcare) incompetency in performing dexterous hands-on examinations (clinical palpation) may result in misdiagnosis of symptoms, serious injuries or even death. Furthermore, a high quality clinical examination can help to exclude significant pathology, and reduce time and cost of diagnosis by eliminating the need for unnecessary medical imaging. Medical palpation is used regularly as an effective preliminary diagnosis method all around the world but years of training are required currently to achieve competency. This paper focuses on a multimodal palpation training system to teach and improve clinical examination skills in relation to the abdomen. It is our aim to shorten significantly the palpation training duration by increasing the frequency of rehearsals as well as providing essential augmented feedback on how to perform various abdominal palpation techniques which has been captured and modelled from medical experts. Twenty three first year medical students divided into a control group (n=8), a semi-visually trained group (n=8), and a fully visually trained group (n=7) were invited to perform three palpation tasks (superficial, deep and liver). The medical students performances were assessed using both computer-based and human-based methods where a positive correlation was shown between the generated scores, r=.62, p(one-tailed)<.05. The visually-trained group significantly outperformed the control group in which abstract visualisation of applied forces and their palmar locations were provided to the students during each palpation examination (p<.05). Moreover, a positive trend was observed between groups when visual feedback was presented, J=132, z=2.62, r=0.55.

Dealing with Ecological Validity and User Needs when Developing Simulation Based Training Equipment – Case Study of a Medical Palpation Task Trainer

Simulation-based training offers a safe and repeatable environment where users can increase their skill level. Development of products facilitating such training is, however, faced with high uncertainty and ambiguous requirements. This complexity is a result of the intended use-cases, required product functionality and required approximation of clinical realism. In order to explore such uncertain aspects of simulation-based training equipment, two interesting factors are considered; user needs and ecological validity. This paper presents a case project where a medical palpation task trainer has been developed. In this case, a medical diagnostics procedure has been enabled through specific product functionality. As the complexity associated with simulators makes it difficult to elicit and fixate requirements, this paper highlights the benefits of using prototypes during the early stages of development. By presenting prototypes and testing these with both experienced and novice users, developers could better understand complex use cases and product interactions. Through case examples it is shown how prototyping extensively in various domains and levels of abstraction could help elicit product requirements as well as enlighten unknown problems and corresponding opportunities. In order to create products facilitating effective training, training scenarios must be made sufficiently realistic, i.e. ensuring ecological validity. To accomplish this, a simulator concept must encompass realistic tasks and physical attributes. Further, the potential of the conceptual prototype is discussed where self-directed learning and learning algorithms are of interest. Future research will, therefore, be focused on improved medical training and consequently training devices.

Examining the Effect of Haptic Factors for Vascular Palpation Skill Assessment Using an Affordable Simulator.

Goal: Simulators that incorporate haptic feedback for clinical skills training are increasingly used in medical education. This study addresses the neglected aspect of rendering simulated feedback for vascular palpation skills training by systematically examining the effect of haptic factors on performance. Methods: A simulator-based approach to examine palpation skill is presented. Novice participants with and without minimal previous palpation training performed a palpation task on a simulator that rendered controlled vibratory feedback under various conditions. Results: Five objective metrics were employed to analyze participants’ performance that yielded key findings in quantifying palpation performance. Participants’ palpation accuracy was influenced by all three haptic factors, ranging from moderate to statistically significant. Duration, Total Path Length and Ratio of Correct Movement also demonstrated utility for quantifying performance. Conclusions: We demonstrate that our affordable simulator is capable of rendering controlled haptic feedback suitable for skills training. Further, metrics presented in this study can be used for structured palpation skills assessment and training, potentially improving healthcare delivery.

Concept and Realization of a Handheld Manipulator System for Transnasal Middle Ear Surgery

Abstract In the context of conductive hearing loss and pathologies of the ossicular chain, invasive procedures are necessary for inspection and treatment. We propose a handheld manipulator concept for the palpation of the ossicles under endoscopic view. The system can access the middle ear through the Eustachian tube without any incision thanks to its small cross-section. Virtual verification tests conducted using medical image data of 30 patients regarding the workspace of the manipulator showed that malleus, incus and stapes can be reached in 83%, 90% and 63% of the cases respectively. Tests regarding manipulation forces showed that the system can apply -0.95 N traction force and 0.80 N compression force, which are sufficient for the palpation task according to the literature.

Additional planning with multiple objectives for reinforcement learning

Most control tasks have multiple objectives that need to be achieved simultaneously, while the reward definition is the weighted combination of all objects to determine one optimal policy. This configuration has a limitation in exploration flexibility and presents difficulty in reaching a satisfied terminate condition. Although some multi-objective reinforcement learning (MORL) methods have been presented recently, they concentrate on obtaining a set of compromising options rather than one best-performed strategy. On the other hand, the existing policy-improve methods have rarely emphasized on solving multiple objectives circumstances. Inspired by the enhanced policy search methods, an additional planning technique with multiple objectives for reinforcement learning is proposed in this paper, which is denoted as RLAP-MOP. This method provides opportunities to evaluate parallel requirements at the same time and suggests several optimal feasible actions to improve long-term performance further. Meanwhile, the short-term planning adopted in this paper has advantages in maintaining safe trajectories and building more accurate approximate models, which contributes to accelerating the training program. For comparison, an RLAP with single-objective optimization is also introduced in theoretical and experimental studies. The proposed techniques are investigated on a multi-objective cartpole environment and a soft robotic palpation task. The superiorities in the improved return values and learning stability prove that the multiple objectives based additional planning is a promising assistant to improve reinforcement learning.

Transferring optimal contact skills to flexible manipulators by reinforcement learning

Flexible/soft manipulators have the potential to maneuver in confined space and reach deeply-seated targets via curvy trajectories, thus enjoy increasing popularity in minimally invasive surgery (MIS) community. We aim to automate palpation movement for this type of robots, an important procedure for disease diagnosis, where multiple force and pose requirements are to be achieved simultaneously. It’s challenging to obtain accurate models due to the system’s inherent nonlinearities and actuation hysteresis. Moreover, unknown contact transitions and high-dimensionality specific to the palpation task, pose great challenges to deriving optimal task policies. We employ the model-free reinforcement learning method for learning palpation skills through deterministic policy gradient, whose reward function was carefully shaped to accommodate all the task objectives. In addition, we design a safety check routine to avoid undesirable collisions and a dedicated initialization process for generalization to various environment conditions. We demonstrate successful implementation of the learning framework in simulation and real world. The trained policy succeeds in automating the designed tasks.

Compensating Uncertainties in Force Sensing for Robotic-Assisted Palpation

Force sensing in robotic-assisted minimally invasive surgery (RMIS) is crucial for performing dedicated surgical procedures, such as bilateral teleoperation and palpation. Due to the bio-compatibility and sterilization requirements, a specially designed surgical tool/shaft is normally attached to the sensor while contacting the organ targets. Through this design, the measured force from the sensor usually contains uncertainties, such as noise, inertial force etc., and thus cannot reflect the actual interaction force with the tissue environment. Motivated to provide the authentic contact force between a robotic tool and soft tissue, we proposed a data-driven force compensation scheme without intricate modeling to reduce the effects of force measurement uncertainties. In this paper, a neural-network-based approach is utilized to automatically model the inertial force subject to noise during the robotic palpation procedure, then the exact contact force can be obtained through the force compensation method which cancels the noise and inertial force. Following this approach, the genuine interaction force during the palpation task can be achieved furthermore to improve the appraisal of the tumor surrounded by the soft tissue. Experiments are conducted with robotic-assisted palpation tasks on a silicone-based soft tissue phantom and the results verify the effectiveness of the suggested method.

Application of a Redundant Haptic Interface in Enhancing Soft-Tissue Stiffness Discrimination

Haptic-enabled teleoperated surgical systems have the potential to enhance the accuracy and performance of surgical interventions. The user interface of such a system can provide haptic feedback to the surgeon to more intuitively perform surgical tasks. In this letter, we study the added benefits of redundant manipulators as haptic interfaces for teleoperated surgical systems. First, we introduce the intrinsic benefits of employing a redundant haptic interface, namely, reduced apparent inertia and increased manipulability (one result of which is reduced friction forces). Next, we demonstrate that the haptic interface redundancy can further reduce its apparent inertia and friction via appropriately manipulating the extra degrees of freedom of the interface. This will consequently enhance the haptic feedback resolution (sensitivity) for the user. Finally, a psychophysical experiment is performed to validate the improved force perception for the user in a virtual soft-tissue palpation task. We conduct a set of perceptual experiments to evaluate how a redundant and non-redundant user interface affects the perception of the virtual stiffness. Experimental results demonstrate that the redundancy in the haptic user interface helps to enhance tissue stiffness discrimination ability of the user by reducing the distortions caused by the kinematics and dynamics of the user interface.

A Modular Wearable Finger Interface for Cutaneous and Kinesthetic Interaction: Control and Evaluation

In this paper, we present a novel modular wearable interface for haptic interaction and robotic teleoperation. It is composed of a 3-degree-of-freedom (3-DoF) fingertip cutaneous device and a 1-DoF finger kinesthetic exoskeleton, which can be either used together as a single device or separately as two different devices. The 3-DoF fingertip device is composed of a static body and a mobile platform. The mobile platform is capable of making and breaking contact with the finger pulp and reangle to replicate contacts with arbitrarily oriented surfaces. The 1-DoF finger exoskeleton provides kinesthetic force to the proximal and distal interphalangeal finger articulations using one servo motor grounded on the proximal phalanx. This paper presents the wearable device as well as three different position, force, and compliance control schemes, together with their evaluations. We also present three human subjects experiments, enrolling a total of 40 different participants: the first experiment considered a curvature discrimination task, the second one a robot-assisted palpation task, and the third one an immersive experience in virtual reality. Results show that providing cutaneous and kinesthetic feedback through our device significantly improve the performance of all the considered tasks. Moreover, although cutaneous-only feedback shows promising performance, adding kinesthetic feedback improves most metrics. Finally, subjects rank our device as highly wearable, comfortable, and effective.

Artificial palpation in robotic surgery using haptic feedback.

The loss of tactile feedback in minimally invasive robotic surgery remains a major challenge to the expanding field. With visual cue compensation alone, tissue characterization via palpation proves to be immensely difficult. This work evaluates a bimodal vibrotactile system as a means of conveying applied forces to simulate haptic feedback in two sets of studies simulating an artificial palpation task using the da Vinci surgical robot. Subjects in the first study were tasked with localizing an embedded vessel in a soft tissue phantom using a single-sensor unit. In the second study, subjects localized tumor-like structures using a three-sensor array. In both sets of studies, subjects completed the task under three trial conditions: no feedback, normal force tactile feedback, and hybrid vibrotactile feedback. Recordings of correct localization, incorrect localization, and time-to-completion were used to evaluate performance outcomes. With the addition of vibrotactile and pneumatic feedback, significant improvements in the percentage of correct localization attempts were detected (p = 0.0001 and p = 0.0459, respectively) during the first experiment with phantom vessels. Similarly, significant improvements in correct localization were found with the addition of vibrotactile (p = 2.57E-5) and pneumatic significance (p = 8.54E-5) were observed in the second experiment involving tumor phantoms. This work demonstrates not only the superior benefits of a multi-modal feedback over traditional single-modality feedback, but also the effectiveness of vibration in providing haptic feedback to artificial palpation systems.

A Surgical Palpation Probe With 6-Axis Force/Torque Sensing Capability for Minimally Invasive Surgery

A novel surgical palpation probe installing a miniature 6-axis force/torque (F/T) sensor is presented for robot-assisted minimally invasive surgery. The 6-axis F/T sensor is developed by using the capacitive transduction principle and a novel capacitance sensing method. The sensor consists of only three parts, namely a sensing printed circuit board, a deformable part, and a base part. The simple configuration leads to simpler manufacturing and assembly processes in conjunction with high durability and low weight. In this study, a surgical instrument installed with a surgical palpation probe is implemented. The 6-axis F/T sensing capability of the probe has been experimentally validated by comparing it with a reference 6-axis F/T sensor. Finally, a vivo tissue palpation task is performed in a simulated surgical environment with an animal organ and a relatively hard simulated cancer buried under the surface of the organ.