A Vision‐Based Approach for Estimating Contact Forces: Applications to Robot‐Assisted Surgery

Abstract

The primary goal of this paper is to provide force feedback to the user using vision‐based techniques. The approach presented in this paper can be used to provide force feedback to the surgeon for robot‐assisted procedures. As proof of concept, we have developed a linear elastic finite element model (FEM) of a rubber membrane whereby the nodal displacements of the membrane points are measured using vision. These nodal displacements are the input into our finite element model. In the first experiment, we track the deformation of the membrane in real‐time through stereovision and compare it with the actual deformation computed through forward kinematics of the robot arm. On the basis of accurate deformation estimation through vision, we test the physical model of a membrane developed through finite element techniques. The FEM model accurately reflects the interaction forces on the user console when the interaction forces of the robot arm with the membrane are compared with those experienced by the surgeon on the console through the force feedback device. In the second experiment, the PHANToM haptic interface device is used to control the Mitsubishi PA‐10 robot arm and interact with the membrane in real‐time. Image data obtained through vision of the deformation of the membrane is used as the displacement input for the FEM model to compute the local interaction forces which are then displayed on the user console for providing force feedback and hence closing the loop.