Abstract
Haptic force feedback is an important perception method for humans to understand the surrounding environment. It can estimate tactile force in real time and provide appropriate feedback. It has important research value for robot-assisted minimally invasive surgery, interactive tactile robots, and other application fields. However, most of the existing noncontact visual power estimation methods are implemented using traditional machine learning or 2D/3D CNN combined with LSTM. Such methods are difficult to fully extract the contextual spatiotemporal interaction semantic information of consecutive multiple frames of images, and their performance is limited. To this end, this paper proposes a time-sensitive dual-resolution learning network-based force estimation model to achieve accurate noncontact visual force prediction. First, we perform continuous frame normalization processing on the robot running the video captured by the camera and use the hybrid data augmentation to improve the data diversity; secondly, a deep semantic interaction model is constructed based on the time-sensitive dual-resolution learning network, which is used to automatically extract the deep spatiotemporal semantic interaction information of continuous multiframe images; finally, we construct a simplified prediction model to realize the efficient estimation of interaction force. The results based on the large-scale robot hand interaction dataset show that our method can estimate the interaction force of the robot hand more accurately and faster. The average prediction MSE reaches 0.0009 N, R 2 reaches 0.9833, and the average inference time for a single image is 6.5532 ms; in addition, our method has good prediction generalization performance under different environments and parameter settings.
Highlights
With the rapid development of artificial intelligence and sensor technology, as well as the urgent demand for robots in medical, intelligent services, and other fields, research on intelligent robots has important value and significance [1, 2]
Our method includes three main steps: performing continuous frame normalization processing on the robot running video collected by the camera, constructing a deep semantic interaction model based on the time-sensitive dual-resolution learning (TDL) network and automatically extract deep fusion spatiotemporal semantic information, and building a streamlined predictive model to achieve efficient estimation of the interaction force of the robot
Compared with traditional contact tactile sensors, which are restricted by biocompatibility or excessive sensor size, force estimation and feedback through noncontact visual information has become a mainstream solution
Summary
With the rapid development of artificial intelligence and sensor technology, as well as the urgent demand for robots in medical, intelligent services, and other fields, research on intelligent robots has important value and significance [1, 2]. Among them, sensing and estimating the force information between the robot and the object are a key step to realize the humanoid robot [3,4,5]. Humans use the tactile information obtained in the process of contact with the outside world to determine their behavior. We hope that robots can freely interact with the outside world like humans to obtain dynamic or static tactile information from the outside world and intelligently judge the current state and execute the action based on the real-time state [6, 7]. Researchers designed a clever touch sensor and embedded it in the hands of the robot and used hardware circuits combined with digital signal processing algorithms to measure the interaction force between the Journal of Sensors manipulator and the object, thereby improving the accuracy of manipulator operation [8]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
