Latency mitigation using applied HMMs for mixed reality-enhanced intuitive teleoperation in intelligent robotic welding

Abstract

Latency mitigation is crucial to increasing operational success, ease of use, and product quality in telemanipulation tasks when remotely guiding complex robotic systems. Hardware limitations have created a gap in performance optimization due to large teleoperation delays, which machine learning techniques could fill with lower time, improved performance, and reduced operating costs. Hidden Markov models (HMMs), in particular, have been explored to alleviate the issue due to their relative ease of use. A mixed reality-enhanced intuitive teleoperation framework for immersive and intuitive telerobotic welding is presented. The proposed system implements an HMM generative algorithm to learn and predict human-welder motion to enable a low-cost solution, combining smoothing and forecasting techniques to minimize robotic teleoperation time delay. The predicted welding motion system is simple to implement, can be used as a general solution to solve time delays, and is accurate. More specifically, it provides a 66% RMSE reduction compared to the application without HMM, which may be further optimized by up to 38%. Experiments show the HMM generative algorithm lets humans conduct tele-robot-assisted welding with better performance.