Next Level of Human-Robot Collaboration by Utilizing AI Pose Estimation and Model Predictive Motion Planning Technologies

Abstract

Over recent years, there has been a significant focus on Human Robot Interaction (HRI). The aim of this interaction is to create a mutually beneficial partnership between humans and robots, with robots contributing precision, speed, and force, while humans provide experience, intuition, and high-level management and control strategy understanding. One of the most crucial factors in achieving a successful HRI is the guidance of these robots. This paper proposes the use of reliable Hand Detection based on Long Short-Term Memory (LSTM) and intelligent robot motion planning based on Model Predictive Control Methods (MPC) to detect hand signals and communicate with the robot. By utilizing vision systems, a micro controller-based Edge AI approach, and a wired connection, the reaction speed of the robot can be optimized to maintain a safe separation distance between the human operator and the robot, thus enabling a collaborative and safe environment. The effectiveness and capabilities of this automation framework are validated through a case study.