Intuitive gesture-based control system with collision avoidance for robotic manipulators

Abstract

PurposeThis paper aims to present an intuitive control system for robotic manipulators that pairs a Leap Motion, a low-cost optical tracking and gesture recognition device, with the ability to record and replay trajectories and operation to create an intuitive method of controlling and programming a robotic manipulator. This system was designed to be extensible and includes modules and methods for obstacle detection and dynamic trajectory modification for obstacle avoidance.Design/methodology/approachThe presented control architecture, while portable to any robotic platform, was designed to actuate a six degree-of-freedom robotic manipulator of our own design. From the data collected by the Leap Motion, the manipulator was controlled by mapping the position and orientation of the human hand to values in the joint space of the robot. Additional recording and playback functionality was implemented to allow for the robot to repeat the desired tasks once the task had been demonstrated and recorded.FindingsExperiments were conducted on our custom-built robotic manipulator by first using a simulation model to characterize and quantify the robot’s tracking of the Leap Motion generated trajectory. Tests were conducted in the Gazebo simulation software in conjunction with Robot Operating System, where results were collected by recording both the real-time input from the Leap Motion sensor, and the corresponding pose data. The results of these experiments show that the goal of accurate and real-time control of the robot was achieved and validated our methods of transcribing, recording and repeating six degree-of-freedom trajectories from the Leap Motion camera.Originality/valueAs robots evolve in complexity, the methods of programming them need to evolve to become more intuitive. Humans instinctively teach by demonstrating the task to a given subject, who then observes the various poses and tries to replicate the motions. This work aims to integrate the natural human teaching methods into robotics programming through an intuitive, demonstration-based programming method.