Research on Manipulator Control Based on RGB-D Sensor

Abstract

AbstractAt present, the most mature way of teaching control of industrial robot manipulator is based on the control of teaching pendant joystick. With the development of human–computer interaction technology, more and more interaction technologies can be studied, such as voice control, gesture control and so on. This paper is devoted to the realization of new interaction methods and control logic, so that the manipulator can be controlled by gesture without contact. The implementation process of this method is as follows: firstly, get depth data by Kinect sensor [1], and use filter to reduce data jitter. Secondly, the user in the depth data is separated from the background to reduce the interference and the amount of calculation. Third, match depth data coordinate system and RGB image coordinate system. Use relative pixel coordinate in RGB image to detect the gesture. Fourth, the obtained coordinate data is transmitted to the rapid program of industrial robot through socket. Finally, the proposed method was tested and the results showed that: users can control manipulator to move and grasp objects by gesture smoothly. Through experiment, we can evaluate the performance and disadvantages of our system. The gesture recognition rate is more than 80% (only one gesture is below 90%. The system can lock the user as an operator target. After practice, the recognition rate of a freshman operator can be higher than 75%. What’s more, this system has stability when program runs long time. The performance degradation is less than 5% after 30 min of operation. Though it doesn’t have a universal protocol in the field of gesture recognition, we still believe the technology of HRI will impact our life in the future, and be expected to extend widely. We develop this system for extensibility. Our method is suitable for remote control and application scenarios under special working conditions, and has a good market application prospect.KeywordsMachine visionIntelligent controlHuman–computer interactionIndustrial robot