Humanoid Robot's Visual Imitation of 3-D Motion of a Human Subject Using Neural-Network-Based Inverse Kinematics

Abstract

A sequence of 3-D motion images of a human facing an imitator [i.e., a humanoid robot (HR)] is captured using a stereo-vision system installed on the HR. After acquiring enough motion sequences via image ratio and background registration, the estimation of seven feature points (i.e., the head, two arm tips, two leg tips, and two elbows) of the human are obtained. Using five feature points (i.e., the head, two arm tips, and two leg tips), the extraction of key posture frames from the recorded videos is achieved. To ensure a stable imitation motion of the HR, the human motions of the lower body (LB) and the upper body (UB) are imitated separately. All 11 stable motions for the LB are classified by the motion direction, the motion state, and the support phase, whereas only the two arm tips are employed to imitate the motions of the UB. Due to the disadvantages of traditional inverse kinematics (IK), neural-network-based IK is developed to reduce the computation time and improve the accuracy of the time response of the system. Finally, the combined motions of the LB and the UB with suitable interpolation and time intervals are applied to imitate the 3-D motion of a human. The corresponding experimental results confirm the efficacy of the proposed technique.