Online 3-D Trajectory Estimation of a Flying Object from a Monocular Image Sequence for Catching

Abstract

Catching a fast moving object can be used to describe work across many subfields of robotics, sensing, processing, actuation, and systems design. The reaction time allowed to the entire robot system: sensors, processor and actuators is very short. The sensor system must provide estimates of the object trajectory as early as possible, so that the robot may begin moving to approximately the correct place as early as possible. High accuracy must be obtained, so that the best possible catching position can be computed and maximum reaction time is available. 3D visual tracking and catching of a flying object has been achieved successfully by several researchers in recent years (Andersson; 1989)-(Mori et al.; 2004). There are two basic approaches to visual servo control: Position-Based Visual Servoing (PBVS), where computer techniques are used to reconstruct a representation of the 3D workspace of the robot, and actuator commands are computed with respect to the 3D workspace; and, Image-Based Visual Servoing (IBVS), where an error signal measured directly in the image is mapped to actuator commands. In most of the research done in robotic catching using PBVS, the trajectory of the object is predicted with data obtained with a stereo vision system (Andersson; 1989)-(Namiki & Ishikawa; 2003), and the catching is achieved using a combination of light weight robots (Hove & Slotine; 1991) with fast grasping actuators (Hong & Slotine; 1995; Namiki & Ishikawa; 2003). A major difference exists between motion and structure estimation from binocular image sequences and that from monocular image sequences. With binocular image sequences, once the baseline is calibrated, the 3-D position of the object with reference with the cameras can be obtained. Using IBVS, catching a ball has been achieved successfully in a hand-eye configuration with a 6 DOF robot manipulator and one CCD camera based on GAG strategy (Mori et al.; 2004). Estimation of 3D trajectories from a monocular image sequence has been researched by (Avidan & Shashua; 2000; Cui et al.; 1994; Chan et al.; 2002; Ribnick et al.; 2009), among others, but to the best of our knowledge, no published work has addressed the 3-D catching of a fast moving object using monocular images with a PBVS system. Our system (see Fig. 1) consists of one high speed stationary camera, a personal computer to calculate and predict the trajectory online of the object, and a 6 d.o.f. arm to approach the manipulator to the predicted position.