A Real-time Visual Servo Architecture for a Golden Fish Catching System

Abstract

Designing a real-time visual tracking system to catch golden fish is a complex task due to difficulties involved in the precise determination of a fish's relative 3D positions in the water tank. In this paper we present a new visual servo control system using stereo vision to acquire the 3D positions of the golden fish. The continuously adaptive mean shift (CAMSHIFT) algorithm and optical flow vision tracking methods are applied to provide, at a high frame-rate, the real-time feedback of the position and motion trajectory of the fish being tracked. A visual servo architecture for controlling two low-cost, well calibrated cameras for tracking the golden fish and a 5-axis robot manipulator for catching a specified golden fish are constructed using standard off-the-shelf hardware and open source software. The designed visual tracking and servo system is less sensitive to lighting influences and thus performs more efficiently. Experiments of the proposed method yielded very good results as the real-time 3D vision was able to successfully track two fish and guide the manipulator to catch one of them with a net attached to the manipulator's end-effector.