Robust tracking of fish schools using CNN for head identification

Abstract

Tracking individuals in a fish school with video cameras is one of the most effective ways to quantitatively investigate their behavior which is of great value for biological research. However, tracking large numbers of fish with complex non-rigid deformation, similar appearance and frequent mutual occlusions is a challenge task. In this paper we propose an effective tracking method that can reliably track a large number of fish throughout the entire duration. The first step of the proposed method is to detect fish heads using a scale-space method. Data association across frames is achieved via identifying the head image pattern of each individual fish in each frame, which is accomplished by a convolutional neural network (CNN) specially tailored to suit this task. Then the prediction of the motion state and the recognition result by CNN are combined to associate detections across frames. The proposed method was tested on 5 video clips having different number of fish respectively. Experiment results show that the correctness of their identities is not affected by frequent occlusions. The proposed method outperforms two state-of-the-art fish tracking methods in terms of 7 performance metrics.