Preliminary studies on an automated 3D fish tracking methodbased on a single video camera

Abstract

The study of fish behavior lays an important foundation for comprehending of fish migration routes,improving fishing efficiency and the protection of fishery resources.A large number of data are necessary in the study,such as stress response,cluster,migration and other measured data.However,getting these data is a time-consuming process.As fish behavior is often recorded in the form of video and a stereo camera recording system is popularly used for measurement and observation in the laboratory study,how to extract the data of fish behavior efficiently from the video has been a major problem in the study of fish behavior.By far fish 3D coordinate is usually calculated by hand,or by self made software which turns an importing fish 2D coordinate into a 3D one.In order to improve fish behavior data extraction efficiency,this paper presents an automated 3D fish tracking method based on a single video camera.A waterproof mirror was set above the experimental aquaria to simulate a camera shooting from the top,which provided a way to use a single camera for 3D imaging.We extract the data of fish behavior automatically by 3D fish tracking method which is divided into four parts: distortion calibration of single camera system,transfer formula between image coordinate to world coordinate,the automated tracking algorithm of fish movement and the automated output of fish behavior 2D and 3D data.Tests find out that while the distance between the camera and the aquaria is 1.5 m,the distortion calibration result shows the pixel error is much more acceptable which is about 0.1 pixels.As the camera tilted slightly during the experiment,the shape of the aquaria in the images changed.So based on the processing of Free-Form Deformation,the deformation of images is rectified during coordinate transform process.Then we implemented the algorithm of Interacting Multiple Model Joint Probabilistic Data Association(IMMJPDA) to automatically track fishes in 3D and output fish behavior data.The result of 6 Hemigrammus rhodostomus tracking experiment shows that: IMMJPDA algorithm can deal with the key issues during fish tracking system,which enables the method to extract individual fish from video images,construct their tracks,output 3D positions and speeds,and finally generate a complete 3D movement track drawing for fish behavior analysis.In a dense clutter situation JPDA requires a fairly large amount of computation to evaluate the joint probabilities.We combined Nearest Neighbor algorithm and JPDA algorithm to reduce the computational burden.