Using a convolutional neural network for fingerling counting: A multi-task learning approach

Abstract

Fingerling counting is an important task for decision-making in the aquaculture context. The counting is usually performed by a human, which is time-consuming and prone to errors. Artificial intelligence methods applied to image interpretation can be a great strategy for solving this task automatically. However, applying machine learning to attend to aquaculture issues is an underexplored field that requires novel investigations, especially of methods that explore temporal information in videos. In this study, we propose a new method to locate and count fingerlings in a sequence of images using convolutional neural networks. The proposed method estimates three tasks in a multi-task approach. The first task consists of predicting the probability of a fingerling occurring in each pixel of the frame, while the second and third tasks estimate the movement performed by the fingerlings. Motion prediction is used as a complement to fingerling detection, including relevant information especially when two or more fingerlings are in contact. Experimental results indicated that the use of temporal information considerably increases the results, reaching F1 of 97.89. The proposed method was evaluated in frames with different numbers of fingerlings (from 0 to 10) and all obtained relevant results, with an F1 of 95.42 or higher. The study also showed that, in most cases, the proposed method can detect the contact of two or more fingerlings, which is considered the main challenge of the detection and counting of fingerlings.