FishNet: Fish visual recognition with one stage multi‐task learning

Abstract

The use of computer vision for fish monitoring in aquaculture fisheries has gained importance. It is crucial to obtain the object box, instance mask and landmarks of the fish to determine their status. There are well-established methods to achieve these tasks, but running them in a serial sequence is inefficient and complex. A multi-tasking framework is proposed that can implement the above three tasks in parallel, FishNet. Unlike other multi-tasking frameworks that use one encoder with multiple decoders, the authors use only one encoder and one decoder to achieve multi-tasking fusion and can be trained end-to-end. A multi-task dataset for fish is produced to validate the framework. It achieved the best speed-accuracy balance on object detection (a 95.3% box AP), instance segmentation (a 53.9% mask AP) and pose recognition (95.1% OKS AP), and reached real-time inference speed (66.3 FPS) on the NVIDIA Tesla V100.