Fish Image Instance Segmentation: An Enhanced Hybrid Task Cascade Approach

Abstract

The fish instance segmentation task plays an important role in fish image analysis. Traditional fish analysis methods (e.g., segmenting the fish curve by hand to obtain the size of fish) cost a mass of manual labor and thus are not efficient. Convolutional Neural Networks (CNNs) become an effective scheme to replace manual labor to decrease costs and improve efficiency. The Hybrid Task Cascade (HTC) is a novel CNN model which applies cascade architecture to achieve boosted performance in the instance segmentation task. However, instance segmentation models cannot handle fish images very well due to small image size and low image quality. Furthermore, HTC still suffers from the incomplete confidence score only consisting of the classification information without the mask information, so the instance segmentation performance would be degraded. To this end, we propose an Enhanced Hybrid Task Cascade (EHTC) model to overcome these limitations. (1) The EHTC conducts data pre-processing before the instance segmentation network through an image super-resolution technology to resize images and optimize features that can be more easily understood by the later instance segmentation network. (2) Our EHTC addresses the incomplete confidence score problem in the HTC by adding one mask scoring block, named MaskIoU, to generate mask confidence scores providing the mask that improves the instance segmentation accuracy. Finally, the experimental results show that our EHTC achieves better performance than the state-of-the-art models on the Fish4knowledge dataset.