Automatic coarse-to-fine joint detection and segmentation of underwater non-structural live crabs for precise feeding

Abstract

Effective biomass detection methods utilizing computer vision techniques should be capable of handling large differences in object posture and various degrees of mutual overlap/occlusion in object scenes. We propose an automatic coarse-to-fine joint detection and instance segmentation network (JDSNet) that can perform real-time detection and instance segmentation on underwater non-structural live crabs in real-time. The method was adapted to non-structural objects and mobile devices by applying the anchor-free mechanism and center-ness strategy of the fully convolutional one-stage prediction head and the improved energy-efficient backbone IVoVNet-19-DW with identity mapping and channel attention. This approach avoided the complicated computations related to the anchor-based mechanism and effectively generated the features of various receiving domains, jointly improving the memory access speed and accuracy of predicting the bounding boxes of different instances. A novel spatial attention-guided mask branch was then added to focus on irregular occluded object pixels and conduct precise pixel-level mask segmentation within the predicted coarse instance-aware rectangular-bounding boxes. The experimental analysis using the proposed method resulted in a quality of detection F1 and segmentation Dic of 97.7% and 94.6%, respectively. The fastest detection speed of a single image was 48.07/13.32 fps (~10 times faster than the existing network Mask RCNN) on a commonly configured GPU/CPU, requiring only 7.04 MB of storage (~25 times smaller than Mask RCNN). It indicates that JDSNet can segment various non-structural live crabs and perform biomass statistics robustly and efficiently, exhibiting significance for precision feeding applications in automatic feeding boats.