A semi-supervised generative adversarial network for amodal instance segmentation of piglets in farrowing pens

Abstract

Occlusions, such as farrowing pens in piggeries, hinder computer vision applications for automated animal monitoring. Amodal instance segmentation (AIS), aiming to predict a complete mask of an occluded target, is a promising solution. However, AIS usually requires amodal datasets, which are challenging to create and limit the application of AIS. To solve this problem, we proposed a novel semi-supervised generative adversarial network (GAN) for AIS, denoted “the AISGAN”. Our AISGAN only requires a regular modal dataset and generate amodal samples by random occlusions, making the AIS method more applicable. A corresponding segmentation loss was added to overcome mode collapse of GAN. The results showed that the AISGAN achieved a mean Intersection of Union (mIoU) of 0.823 and outperformed the mIoUs of Mask RCNN, Raw, and Convex Hull (0.801, 0.780, and 0.778, respectively). As a semi-supervised method, the mIoU of our AISGAN was further enhanced (by 0.6%) when we fine-tuned it with unlabeled new data, showing its extensibility to new unseen scenarios. The visualization demonstrates that the AISGAN can produce realistic masks of piglets, including details of their noses and legs, even under heavily occluded conditions. With the AISGAN, we achieved an occlusion-resistant spatial distribution analysis of the piglets in farrowing pens. Thus, the AISGAN is a promising tool to manage occlusion problems for automated animal monitoring in complex housing environments.