An automatic recognition framework for sow daily behaviours based on motion and image analyses

Abstract

The aim of this study was to develop a general and automatic recognition framework for recognising the daily behaviours of lactating sows to save manual labour and promote smart management. The proposed framework used both image analysis techniques in still image and motion analysis techniques in spatiotemporal videos to recognise sow drinking, feeding, nursing, moving, medium active and inactive behaviours in a loose pen. The image analysis techniques, which are based on fully convolutional networks (FCNs) for high-accuracy segmentation, were used to extract spatial features that evaluated the spatial relationships between objects and the appearance of sows. The motion analysis techniques in spatiotemporal videos, which are based on optical flow analysis and changes in the animal centroid, were used to extract temporal features that evaluated the temporal motions of the animals. In the recognition process, these spatial and temporal features were input into a hierarchical classifier for behaviour recognition. The final recognition results were obtained by a temporal-correlation-based correction module for promoting the recognition rate. Testing on 26 h of videos (468,000 frames) of 3 sows, the algorithm realised the following accuracies of behavioural classification compared with the manual observations: 97.49% for drinking, 95.36% for feeding, and 88.09% for nursing. In addition, the amounts of time that the sow spent on the considered behaviours in the daytime (from 8:00 to 17:00), were as follows: 69.34% on inactive behaviours, 14.50% on nursing, 8.38% on medium activities, 4.04% on feeding, 2.26% on drinking and 1.48% on moving. Hence, the proposed method provides an effective approach for the automatic recognition of sow behaviours from video sequences, which facilitates the pig farmer in improving livestock-farming management.