Automated Video Behavior Recognition of Pigs Using Two-Stream Convolutional Networks.

Jiayun Huang,Dan Li,Yifei Chen,Kaifeng Zhang

doi:10.3390/s20041085

Abstract

The detection of pig behavior helps detect abnormal conditions such as diseases and dangerous movements in a timely and effective manner, which plays an important role in ensuring the health and well-being of pigs. Monitoring pig behavior by staff is time consuming, subjective, and impractical. Therefore, there is an urgent need to implement methods for identifying pig behavior automatically. In recent years, deep learning has been gradually applied to the study of pig behavior recognition. Existing studies judge the behavior of the pig only based on the posture of the pig in a still image frame, without considering the motion information of the behavior. However, optical flow can well reflect the motion information. Thus, this study took image frames and optical flow from videos as two-stream input objects to fully extract the temporal and spatial behavioral characteristics. Two-stream convolutional network models based on deep learning were proposed, including inflated 3D convnet (I3D) and temporal segment networks (TSN) whose feature extraction network is Residual Network (ResNet) or the Inception architecture (e.g., Inception with Batch Normalization (BN-Inception), InceptionV3, InceptionV4, or InceptionResNetV2) to achieve pig behavior recognition. A standard pig video behavior dataset that included 1000 videos of feeding, lying, walking, scratching and mounting from five kinds of different behavioral actions of pigs under natural conditions was created. The dataset was used to train and test the proposed models, and a series of comparative experiments were conducted. The experimental results showed that the TSN model whose feature extraction network was ResNet101 was able to recognize pig feeding, lying, walking, scratching, and mounting behaviors with a higher average of 98.99%, and the average recognition time of each video was 0.3163 s. The TSN model (ResNet101) is superior to the other models in solving the task of pig behavior recognition.

Highlights

Pig behavior reflects the animal’s welfare status, well-being conditions, and social interactions [1,2].Appropriate feeding behavior can ensure the healthy growth of pigs and help determine their food intake
The dataset was divided into a training set and test set in a 4:1 ratio randomly in a non-overlapping manner
The stochastic selected 227 test samples were input into the inflated 3D convnet (I3D) and temporal segment networks (TSN) networks, which were trained by the training set, and we obtained the accuracy of each category of the videos

Summary

Introduction

Pig behavior reflects the animal’s welfare status, well-being conditions, and social interactions [1,2]. Appropriate feeding behavior can ensure the healthy growth of pigs and help determine their food intake. Reduction in food intake means that pig health and welfare are compromised [3,4] and can be considered as a signal for alarming suspected cases [1]. Walking and lying behaviors can reflect the activity level of pigs. Pigs generally reduce activity, posture in protective positions, and increase lying duration [5,6]. The timely detection and intervention of mounting behavior can increase animal welfare and further ensure pig health [10]. The infection of skin diseases can be evaluated by Sensors 2020, 20, 1085; doi:10.3390/s20041085 www.mdpi.com/journal/sensors

Results

Discussion

Conclusion