Detection of the respiratory rate of standing cows by combining the Deeplab V3+ semantic segmentation model with the phase-based video magnification algorithm

Abstract

The respiratory rate (RR) of dairy cows is closely related to heat stress and reproductive diseases. In this study, a method was proposed to detect the RR of standing resting dairy cows by using deep learning and video analysis. First, a Deeplab V3+ semantic segmentation model was developed using the framework of ResNet-101, and the model was trained and tested using 3,000 manually labelled images to realise the segmentation of the cows. Next, a phase-based video magnification (PBVM) algorithm was applied to amplify the weak breathing movements of the cow. Finally, the Lucas-Kanade (LK) optical flow algorithm was used to detect breathing direction, and an RR detection model was used to detect the RR. The effectiveness of the method was verified by testing a total of 67,812 frames of data collected from 70 video segments of different cows. A comparison with the original video LK optical flow method (Method 1), target segmentation video LK optical flow method (Method 2), and original video motion amplification LK optical flow method (Method 3) showed that the accuracy of the proposed method was between 80.00% and 100.00%, and the mean accuracy was 93.04%. These values were 45.82%, 17.72%, and 22.94% higher than those of the other three methods, respectively. In addition, the method was tested in 105 videos of cows under heat stressed, and the RR detection accuracy was 98.69%. The results showed that the method could provide a reference for the automatic detection of RR and remote diagnosis of respiratory-related diseases.