Cows' legs tracking and lameness detection in dairy cattle using video analysis and Siamese neural networks

Abstract

The analysis of a cow’s motion behavior is based on the positioning and tracking of its legs. This paper proposes a Siamese attention model (Siam-AM) fusion attention mechanism to obtain the automatic tracking and monitoring of cow legs in large-scale farms. Firstly, the features of the cow’s leg are extracted using the first frame of motion image data. The search area in the subsequent frames is also input into the network to extract additional features. Then an attention module is used to assign weights to the features. The similarity between different regions and image features of the first frame was compared. The image with the most significant similarity was taken as the predicted position of the cow’s leg. Finally, the relative step size of the cow’s front and rear legs is calculated using the leg’s position coordinates. A Support Vector Machine (SVM) classifier is constructed to realize the lameness detection of the cow. The experimental results show that the proposed Siam-AM algorithm has an average tracking accuracy of 93.80 %, an average frame rate of 57 fps, and a lameness detection accuracy of 94.73 %. These results provide an effective method for accurate tracking and lameness detection of a cow’s leg.