A novel lameness detection method for dairy cows based on temporal gait and spatial post features

Abstract

Lameness is one of the common diseases in dairy farms, which seriously affects the health and welfare of dairy cows. The existing lameness detection methods based on computer vision have poor ability to extract spatiotemporal features, which reduces the lameness detection accuracy in dairy cows. In this paper, a spatiotemporal energy network (SEN) is proposed to detect lameness in dairy cows. It compresses the cow walking video into history energy image (HEI) and gait energy image (GEI), and then accurately extracts the temporal gait and spatial pose features of dairy cows. First, the YOLOv8n algorithm is used to remove background information from the video and segment the cow’s body. Second, the body contour sequence of the dairy cow is stacked based on the time series, and then different body parts of dairy cows are stacked on the channel to form the HEI. The HEI preserves the movement trajectory of the dairy cows. The first lameness classification model is obtained by training the MobileNetv2 network with HEI dataset. Third, the body contour sequence of the dairy cow is cropped according to the center of mass and body proportion and then stacked into an image, which is named GEI. The second lameness classification model is obtained by training the MobileNetv2 network with GEI dataset. Finally, the prediction results of the two models are fused according to the weight ratio to detect lameness in dairy cows. To verify the lameness detection effect of this method, 264 samples were randomly selected from 330 samples as the training set and the remaining 66 samples as the test set. The proposed method obtained competitive lameness detection results with classification accuracy of 96.22 %, sensitivity of 96.35 % and specificity of 98.14 %. By combining HEI and GEI, the spatiotemporal features of the cow walking are extracted more comprehensively. This method provides guidance for the lameness detection of dairy cows and reference for the gait analysis of animals.