Leveraging computer vision-based pose estimation technique in dairy cows for objective mobility analysis and scoring system

Abstract

This study investigated the feasibility of utilizing a computer vision-based pose estimation technique for quantitative mobility analysis in dairy cows, specifically focusing on commonly used variables in visual mobility scoring. Additionally, the study determined the potential of a machine learning classification algorithm to predict mobility scores based on data obtained from the aforementioned pose estimation technique. A dataset comprising 204 individual cows' video clips was collected, with each video clip recorded from a side-view perspective during walking. The cows were scored using a 4-level mobility scoring system: Score 0 (good mobility: 64 cows), Score 1 (imperfect mobility: 65 cows), Score 2 (impaired mobility: 57 cows), and Score 3 (severely impaired mobility: 18 cows). The video clips were analyzed using a software for cattle pose estimation, capable of detecting 25 keypoints and generating time-series XY-coordinates of those keypoints. Based on the data, a total of 17 mobility variables were calculated, such as head bob, stride length, stride duration, walking speed, back angle, and range of motion in leg joints. The measurements of these variables closely align with previously reported and comparable data derived from precise sensing technologies (e.g., walkway pressure mapping systems) and labor-intensive techniques (e.g., attaching markers to cows and manually annotating on sequential images). The relationships between these measurements and the mobility scores were also consistent with the findings reported before. To account for the limited number of cows classified as Score3, the cows classified as Score 2 and Score 3 were merged into a single class, and a classification model for the 3-level mobility score (Score 0, 1, and 2 + 3) was developed using a random forest algorithm. The model's performance was evaluated using a repeated holdout data split method. In this process, the dataset was randomly divided into an 80 % training set and a 20 % test set, and this was replicated ten times to ensure a robust assessment of the model's predictive ability. The overall 3-class classification performance of the model resulted in a weighted kappa coefficient of 0.69 and area under the curve of the receiver operating characteristic curve of 0.86. The findings suggest that computer vision-based pose estimation technique can be utilized for quantitative mobility analysis, and the machine learning model based on the data derived from the pose estimation technique has the potential for objective mobility scoring in dairy cows.