A comparison of machine learning and logistic regression in modelling the association of body condition score and submission rate

Abstract

The effect of body condition score (BCS) on reproductive outcomes is complex, dynamic and non-linear with interaction and confounding. The flexibility inherent in machine learning algorithms makes them attractive for analysing complex data. This study was designed to compare the ability of a range of machine learning techniques in estimating the probability of service within 21 days of the planned start of mating. We hypothesised that if there were complex and unknown interactions or non-linearity in the data, some machine learning algorithms would result in superior model performance compared to regression models.For a period of six months from the planned start of calving, BCS was visually assessed once a month for 6127 cows on 8 commercial New Zealand dairy farms by a trained veterinarian using the DairyNZ 10-point range for every cow in the herd. Cow, lactation and reproductive data was extracted from the national herd database. This data was used to predict probability of service within 21 days of planned start of mating (PSM) using mixed multivariable logistic regression and decision tree, k-nearest neighbour, random forest and neural network analysis.Models were adjusted for herd, cow age, breed, days in milk, BCS at calving, BCS change between calving and mating, BCS change after mating, volume adjusted milk protein and fat concentration pre-mating. Models were constructed on a training data set using 10-fold cross validation repeated 10 times and evaluated on a test data set using discrimination and calibration techniques.In all models, days calved at PSM was the most important variable for predicting submission rate, followed by BCS at PSM. Factors associated with an increased probability of insemination were calving at a BCS of 5.0, losing less BCS after calving, having a higher BCS at nadir, losing BCS rapidly after calving, nadir occurring before PSM and calving early. All the models except for the decision tree had an area under the receiver operating characteristic curve (AUC) in the range 0.68-0.73 indicating good overall discriminatory power, but calibration analysis suggested all models were better at predicting cows that got inseminated than correctly identifying animals that did not get inseminated. Overall, the machine learning techniques were no better than a generalised logistic regression model.These results highlight the importance of BCS targets at calving and indicate BCS loss, milk characteristics and days calves may be useful indicators identifying cows at risk of poor reproductive outcomes.