We trained machine learning models to identify intramammary infections (IMI) in late lactation cows at dry-off to guide antibiotic treatment, and compared their performance to a rule-based algorithm that is currently used on dairy farms in the US. We conducted an observational test-characteristics study using a data set of 3,645 cows approaching dry-off from 68 US dairy herds. The outcome variables of interest were cow-level IMI caused by all pathogens, major pathogens, and Streptococcus and Strep-like organisms (SSLO), which were determined using aerobic culture of aseptic quarter-milk samples and identification of isolates using MALDI-TOF. Individual cow records were extracted from the farm software to create 53 feature variables at the cow and 39 at the herd-level which were derived from cow-level descriptive data, records of clinical mastitis events, results from routine testing of milk for volume and concentrations of somatic cell count (SCC), fat, and protein. ML algorithms evaluated were logistic regression, decision tree, random forest, light gradient-boosting machine, naïve bayes, and neural networks. For comparison, cows were also classified according to a conventional rule-based algorithm that considered a cow as high risk for IMI if she had at one or more high SCC (>200,000 cells/ml) tests or ≥2 cases of clinical mastitis during the lactation of enrollment. Area under the curve (AUC) and Youden's index were used to compare models, in addition to binary classification metrics, including sensitivity, specificity, and predictive values. ML models had slightly higher AUC and Youden's index values than the rule-based algorithm for all IMI outcomes of interest. However, these improvements in prediction accuracy were substantially less than what we had considered necessary for the technology to be a worthwhile alternative to the rule-based algorithm. Therefore, evidence is lacking to support the wholesale use of ML-guided selective dry cow therapy at the moment. We recommend that producers wanting to implement algorithm-guided SDCT use a rule-based method.