Assessment of spatial learning abilities is central to behavioral neuroscience and a useful tool for animal model validation and drug development. However, biases introduced by the apparatus, environment, or experimentalist represent a critical challenge to the test validity. We have recently developed the Modified Barnes Maze (MBM) task, a spatial learning paradigm that overcomes inherent behavioral biases of animals in the classical Barnes maze. The specific combination of spatial strategies employed by mice is often considered representative of the level of cognitive resources used. Herein, we have developed a convolutional neural network-based classifier of exploration strategies in the MBM that can effectively provide researchers with enhanced insights into cognitive traits in mice. Following validation, we compared the learning performance of female and male C57BL/6J mice, as well as that of Ts65Dn mice, a model of Down syndrome, and 5xFAD mice, a model of Alzheimer’s disease. Male mice exhibited more effective navigation abilities than female mice, reflected in higher utilization of effective spatial search strategies. Compared to wildtype controls, Ts65Dn mice exhibited delayed usage of spatial strategies despite similar success rates in completing this spatial task. 5xFAD mice showed increased usage of non-spatial strategies such as Circling that corresponded to higher latency to reach the target and lower success rate. These data exemplify the need for deeper strategy classification tools in dissecting complex cognitive traits. In sum, we provide a machine-learning-based strategy classifier that extends our understanding of mice’s spatial learning capabilities while enabling a more accurate cognitive assessment.