Efficacy of machine learning image classification for automated occupancy‐based monitoring

Abstract

AbstractRemote cameras have become a widespread data‐collection tool for terrestrial mammals, but classifying images can be labor intensive and limit the usefulness of cameras for broad‐scale population monitoring. Machine learning algorithms for automated image classification can expedite data processing, but image misclassifications may influence inferences. Here, we used camera data for three sympatric species with disparate body sizes and life histories – black‐tailed jackrabbits (Lepus californicus), kit foxes (Vulpes macrotis), and pronghorns (Antilocapra americana) – as a model system to evaluate the influence of competing image classification approaches on estimates of occupancy and inferences about space use. We classified images with: (i) single review (manual), (ii) double review (manual by two observers), (iii) an automated‐manual review (machine learning to cull empty images and single review of remaining images), (iv) a pretrained machine‐learning algorithm that classifies images to species (base model), (v) the base model accepting only classifications with ≥95% confidence, (vi) the base model trained with regional images (trained model), and (vii) the trained model accepting only classifications with ≥95% confidence. We compared species‐specific results from alternative approaches to results from double review, which reduces the potential for misclassifications and was assumed to be the best approximation of truth. Despite high classification success, species‐level misclassification rates for the base and trained models were sufficiently high to produce erroneous occupancy estimates and inferences related to space use across species. Increasing the confidence thresholds for image classification to 95% did not consistently improve performance. Classifying images as empty (or not) offered a reasonable approach to reduce effort (by 97.7%) and facilitated a semi‐automated workflow that produced reliable estimates and inferences. Thus, camera‐based monitoring combined with machine learning algorithms for image classification could facilitate monitoring with limited manual image classification.