InsectNet: Real-time identification of insects using an end-to-end machine learning pipeline

Abstract

Listen

Translate article

Abstract Insect pests significantly impact global agricultural productivity and crop quality. Effective integrated pest management strategies require the identification of insects, including beneficial and harmful insects. Automated identification of insects under real-world conditions presents several challenges, including the need to handle intraspecies dissimilarity and interspecies similarity, life-cycle stages, camouflage, diverse imaging conditions, and variability in insect orientation. An end-to-end approach for training deep-learning models, InsectNet, is proposed to address these challenges. Our approach has the following key features: (a) uses a large dataset of insect images collected through citizen science along with label-free self-supervised learning to train a global model; (b) fine-tuning this global model using smaller, expert verified regional datasets to create a local insect identification model; (c) which provides high prediction accuracy even for species with small sample sizes, (d) is designed to enhance model trustworthiness, and (e) democratizes access through streamlined machine learning operations (MLOps). This global-to-local model strategy offers a more scalable and economically viable solution for implementing advanced insect identification systems across diverse agricultural ecosystems. We report accurate identification (>96% accuracy) of numerous agriculturally and ecologically relevant insect species, including pollinators (e.g., butterflies, bees), parasitoids (e.g., some wasps and flies), predators (e.g., lady beetles and mantises), and harmful insects (e.g., armyworms, cutworms, grasshoppers, stink bugs). InsectNet provides fine-grained insect species identification, works effectively in challenging backgrounds, and avoids making predictions when uncertain, increasing its utility and trustworthiness. The model and associated workflows are available through a web-based portal accessible through a computer or mobile device. We envision InsectNet to complement existing approaches, and be part of a growing suite of AI technologies for addressing agricultural challenges.