Globally, the agricultural industry has benefited from using pesticides to minimize crop losses. Nevertheless, the indiscriminate overuse of pesticides has led to significant risks associated with a detrimental impact on the environment and human health. Therefore, emerging concerns of pesticide residue found in crops, food, and livestock are a pressing issue. To address the above challenges, there have been many efforts made towards implementing machine learning to enable precision agricultural practices to reduce pesticide overuse. As of today, there are no guiding digital tools available for citrus growers to provide pesticide residue leaf coverage analysis after foliar applications. Herein, we are the first to report software assisted by lightweight machine learning (ML) to determine the Kocide 3000 and Oxytetracycline (OTC) residue coverage on citrus leaves based on image data analysis. This tool integrates a foundational Segment Anything Model (SAM) for image preprocessing to isolate the area of interest. In addition, Kocide 3000 and Oxytetracycline (OTC) residue coverage analysis was carried out using a specialized Mask Region-Based Convolutional Neural Network (CNN). This CNN was pre-trained on the MS COCO dataset and fine-tuned by training with acquired datasets in laboratory and field conditions. The developed software demonstrated excellent performance on both pesticides’ accuracy, precision, and recall, and F1 score metrics. In summary, this tool has the potential to assist growers with the decision-making process for controlling pesticide use rate and frequency, minimizing pesticide overuse.