Structured-illumination reflectance imaging combined with deep learning for detecting early decayed oranges

Abstract

Fungal rot is the most serious defect in post-harvest citrus, and the timely detection and removal of early rotten citrus are particularly crucial in reducing economic losses. Imaging technology under structured illumination is a promising method to fruit surface defects. This study aims to assess the feasibility of combining the structured-illumination reflectance imaging (SIRI) system with deep learning technology for the identification and segmentation of early rot in citrus. Phase-shifted images of oranges were acquired at four spatial frequencies and demodulated to obtain direct component (DC) and amplitude component (AC) images. The optimal spatial frequency for detecting rot was determined to be 0.20 cycles mm−1 based on the contrast index between the decay and sound areas in the images. Then, the AC images are subjected to brightness correction and augmentation. Three segmentation methods, global thresholding, watershed segmentation and Unet were used to segment the rotten areas in the images. Unet achieved optimal results on AC images, with an overall accuracy of 99.4 % and an IoU of 0.903. The gradient-weighted class activation mapping (Grad-CAM) was used to visualize the areas recognized by Unet for orange rot, yielding satisfactory results. This study effectively demonstrated the defect recognition capability of SIRI combined with deep learning and providing a reliable solution for early decayed detection in oranges.