Detection of Surface and Subsurface Defects of Apples Using Structured- Illumination Reflectance Imaging with Machine Learning Algorithms

Abstract

Abstract. Machine vision technology coupled with uniform illumination is now widely used for automatic sorting and grading of apples and other fruits, but it still does not have satisfactory performance for defect detection because of the large variety of defects, some of which are difficult to detect under uniform illumination. Structured-illumination reflectance imaging (SIRI) offers a new modality for imaging by using sinusoidally modulated structured illumination to obtain two sets of independent images: direct component (DC), which corresponds to conventional uniform illumination, and amplitude component (AC), which is unique for structured illumination. The objective of this study was to develop machine learning classification algorithms using DC and AC images and their combinations for enhanced detection of surface and subsurface defects of apples. A multispectral SIRI system with two phase-shifted sinusoidal illumination patterns was used to acquire images of ‘Delicious’ and ‘Golden Delicious’ apples with various types of surface and subsurface defects. DC and AC images were extracted through demodulation of the acquired images and were then enhanced using fast bi-dimensional empirical mode decomposition and subsequent image reconstruction. Defect detection algorithms were developed using random forest (RF), support vector machine (SVM), and convolutional neural network (CNN), for DC, AC, and ratio (AC divided by DC) images and their combinations. Results showed that AC images were superior to DC images for detecting subsurface defects, DC images were overall better than AC images for detecting surface defects, and ratio images were comparable to, or better than, DC and AC images for defect detection. The ensemble of DC, AC, and ratio images resulted in significantly better detection accuracies over using them individually. Among the three classifiers, CNN performed the best, with 98% detection accuracies for both varieties of apples, followed by SVM and RF. This research demonstrated that SIRI, coupled with a machine learning algorithm, can be a new, versatile, and effective modality for fruit defect detection. Keywords: Apple, Defect, Bi-dimensional empirical mode decomposition, Machine learning, Structured illumination.