Structured-illumination Reflectance Imaging Research Articles

Construction of a stable YOLOv8 classification model for apple bruising detection based on physicochemical property analysis and structured-illumination reflectance imaging

Effective and accurate detection of bruises at all stages has always been a challenge in non-destructive grading of apples. In this study, the visible structured-illumination reflectance imaging (SIRI) combing with deep learning method was proposed to identify bruised ‘Fuji’ apples at four different time stages (0, 6, 12 and 24 h). The macroscopic/microscopic structures and physicochemical properties of bruised tissue were measured and analyzed to determine the relationship between bruising time and these properties, as well as how they affect the accuracy of bruising detection. Results indicated that classification accuracy increased with the decrease of water and total phenolic content of the bruised tissue, as well as with the increase of color browning and bruised area. The YOLOv8 model achieved the highest detection accuracy (99.5 %) and stability. This research enhances understanding of apple bruise optics and aids in developing advanced nondestructive testing techniques.

Structured-illumination reflectance imaging combined with deep learning for detecting early decayed oranges

Fungal rot is the most serious defect in post-harvest citrus, and the timely detection and removal of early rotten citrus are particularly crucial in reducing economic losses. Imaging technology under structured illumination is a promising method to fruit surface defects. This study aims to assess the feasibility of combining the structured-illumination reflectance imaging (SIRI) system with deep learning technology for the identification and segmentation of early rot in citrus. Phase-shifted images of oranges were acquired at four spatial frequencies and demodulated to obtain direct component (DC) and amplitude component (AC) images. The optimal spatial frequency for detecting rot was determined to be 0.20 cycles mm−1 based on the contrast index between the decay and sound areas in the images. Then, the AC images are subjected to brightness correction and augmentation. Three segmentation methods, global thresholding, watershed segmentation and Unet were used to segment the rotten areas in the images. Unet achieved optimal results on AC images, with an overall accuracy of 99.4 % and an IoU of 0.903. The gradient-weighted class activation mapping (Grad-CAM) was used to visualize the areas recognized by Unet for orange rot, yielding satisfactory results. This study effectively demonstrated the defect recognition capability of SIRI combined with deep learning and providing a reliable solution for early decayed detection in oranges.

Structured-illumination reflectance imaging for the evaluation of microorganism contamination in pork: Effects of spectral and imaging features on its prediction performance

Structured-illumination reflectance imaging for the evaluation of microorganism contamination in pork: Effects of spectral and imaging features on its prediction performance

Non-Destructive Assessment of Microbial Spoilage of Broiler Breast Meat Using Structured Illumination Reflectance Imaging with Machine Learning

Non-Destructive Assessment of Microbial Spoilage of Broiler Breast Meat Using Structured Illumination Reflectance Imaging with Machine Learning

Beef marbling assessment by structured-illumination reflectance imaging with deep learning

Beef marbling is primarily assessed in terms of the abundance and spatial distribution of visible fat flecks within Longissimus Dorsi (LD) muscle. Visual appraisal by trained professionals is the currently prevailing practice in beef marbling assessment, but it is subjective, labor-intensive, and time-consuming. Structured-Illumination Reflectance Imaging (SIRI) offers a potentially useful approach to meat quality assessment as opposed to conventional imaging techniques using uniform illumination. This study presents a new evaluation of a custom-assembled, broadband SIRI system, combined with Deep Learning (DL) approaches, for beef marbling assessment. Beef samples of varying marbling degrees were imaged under sinusoidal illumination at spatial frequencies of 0.05–0.40 cycles/mm. The acquired images were demodulated into Direct Component (DC) and Amplitude Component (AC) images at each spatial frequency. Three DL-based segmentation models, including Unet++, DeepLabv3+, and SegFormer, were built using DC and AC (0.05–0.40 cycles/mm) images for segmenting the LD muscle from the surrounding backfat. Two scenarios of image input, i.e., 1) original beef images without backfat removal and 2) segmented LD muscle images, were examined for feature extraction using a pre-trained ResNeXt-101 model, followed by discriminant modeling to classify beef samples of three marbling categories. SegFormer achieved better LD segmentation than Unet++ and DeepLabv3+ for both DC and AC images, with the best IoU (Intersection over Union) of 96.8% obtained at 0.10 cycles/mm. For marbling classification, the models based on LD segments (scenario 1) consistently produced higher accuracies than those using the original beef images (scenario 2). With the deep features extracted from LD segments, DC images yielded an overall classification accuracy of 83.19%, while the AC images resulted in better accuracies of 85.96%–88.72% (P<0.05), with the greatest improvement of 5.53% (P<0.01) at 0.40 cycles/mm. A further improved accuracy of 90.85% in the marbling classification was achieved by modeling a reduced set of relevant features identified based on MRMR (Minimum Redundancy Maximum Relevance). This study has shown the effectiveness of SIRI for beef marbling assessment.

Early bruising detection of 'Korla' pears by low-cost visible-LED structured-illumination reflectance imaging and feature-based classification models.

Nondestructive detection of thin-skinned fruit bruising is one of the main challenges in the automated grading of post-harvest fruit. The structured-illumination reflectance imaging (SIRI) is an emerging optical technique with the potential for detection of bruises. This study presented the pioneering application of low-cost visible-LED SIRI for detecting early subcutaneous bruises in 'Korla' pears. Three types of bruising degrees (mild, moderate and severe) and ten sets of spatial frequencies (50, 100, 150, 200, 250, 300, 350, 400, 450 and 500 cycles m-1) were analyzed. By evaluation of contrast index (CI) values, 150 cycles m-1 was determined as the optimal spatial frequency. The sinusoidal pattern images were demodulated to get the DC, AC, and RT images without any stripe information. Based on AC and RT images, texture features were extracted and the LS-SVM, PLS-DA and KNN classification models combined the optimized features were developed for the detection of 'Korla' pears with varying degrees of bruising. It was found that RT images consistently outperformed AC images regardless of type of model, and LS-SVM model exhibited the highest detection accuracy and stability. Across mild, moderate, severe and mixed bruises, the LS-SVM model with RT images achieved classification accuracies of 98.6%, 98.9%, 98.5%, and 98.8%, respectively. This study showed that visible-LED SIRI technique could effectively detect early bruising of 'Korla' pears, providing a valuable reference for using low-cost visible LED SIRI to detect fruit damage.

Identification of early decayed oranges using structured-illumination reflectance imaging coupled with fast demodulation and improved image processing algorithms

Effective detection of decayed oranges (Citrus genus) at the early stage is challenging because there are no or few visual symptoms on the infected fruit. Structured-illumination reflectance imaging (SIRI) has been proven effective for enhanced detection of subsurface defects in fruit. Amplitude component (AC) images retrieved from the original SIRI patterned images are useful for defect detection, but generally require acquiring three phase-shifted pattern images, which limits the imaging and detection speed. Moreover, the AC images may also suffer from noticeable uneven brightness due to fruit curvature, which affects the identification of decayed areas. This study was therefore aimed to explore faster image demodulation, enhancement and processing algorithms, based on a specially developed SIRI technology, for effective identification of early decayed oranges. Pattern images were acquired, using three phase-shifted sinusoidal illumination patterns at the wavelength of 810 nm and a spatial frequency of 0.20 cycles mm−1, from the orange samples infected with Penicillium digitatum fungus, the most serious and devastating pathogen for orange fruit. Two-dimensional spiral phase transform was used to obtain AC images from one or two pattern images. The acquired AC images were then processed by using simple brightness adjustment and integral image-based fast average filtering for brightness correction, and improved watershed algorithm and global threshold for segmentation of decayed areas. Three different combinations of these image processing procedures for single and two pattern images were proposed to distinguish decayed oranges from sound ones. The three methodologies all achieved high overall identification rates of 97.5%, 95.0% and 95.3%, when the stem-end effect was also considered. This study showed that accurate detection of early decayed orange fruit can be achieved by using one or two phase-shifted pattern images, which would be beneficial for real-time implementation of the technique.

Feasibility of imaging under structured illumination for evaluation of white striping in broiler breast fillets

Muscle myopathies or defects, such as white striping (WS), downgrade the quality of broiler meat, leading to the loss of millions of dollars annually for the U.S. poultry industry. Visual inspection is currently practiced for assessing myopathies in broiler meat, which is however prone to human evaluation error, labor-intensive, and costly. Imaging under uniform or diffuse illumination has been investigated for assessing broiler meat quality, but their performance is not always satisfactory, especially for detecting subtle defects with few visual symptoms. This study presents the first proof-of-concept evaluation of the utility of an innovative technique for improved imaging of structures of biological materials, i.e., structured-illumination reflectance imaging (SIRI), for detecting WS conditions in broiler breast meat. Broiler breast fillets with varying degrees of WS severity were imaged by an in-house assembled SIRI system under sinusoidal illumination at different spatial frequencies (0.05–0.40 cycles/mm). The acquired pattern images were demodulated into direct components (DC) and amplitude components (AC) at each spatial frequency. Texture features were extracted from DC and AC images for building classification models using regularized linear discriminant analysis, which were to classify meat samples into 2, 3, and 4 classes, respectively, according to WS conditions. AC images were found by visual inspection more effective than DC in resolving WS characteristics, depending on the spatial frequency of illumination patterns. Compared to DC, AC yielded the maximum accuracy improvements of 8.33%, 7.93%, and 12.3% in 2-, 3- and 4-class classifications, respectively. Models using ranked features yielded similar improvements of up to 9.53% by AC over DC. The SIRI technique is promising for enhanced detection of WS and potentially other defects of broiler breast meat. • Structured-illumination reflectance imaging (SIRI) was used for poultry quality detection. • Amplitude component (AC) images enabled resolving white striping (WS) of broiler breast meat. • Broiler breast fillets were classified by models into 2, 3 and 4 classes according to WS conditions. • SIRI offered substantial improvements of up to 12% in classifying WS-affected poultry samples.

Non-destructive Assessment of White Striping in Broiler Breast Meat Using Structured-Illumination Reflectance Imaging with Deep Learning

Highlights Broiler breast meat with white striping (WS) was imaged under sinusoidally modulated structured illumination. Amplitude component (AC) images resolved better WS characteristics than direct component (DC) images. Models built on deep features classified meat samples into 2 and 3 classes according to WS severity. The AC images consistently outperformed the DC images and achieved the best accuracy of 96.6% in 2-class modeling. Abstract. Visual inspection is the prevailing practice in the industry for assessing white striping (WS) in broiler breast meat. However, this approach is subjective, laborious, and prone to error. Several studies have utilized imaging technology under uniform illumination; however, detecting defects, such as WS, remains challenging. This study investigated the efficacy of the emerging structured illumination reflectance imaging (SIRI) combined with deep learning (DL) for the WS assessment of broiler meat. Broiler fillets with varying degrees of WS were imaged using a custom-assembled monochromatic SIRI system (0.05-0.40 cycles/mm). The acquired SIRI pattern images were demodulated into direct components (DC) and amplitude components (AC) at each spatial frequency. Pre-trained DL models, including two Very Deep Convolutional Networks (VGG16 and VGG19) and two Densely Connected Convolutional Neural Networks (DenseNet121 and DensetNet201), were evaluated as fine-tuned end-to-end classifiers and feature extractors separately to differentiate the meat samples. Fine-tuned VGG16 resulted in the best 2-class and 3-class classification accuracies of 94.5% and 74.6%, respectively, based on the AC images at 0.30 cycles/mm, improving over the accuracies of the corresponding DC images by 10.4% and 8.9%, respectively. Fine-tuned VGG19 and DenseNet201 also achieved substantial improvements of AC over DC images by 12% or higher. Linear discriminant analysis, in conjunction with principal component analysis for dimension reduction, yielded better accuracies of 96.6% in 2-class and 83.4% in 3-class classification, based on the deep features extracted by DenseNet121 from the AC images at 0.40 cycles/mm and 0.30 cycles/mm, respectively, representing improvements of 5.0% and 2.9% based on the features of the corresponding DC images. The SIRI technique combined with DL is effective for differentiating between normal and WS-affected broiler meat. Keywords: Deep learning, Imaging, Meat quality, Poultry, Structured illumination, White striping.

Identification of Chilling Injury in Kiwifruit Using Hyperspectral Structured-Illumination Reflectance Imaging System (SIRI) with Support Vector Machine (SVM) Modelling

Accurate detection of chilling injury in kiwifruit is challenging because the symptoms are mainly manifested in the interior. This work reports a method for detecting the chilling injury of ‘Hongyang’ kiwifruit to provide nondestructive discrimination. Kiwifruit samples with varying levels of chilling injury were analyzed by a hyperspectral structured-illumination reflectance imaging (SIRI) system. After demodulation, direct current (DC) and alternating current (AC) images with spatial frequencies of 30, 60, and 120 m−1 were obtained and labeled as F30, F60, and F120. Predictive models were developed to optimize the preprocessing and modeling methods. Prediction models established the results of DC and AC with different spatial frequencies and were compared. The autoscale-support vector machine (SVM) models were optimal for AC at different spatial frequencies, and the multiplicative scatter correction (MSC)-SVM model was optimal for DC. The combined features of F30, F60, and F120, as well as the spectral features of DC, had better accuracy for classifying chilling injury. The optimal model of hyperspectral SIRI system for detecting chilling injury was the F30 based on combined features, with calibration accuracy of 98.1% and prediction accuracy of 94.2%. This study has shown that structured illumination had higher accuracy than uniform illumination in predicting chilling injury. Further, this approach allows the identification of kiwifruit with chilling injury using a hyperspectral structured-illumination reflectance imaging system.

Detection of early decay in navel oranges by structured-illumination reflectance imaging combined with image enhancement and segmentation

Early detection and removal of decayed fruit is critical for reducing product and economic losses for the fruit industry. It is, however, challenging to detect tissue decay in the early stage, which has no or few visual symptoms. This study was intended to evaluate the feasibility of an emerging structured-illumination reflectance imaging (SIRI) technique for early detection of decay in navel oranges. Pattern images were acquired from navel oranges five days after the fruit had been inoculated with fungal spores of Penicillium digitatum (P. digitatum) (the most important pathogen responsible for citrus decay), using an inhouse assembled multispectral imaging platform under phase-shifted sinusoidal illumination at four spatial frequencies (0.05, 0.10, 0.15 and 0.20 cycles mm−1) and three wavelengths (690, 730 and 810 nm). The pattern images were processed through demodulation to obtain direct component (DC) and amplitude component (AC) images for each wavelength and spatial frequency. The AC images were able to reveal decayed areas in the fruit. The wavelength of 810 nm and a spatial frequency of 0.20 cycles mm−1 were determined to be optimal for decayed tissue detection, based on quantitative assessment of the contrast between decayed and sound tissues in images. Moreover, the ratio image (RDC/AC) between the AC and DC images and the corrected ratio image (Rc), which was obtained by applying the image brightness transform to RDC/AC, have showed improved defect contrast and background uniformity. Three image segmentation methods including watershed, Otsu and global thresholding were applied for segmenting decayed areas in the AC, RDC/AC and RC images. The RC images produced the best detection performance with an overall detection accuracy of over 97 %. This study has demonstrated that when combined with appropriate image processing methods, the SIRI technique is effective for the early detection of decay in citrus fruit.

Detection of early decayed oranges by structured-illumination reflectance imaging coupling with texture feature classification models.

Citrus fruits are susceptible to fungal infection after harvest. To reduce the economic loss, it is necessary to reject the infected citrus fruit before storage and transportation. However, the infected area in the early stage of decay is almost invisible on the fruit surface, so the detection of early decayed citrus is very challenging. In this study, a structured-illumination reflectance imaging (SIRI) system combined with a visible light-emitting diode (LED) lamp and a monochrome camera was developed to detect early fungal infection in oranges. Under sinusoidal modulation illumination with spatial frequencies of 0.05, 0.15, and 0.25 cycles mm–1, three-phase-shifted images with phase offsets of − 2π/3, 0, and 2π/3 were acquired for each spatial frequency. The direct component (DC) and alternating component (AC) images were then recovered by image demodulation using a three-phase-shifting approach. Compared with the DC image, the decayed area can be clearly identified in the AC image and RT image (AC/DC). The optimal spatial frequency was determined by analyzing the AC image and pixel intensity distribution. Based on the texture features extracted from DC, AC, and RT images, four kinds of classification models including partial least square discriminant analysis (PLS-DA), support vector machine (SVM), least squares-support vector machine (LS-SVM), and k-nearest neighbor (KNN) were established to detect the infected oranges, respectively. Model optimization was also performed by extracting important texture features. Compared to all models, the PLS-DA model developed based on eight texture features of RT images achieved the optimal classification accuracy of 96.4%. This study showed for the first time that the proposed SIRI system combined with appropriate texture features and classification model can realize the early detection of decayed oranges.

Detection of subsurface bruising in fresh pickling cucumbers using structured-illumination reflectance imaging

Pickling cucumbers are susceptible to bruising during harvest and postharvest handling. It is thus desirable to segregate bruised fruit before they are marketed as fresh products or processed as pickles. Structured-illumination reflectance imaging (SIRI) is an emerging optical imaging modality for food quality inspection. This study reported the first demonstration of SIRI for detecting subsurface bruising in fresh pickling cucumbers. Two independent sets of images, i.e., direct component (DC) and amplitude component (AC), were demodulated from phase-shifted sinusoidal pattern images at 740 nm; AC was found more effective than DC for ascertaining bruises that exhibited no visual symptoms. Classification models based on support vector machine were built using extracted image features, to classify cucumbers into bruised and normal classes. The highest classification accuracy of 91 % was achieved by the ensemble of DC, AC and their ratio (AC/DC) images, which represented 7.6 percentage-point improvement over that obtained using the DC images alone. Using features selection for five sets of image features led to further improvements in the classification performance. Incremental evaluation of top 50 most informative features resulted in an averaged overall accuracy of 94 %, with the highest accuracy of 97 % attained by 31 features; and using a subset of only 5 features, 3 from AC and 2 from DC, also produced a high overall accuracy of 96 %. This study demonstrates that SIRI can provide a potentially effective means for visualizing subsurface bruising in pickling cucumbers, which otherwise could barely be achieved by imaging under uniform illumination, and thus for enhancing the differentiation of normal and bruised fruit. More research is, however, needed to optimize and implement SIRI for real-time inspection of cucumber defects.

Siritool: A Matlab Graphical User Interface for Image Analysis in Structured-Illumination Reflectance Imaging for Fruit Defect Detection

HIGHLIGHTSA Matlab GUI, siriTool, was developed for structured-illumination reflectance imaging.siriTool enables image preprocessing, feature extraction, and classification.siriTool was demonstrated for detection of spot defects on pickling cucumbers.Abstract. Structured-illumination reflectance imaging (SIRI) is an emerging imaging modality that provides more useful discriminative features for enhancing detection of defects in fruit and other horticultural and food products. In this study, we developed a Matlab graphical user interface (GUI), siriTool (available at https://codeocean.com/capsule/5699671/tree), to facilitate image analysis in SIRI for fruit defect detection. The GUI enables image preprocessing (i.e., demodulation, object segmentation, and image enhancement), feature extraction and selection, and classification. Demodulation is done using a three-phase or two-phase approach depending on the image data acquired, object segmentation (or background removal) is implemented based on automatic unimodal thresholding, and image enhancement is achieved using fast bi-dimensional empirical decomposition followed by selective image reconstructions. For defect detection, features of different types are extracted from the enhanced images, and feature selection is performed to reduce the feature set. Finally, the full or reduced set of features are then input into different classifiers, e.g., support vector machine (SVM), for image-level classifications. An application example is presented on the detection of yellowish subsurface spot defects in pickling cucumbers. SIRI achieved over 98% classification accuracies based on SVM modeling with the extracted features, which were significantly better than the accuracies obtained under uniform illumination. Keywords: Defect detection, Demodulation, Image enhancement, Machine learning, Matlab, Structured illumination.

Detection of early decay in peaches by structured-illumination reflectance imaging

Peaches are susceptible to fungal infection after harvest, and detection of the symptom at early stage is critical to reducing economic loss for the industry, but is challenging because the symptom is not visible at the surface of infected fruit. This research was therefore aimed to develop a non-destructive and accurate method, based on structured-illumination reflectance imaging (SIRI), for detection of early fungal infection in peaches. Patterned spectral images for seven wavelengths between 690 nm and 810 nm were acquired from 600 peaches of different decayed levels, using a multispectral SIRI system, under sinusoidally-modulated illumination at the spatial frequencies of 60, 100 and 150 m−1. The resultant direct component (DC) images, which are equivalent to images acquired under uniform, diffuse illumination, could not reveal the slightly diseased symptom for peaches, but the symptom was visible from the alternating component (AC) images and ratio images calculated from the AC and DC images for the three frequencies. Watershed algorithm and partial least squares discriminant analysis were used for classification of diseased peaches based on the AC and ratio images, which achieved detection rates in the range of 65%–87%. Consistently better detections of diseased peaches were achieved with the AC images at the wavelength of 730 nm and the spatial frequency of 100 m−1. The pixel-based convolutional neural network for the AC images of 730 nm and 100 m−1 frequency achieved an excellent detection rate of 98.6% for all peach samples, and it also demonstrated superior performance for detecting early decayed peaches with non-visible disease infection symptom at a detection rate of 97.6%. For comparison, detection rates of diseased peaches by the three classification methods for the DC images were consistently lower. This study has shown that SIRI, coupled with an appropriate image classification method, can be effective for early disease detection of peaches.

Structured-illumination reflectance imaging for the detection of defects in fruit: Analysis of resolution, contrast and depth-resolving features

Structured-illumination reflectance imaging (SIRI), compared to conventional uniform illumination based modalities, provides a new means for defect detection of fruit. The capability of SIRI for defect detection, however, mainly depends on its image resolution and contrast and depth-resolving features. This study was therefore aimed at providing theoretical and experimental analyses of these features for SIRI, so as to evaluate its potential for detecting surface and subsurface defects of fruit and other food products. The image formation in SIRI was first modelled in terms of the optical transfer functions, which led to a general image demodulation methodology and also provided insights into the features of direct component (DC) and amplitude component (AC) images that were demodulated from the original SIRI pattern images. A set of experiments were performed on standard optical targets and apple samples by using illumination patterns over a wide range of spatial frequencies of 0.01–1.0 cycles mm−1 to examine the features of SIRI. The imaging process acted as a low-pass filter, which, coupled with the sample absorption and scattering, accounted for the resolution and contrast loss of resulting images. As the hallmark of SIRI, AC images possessed enhanced resolution and contrast, which could be explained by the synergistic effect of frequency shifting in the Fourier domain and suppressed light scattering in the sample, and also a depth-resolved ability by varying the spatial frequency of illumination patterns. However, the depth of tissue interrogation by the AC images decreased as the spatial frequency of illumination patterns increased. The effectiveness of AC in defect detection of apples depended on such factors as defect type, fruit surface morphology (thus variety) and spatial frequency of illumination. Without resorting to an inverse computational approach, SIRI should be positioned as a subsurface-surface modality for detecting defects that are on or close to the surface of fruits, especially thin-skinned fruits like apple. Further research is therefore needed to explore the full potential of SIRI in defect detection of fruits and other food products.

Fast Bi-dimensional empirical mode decomposition as an image enhancement technique for fruit defect detection

Image enhancement is critical to detection of fruit defects by imaging techniques. Vignetting and noise are major image artifacts, which can seriously affect image segmentation results, especially in inspecting the curved-surface objects like fruit. It is common to use a calibration object or a mathematic model to reduce the vignetting effect in defect detection, but the approach is often cumbersome, inflexible, and difficult to achieve desired results. In this study, a new image enhancement method based on bi-dimensional empirical mode decomposition (BEMD) of images was proposed to isolate and subsequently remove the effects of vignetting and noise by means of selective image reconstruction. The new BEMD method, along with three other BEMD methods, was first tested on decomposing a synthetic image with artificially added vignetting and noise. The BEMD was found to be the most efficient for image decomposition in terms of computation time, and also give high-quality reconstructed images. Experiments were further conducted by applying the BEMD to the direct and amplitude component images of apple samples with subsurface bruising and surface defects, which were acquired by using a structured-illumination reflectance imaging (SIRI) system. BEMD effectively reduced the image vignetting and greatly enhanced the defect features of the apples, based on both visual inspection and quantitative evaluation. BEMD offers an effective tool for enhancing SIRI images, and it is also promising for image enhancement with other imaging modalities for fruit defect detection.

Structured-illumination reflectance imaging coupled with phase analysis techniques for surface profiling of apples

Three-dimensional (3-D) geometry information is valuable for fruit quality evaluation. This study was aimed at exploring an emerging structured-illumination reflectance imaging (SIRI) system, coupled with phase analysis, for reconstructing surface profiles of fruit. Phase-shifted sinusoidal patterns, distorted by the fruit geometry, were acquired and processed through phase demodulation, phase unwrapping and other post-processing procedures to obtain phase difference maps relative to the phase of a reference plane. The phase maps were then transformed into height profiles based on phase-to-height calibrations. A reference plane-based approach, in conjunction with the curve fitting technique using polynomials of order 3 or higher, was utilized for phase-to-height calibrations, which achieved superior accuracies with the root-mean-squared errors (RMSEs) of 0.027–0.033 mm for a height measurement range of 0–91 mm. The 3rd-order polynomial curve fitting technique was further tested on two reference blocks with known heights, resulting in relative errors of 3.75% and 4.16%. Tests of the calibrated system for reconstructing the surface of apple samples showed that surface concavities (i.e., stem/calyx regions) could be readily discriminated from bruising defects from both the phase difference images and reconstructed height profiles. This study has laid a foundation for using SIRI for reconstructing the 3-D geometry, and thus expanded the capability of the technique for quality evaluation of horticultural products. Further research is needed to utilize the phase analysis techniques for detecting surface concavities of apples, and optimize the phase demodulation and unwrapping algorithms for faster and more reliable detection.

Structured Illumination Reflectance Imaging for Enhanced Detection of Subsurface Tissue Bruising in Apples

Abstract. In this research, a structured illumination reflectance imaging (SIRI) system was used as a novel method for detection of fresh bruises on apples. The SIRI system projects sinusoidal patterns of illumination onto samples, and image demodulation is then used to recover depth-specific information through varying the spatial frequency of the illumination pattern. The capability of SIRI was demonstrated through the detection of artificially induced bruises on ‘Golden Delicious’ and ‘Delicious’ apples with varying levels of bruising. It was hypothesized that by confining the light penetration depth near the surface of each fruit, subsurface defects such as bruising should be more apparent under SIRI than conventional planar illumination imaging. Three 120° phase-shifted reflectance images were acquired from 60 fruit each of ‘Golden Delicious’ and ‘Delicious’ varieties at 0 h, 4 to 6 h, and 24 h after impact bruising for each of the four spatial frequencies (i.e., 0, 0.10, 0.15, and 0.25 cycles mm-1). The reflectance images acquired by the system were then demodulated into an alternating component (AC) and direct component (DC), where the AC contained depth-specific information and the DC image represented the diffuse reflectance from the apple sample under uniform (or planar) illumination. Bruise detection algorithms were developed and applied to the demodulated AC and DC images. The SIRI system achieved 70% to 100% bruise detection rates, compared to 0% to 50% detection rates under conventional planar illumination. However, detection results were influenced by both severity of bruising and bruise development after impact; better bruise detection results were obtained when bruises had developed for 4 to 6 h after impact. SIRI has demonstrated a superior capability of detecting fresh bruises, and it is promising as a new imaging modality for quality detection of agricultural products. Keywords: Apples, Bruises, Defects, Fruit, Imaging, Nondestructive, Quality, Sorting, Structured illumination.

Detection of Surface and Subsurface Defects of Apples Using Structured- Illumination Reflectance Imaging with Machine Learning Algorithms

Abstract. Machine vision technology coupled with uniform illumination is now widely used for automatic sorting and grading of apples and other fruits, but it still does not have satisfactory performance for defect detection because of the large variety of defects, some of which are difficult to detect under uniform illumination. Structured-illumination reflectance imaging (SIRI) offers a new modality for imaging by using sinusoidally modulated structured illumination to obtain two sets of independent images: direct component (DC), which corresponds to conventional uniform illumination, and amplitude component (AC), which is unique for structured illumination. The objective of this study was to develop machine learning classification algorithms using DC and AC images and their combinations for enhanced detection of surface and subsurface defects of apples. A multispectral SIRI system with two phase-shifted sinusoidal illumination patterns was used to acquire images of ‘Delicious’ and ‘Golden Delicious’ apples with various types of surface and subsurface defects. DC and AC images were extracted through demodulation of the acquired images and were then enhanced using fast bi-dimensional empirical mode decomposition and subsequent image reconstruction. Defect detection algorithms were developed using random forest (RF), support vector machine (SVM), and convolutional neural network (CNN), for DC, AC, and ratio (AC divided by DC) images and their combinations. Results showed that AC images were superior to DC images for detecting subsurface defects, DC images were overall better than AC images for detecting surface defects, and ratio images were comparable to, or better than, DC and AC images for defect detection. The ensemble of DC, AC, and ratio images resulted in significantly better detection accuracies over using them individually. Among the three classifiers, CNN performed the best, with 98% detection accuracies for both varieties of apples, followed by SVM and RF. This research demonstrated that SIRI, coupled with a machine learning algorithm, can be a new, versatile, and effective modality for fruit defect detection. Keywords: Apple, Defect, Bi-dimensional empirical mode decomposition, Machine learning, Structured illumination.