Structured-illumination Reflectance Imaging Research Articles

Development of a Multispectral Structured Illumination Reflectance Imaging (SIRI) System and Its Application to Bruise Detection of Apples

Abstract. SIRI is a promising new imaging modality for enhancing quality detection of food. A liquid-crystal tunable filter (LCTF)-based multispectral SIRI system was developed and used for selecting optimal wavebands to detect bruising in apples. Immediately after impact bruising, ‘Delicious’, ‘Royal Gala’, ‘Granny Smith’, and ‘Golden Delicious’ apples were imaged by the system over the spectral region of 650 to 950 nm with 20 nm increments under sinusoidally modulated illumination at a spatial frequency of 100 cycles m-1. Each sample was subjected to two phase-shifted sinusoidal patterns of illumination with phase offsets of 0 and 2p/3 that were generated by a digital light projector. For comparison, spectral images were also captured under conventional uniform illumination. Spiral phase transform, a newly developed two-phase based demodulation method, was then used to retrieve amplitude component (AC) and direct component (DC) images from the SIRI images, from which ratio images were obtained by dividing the AC images by the DC images. It was found that the uniform illumination images failed to reveal the bruises in apples, whereas bruises were distinctly visible in the ratio images, with contrast varying with wavelength. Principal component analysis (PCA) showed that seven wavelengths from 710 to 830 nm were more relevant to bruise detection. A modified Otsu thresholding method based on the between-class variance was proposed for bruise segmentation from the ratio images at each of the seven wavelengths as well as the first principal component (PC1) images, which resulted in overall detection errors of 11.7% to 14.2%. This study has shown the potential of using a multispectral SIRI system for defect detection of fruit. Further research is needed to develop a general algorithm for defect detection of apples and upgrade the system toward real-time detection. Keywords: Defects, Detection, Fruit, Image analysis, LCTF, Structured illumination.

Non-Destructive Defect Detection of Apples by Spectroscopic and Imaging Technologies: A Review

Abstract. Apples are susceptible to a wide range of defects that can occur in the orchard and during the post-harvest period. Detection of these defects by non-destructive sensing techniques is of great importance for the apple industry and has been an intensive research topic over the past two decades. This review presents an overview of common defects in apples, encompassing physiological disorders, mechanical damage, pathological disorders, and contamination. Presented and discussed in this review is research progress on the detection of defects in apples using various non-destructive spectroscopic and imaging techniques, including visible/near-infrared spectroscopy, fluorescence spectroscopy and imaging, monochromatic and color imaging, hyperspectral and multispectral imaging, x-ray imaging, magnetic resonance imaging, thermal imaging, time-resolved and spatially resolved spectroscopy, Raman spectroscopy, biospeckle imaging, and structured-illumination reflectance imaging. This review concludes with remarks on the prospects of these techniques and research needs in the future. Keywords: Apples, Defects, Imaging, Non-destructive detection, Quality, Safety, Spectroscopy.

Using composite sinusoidal patterns in structured-illumination reflectance imaging (SIRI) for enhanced detection of apple bruise

Structured-illumination reflectance imaging (SIRI) provides a new means for enhanced detection of defects in horticultural products. Implementing the technique relies on retrieving amplitude images by illuminating the object with sinusoidal patterns of single spatial frequencies, which, however, are limited in interrogating the tissues at different depths. This study presented a first exploration of using composite sinusoidal patterns that integrated two and three spatial frequencies of interest, in SIRI for enhanced detection of defects in food (e.g., bruises in apples). Three methods based on Fourier transform were proposed to retrieve amplitude images at different frequencies by using either phase shifting with or without spiral phase transform (SPT) or frequency-domain filtering. The phase-shifting method involves solving a linear system that is composed of multiple phase-shifted pattern images in the Fourier space, and SPT that acts as a two-dimensional quadrature transform operator is used to reduce the images needed for amplitude retrieval; while the filtering-based method directly extracts different frequency components from only one pattern image that are then subjected to SPT processing. The three methods were tested for dual-frequency and triple-frequency patterns through numerical simulations and experiments on the detection of fresh bruises in apples by SIRI. The phase-shifting methods showed good performance in terms of small demodulation errors and strong image contrast for bruise detection; the filtering-based method, although viable in numerical simulation, needed improvement due to the worst practical performance. In addition, more frequency components introduced in the pattern would deteriorate the performance of these methods, and grid composite patterns were superior over the fringe ones due to reduced interactions between different frequency components.

Gram-Schmidt orthonormalization for retrieval of amplitude images under sinusoidal patterns of illumination.

Structured illumination using sinusoidal patterns has been used for optical imaging of biological tissues in biomedical research, and of horticultural products in food quality evaluation. Implementation of structured-illumination imaging relies on retrieval of amplitude images, which is conventionally achieved by a phase-shifting technique that requires collecting a minimum of three phase-shifted images. In this study, we have proposed Gram-Schmidt orthonormalization (GSO) to retrieve amplitude component (AC) images using only two phase-shifted images. We have proposed two forms of GSO implementation, and prior to GSO processing, we eliminated the direct component (DC) background by subtracting a DC image we recovered using a spiral phase function (SPF) in the Fourier space. We demonstrated the GSO methods through numerical simulations and application examples of detection of bruise defects in apples by structured-illumination reflectance imaging (SIRI). GSO performed comparably to conventional three-phase-based demodulation. It is simple, fast and effective for amplitude retrieval and requires no prior phase information, which could facilitate fast implementation of structured-illumination imaging.

Fast demodulation of pattern images by spiral phase transform in structured-illumination reflectance imaging for detection of bruises in apples

A structured-illumination reflectance imaging (SIRI) system was recently developed in our laboratory for enhanced quality evaluation of horticultural products. It was implemented using a digital camera to acquire reflectance images from food products subjected to sinusoidal patterns of illumination, instead of conventional diffuse or uniform illumination. The reconstruction of amplitude images is a key step in implementing SIRI technique. Conventional methods require acquisition and subsequent demodulation of three phase-shifted pattern images, which limits the speed of image acquisition. This study proposed the use of spiral phase transform (SPT) for demodulation of pattern images acquired by SIRI. Three SPT-based methods involving only one or two images were investigated through numerical simulation, followed with experiments on the detection of fresh bruises in apples. Compared to conventional three-phase based and two-phase based Gram-Schmidt (GS) orthonormalization demodulation methods, the two-phase SPT method achieved the same or even higher demodulation accuracy, which applies to two pattern images that can be arbitrarily phase-shifted. SPT also allowed single-image based demodulation, although its performance still needs to be improved. This study demonstrated that SPT for amplitude demodulation can increase the imaging speed of SIRI by using one or two pattern images, which is a significant step towards real-time implementation of the technique.

Structured-illumination reflectance imaging (SIRI) for enhanced detection of fresh bruises in apples

A structured-illumination reflectance imaging (SIRI) system was developed for the detection of fresh bruises in apples. The system consisted of a high-performance digital camera, a broadband DC-regulated light source, and a digital light engine for generating specific patterns of illumination. Sinusoidal patterns of illumination at a series of spatial frequencies were created to illuminate test samples. Experiments were first conducted on a strongly scattering nylon sample embedded with foreign objects at different depths and then on apples of three cultivars with fresh bruises. The acquired images were demodulated into two separate sets of direct component (representing uniform diffuse illumination) and alternating component (or amplitude) images for each spatial frequency. The amplitude images were found to be effective for ascertaining the sub-surface defects, and the detection resolution and depth depended on the spatial frequency of illumination. SIRI technique was able to detect fresh bruises in apples immediately after they were induced, which, otherwise, could be difficult to achieve by conventional uniform illumination-based imaging techniques. The technique is promising for effective detection of defects, like bruise, in fruit.

A novel watershed method using second order gradient image

Thresholding is an important step in the segmentation of image features, and the existing methods are not all effective when the image histogram exhibits a unimodal pattern, which is common in defect detection of fruit. This study was aimed at developing a general automatic thresholding methodology for fast and effective segmentation of bruises from the images acquired by structured-illumination reflectance imaging (SIRI). SIRI images, under sinusoidal patterns of illumination at a spatial frequency of 100 cycles m−1, were acquired from 120 apple samples of four varieties with artificially created bruises and from another 40 apples with naturally occurred bruises. Subsequently, three sets of images, i.e., amplitude component (AC), direct component (DC) and ratio (i.e., dividing AC by DC), were derived from the original SIRI images. A unimodal thresholding method, called UNIMODE, was first applied to DC images for background removal, and then nine automatic thresholding techniques, including one unimodal and eight bimodal, were applied to the ratio images for bruise segmentation. It was found that severe over-segmentation occurred when using the bimodal thresholding methods, and this problem was mitigated by confining threshold selection to the lower part of the histogram that contained bruise information. Three bimodal thresholding techniques, i.e., INTERMODE (histogram valley emphasized), RIDLER (iterative thresholding), OTSU (clustering based) achieved the best bruise detection results with the overall accuracies of more than 90%. The overall detection results were further improved by integrating these techniques with the unimodal thresholding, due to reductions in the false positive error. The three bimodal thresholding techniques resulted in overall detection accuracies of 77–85% for naturally occurred bruises. This study has showed that the proposed automatic thresholding methodology provides a simple and effective tool for bruise detection of apples.