Vision system for detection of defects on apples using hyperspectral imaging coupled with neural network and Haar cascade algorithm

Abstract

The article is devoted to the development of algorithms for detecting defective apples transported on a roller conveyor using a vision system. In developing the algorithms, the possibility of classifying various regions of interest (intact and damaged by rot, scab, codling moth, as well as the conveyor) by the principal component method was investigated. When choosing the optimal spectral region for cluster analysis, spectrograms obtained in various spectral ranges, including Vis-NIR (400–1000 nm), NIR (780–1000 nm), and Vis (400–780 nm) were used. The PCA method showed that for the successful classification of the conveyor area, intact, decayed and damaged by the codling moth, it is necessary to use spectrograms in the Vis-NIR range. To classify these ROIs, it was proposed to use a direct distribution neural network with two hidden layers of 128 and 64 layers, respectively, the “relu” activation function in the hidden layers and the “softmax” activation function in the output layer. The optimal network configuration was determined experimentally. This configuration showed a classification accuracy of 0.847 on a test sample of 6,000 apples. Since the samples of spectrograms of scab and stem regions do not differ, for their classification in parallel with the neural network, it was proposed to use the Haar cascade classifier trained on 2000 two-dimensional images of apples in the visible region containing scab and stem regions. The classification accuracy was at least 0.95. The developed algorithm is intended for use in the robotic sorting of apples.