A non-destructive dropped fruit impact signal imaging-based deep learning approach for smart sorting of kiwifruit

Abstract

This paper focuses on developing an intelligent kiwifruit sorting system that uses an imaging method based on the dropped fruit impact signal, which combines with convolutional neural network (CNN) for the high-quality sorting of soft and hard kiwifruits. The kiwifruit sorting system detects the hardness of kiwifruit by analyzing the change in the force signal during the fall of kiwifruit on the sorting tray. The proposed signal imaging method constructs peak image (PI) by intelligently extracting the peak features of the data directly from the original time-domain force and force acceleration data in chronological order, and learns the features of PI for hardness classification by two-dimensional (2D) CNN. In addition, the performance of PI based on different peak, force signal or force acceleration signal in 2D CNN classifier is tested, and the 2D CNN classifier based on PI is compared with the 2D CNN classifier based on classical time-series imaging methods such as recurrence plot (RP) and Markov transition field (MTF), and further compared with the one-dimensional (1D) CNN model based on original time-domain features. A total of 18 CNN models are constructed for experiments in this paper, and the accuracy of the 2D CNN classifiers based on PI, MTF and RP are 98.89%, 94.75% and 98.34% at the time series length of 75 and the number of PI peaks of 5, respectively. The experimental results show that the PI-based 2D CNN classifier proposed in this paper achieves the best performances compared with other similar models, and demonstrates the feasibility of the MTF and RP-based 2D CNN classifier in kiwifruit hardness sorting.