Impurity monitoring study for corn kernel harvesting based on machine vision and CPU-Net

Abstract

Kernel impurity rate is a critical metric to evaluate the quality of corn harvester operation. Restrictions such as complex harvesting environments, unstable lighting, and high vibration frequency of the harvester body lead to the problems of low accuracy and poor reliability of existing kernel impurity monitoring methods, which cannot meet the requirements of automatic and intelligent real-time monitoring of harvesters. The monitoring method of impurity rate was systematically carried out by combining the detailed characteristics of corn materials and the disturbing factors of the harvesting environment. First, a machine vision-based monitoring device for kernel impurity rate was designed to ensure the effect of monolayered material transfer and the effective online acquisition of images. Then, the corn material mass-pixel linear regression model was established by graying, binarizing, and morphologically processing the corn kernel, corncob, and bract images to calculate the kernel impurity rate. Next, a CPU-Net semantic segmentation model dedicated to corn impurity monitoring was proposed, which integrated the convolutional block attention module (CBAM) and pyramid pooling module (PPD) based on the U-Net network. CBAM used channel attention and spatial attention modules in cascade instead of skip connection to reduce the semantic gap between encoder and decoder. PPM used multi-scale pooling operations to increase the perception field of the network. Finally, the SegNet, U-Net, Attention U-Net, and CPU-Net were trained using a custom corn image set. The experimental results showed that the CPU-Net network has the best-combined segmentation performance with an average MIoU, MPA, and ST of 97.31 %, 98.71 %, and 158.4 ms, respectively, the average relative error between the impurity rate obtained by the monitoring device and the manual statistics was 4.64 %. The real-time impurity monitoring results provide a basis for the driver or automatic control system to regulate operating parameters timely, thereby improving the quality and efficiency of harvester operations and reducing economic losses.