SSRNet: In-Field Counting Wheat Ears Using Multi-Stage Convolutional Neural Network

Abstract

Fast and accurate counting of wheat ears in field conditions is a key element for determining wheat yield. To obtain the number of wheat ears in a field, we propose a new counting algorithm based on computer vision. This algorithm counts wheat ears in remote images through semantic segmentation regression network (SSRNet). SSRNet is a multistage convolutional neural network that we propose to achieve counting problems through regression. In SSRNet, first, the original image is cropped to increase the amount of data. This method effectively solves the small sample dataset. Next, based on the cropping results, we build a fully convolutional neural network (FCNN) to segment wheat ears in field conditions. FCNN increases the accuracy of wheat ears counting by accurately segmenting wheat ears in a complex background. Then, we build a regression convolutional neural network (RCNN) to count wheat ears based on the segmentation results of FCNN. In RCNN, we propose a new activation function positive rectification linear unit (PrLU) to process the last layer of the fully connected layer, so that RCNN can effectively count the number of wheat ears in the image. Finally, a counting strategy is proposed to count the number of wheat ears in the original image. To verify the counting performance of SSRNet, we compare the counting result of SSRNet with the real value of manual statistics. The results show that the average accuracy (Acc), <inline-formula> <tex-math notation="LaTeX">$R^{2}$ </tex-math></inline-formula>, and root mean squared error (RMSE) of the SSRNet count results on the test set in this article are 0.980, 0.996, and 9.437, respectively. It can be seen from the results that our proposed method can accurately count wheat ears in field conditions. At the same time, the counting time (0.11 s) shows that SSRNet can quickly estimate the number of wheat ears in field conditions. We concluded that this study can provide important technical support for the high-throughput field wheat ears counting task in large-scale phenotyping work.