Wheat Spike Detection and Counting in the Field Based on SpikeRetinaNet.

Changji Wen,Jianshuang Wu,Xiao Chen,Hongrui Chen,Zhuoshi Li,Ce Yang,Hengqiang Su

doi:10.3389/fpls.2022.821717

Abstract

The number of wheat spikes per unit area is one of the most important agronomic traits associated with wheat yield. However, quick and accurate detection for the counting of wheat spikes faces persistent challenges due to the complexity of wheat field conditions. This work has trained a RetinaNet (SpikeRetinaNet) based on several optimizations to detect and count wheat spikes efficiently. This RetinaNet consists of several improvements. First, a weighted bidirectional feature pyramid network (BiFPN) was introduced into the feature pyramid network (FPN) of RetinaNet, which could fuse multiscale features to recognize wheat spikes in different varieties and complicated environments. Then, to detect objects more efficiently, focal loss and attention modules were added. Finally, soft non-maximum suppression (Soft-NMS) was used to solve the occlusion problem. Based on these improvements, the new network detector was created and tested on the Global Wheat Head Detection (GWHD) dataset supplemented with wheat-wheatgrass spike detection (WSD) images. The WSD images were supplemented with new varieties of wheat, which makes the mixed dataset richer in species. The method of this study achieved 0.9262 for mAP50, which improved by 5.59, 49.06, 2.79, 1.35, and 7.26% compared to the state-of-the-art RetinaNet, single-shot multiBox detector (SSD), You Only Look Once version3 (Yolov3), You Only Look Once version4 (Yolov4), and faster region-based convolutional neural network (Faster-RCNN), respectively. In addition, the counting accuracy reached 0.9288, which was improved from other methods as well. Our implementation code and partial validation data are available at https://github.com/wujians122/The-Wheat-Spikes-Detecting-and-Counting.

Highlights

As one of the three major cereal crops, wheat provides food for approximately one-third of the world’s population
The results indicate that our method has better detection and counting effect than Faster region-based convolutional neural network (R-CNN), You Only Look Once version3 (Yolov3), You Only Look Once version4 (Yolov4), and singleshot multiBox detector (SSD) in the mixed dataset
We developed a wheat spike detection method based on the SpikeRetinaNet to address the issue of small dense object detection and counting in complex scenes

Summary

INTRODUCTION

As one of the three major cereal crops, wheat provides food for approximately one-third of the world’s population. Texture, shape, and edge are fused in the classifier to detect wheat spikes using traditional image processing methods. These improvement points solve spike error detection and miss detection caused by occlusion conditions in UAV images These studies used deep learning methods to overcome the disadvantages of traditional image processing methods that require manual feature design. We introduced the BiFPN (Tan et al, 2020) and double SA (DSA) (split attention block and spatial attention block) into the backbone network to realize fine-grained feature extraction and representation across feature map groups and strengthen the fusion of global information and local information. Introducing DSA into the backbone realizes the interaction between feature map channels and receptive field regions In this way, fine-grained discriminant feature of detecting wheat spikes, such as the shape, texture, and color, can be better extracted and represented. We introduced soft non-maximum suppression (Neubeck and Van Gool, 2006) (Soft-NMS) (Bodla et al, 2017), an algorithm that decays the detection scores of all other objects as a continuous function of their overlap, to solve the problem of missed detection caused by mutual occlusion

MATERIALS AND METHODS

Method

RESULTS

CONCLUSION

DATA AVAILABILITY STATEMENT