Sweet potato leaf detection in a natural scene based on faster R-CNN with a visual attention mechanism and DIoU-NMS

Abstract

Accurate detection of plant leaves is a meaningful and challenging task for developing smart agricultural systems. To improve the performance of detecting plant leaves in natural scenes containing severe occlusion, overlapping, or shape variation, we developed an in situ sweet potato leaf detection method based on a modified Faster R-CNN framework and visual attention mechanism. First, a convolutional block attention module was added to the backbone network to enhance and extract critical features of leaf images by fusing cross-channel information and spatial information. Subsequently, the DIoU-NMS algorithm was adopted to modify the regional proposal network by replacing the original NMS. DIoU-NMS was utilized to reduce missed and incorrect detection in scenes of densely distributed leaves by considering the targets' overlap ratio, distance, and scale. The proposed leaf detection method was tested and evaluated on sweet potato plant images collected in agricultural fields. In the datasets, sweet potato leaves were presented in various sizes and poses, and a large proportion of leaves were occluded or overlapped with each other. The experimental results showed that the proposed leaf detection method outperforms state-of-the-art object detection methods. The mean average precision of the proposed method reached 95.7%, which was 2.9% higher than that of the original Faster R-CNN and 7.0% higher than that of YOLOv5. The proposed method achieved promising performance in detecting dense leaves or occluded leaves and could provide key techniques for applications in smart agriculture and ecological monitoring, such as growth monitoring or plant phenotyping.