MTYOLOX: Multi-transformers-enabled YOLO for tree-level apple inflorescences detection and density mapping

Abstract

Fruit tree-specific management plays a vital role in ensuring fruit quality and quantity. Monitoring inflorescence information is essential for apple tree cultivation activity such as inflorescence thinning in the early flowering phase. The proportion to be thinned is mainly determined by the inflorescence intensity of the whole tree. Most existing researches pay more attention to apple flower detection in local regions of the tree, but a small subset of trees cannot provide tree-level information on inflorescences. Therefore, a novel transformer-based CNN model, MTYOLOX, is presented for robustly detecting full tree inflorescences. The DAT-Darknet and ST-PAFPN modules based on multiple self-attention mechanisms are designed and rationally embedded into the backbone and neck of the network to explore the potential global context information and extract more distinguished features for inflorescences detection. The MTYOLOX exhibits robust adaptability to variable illumination directions in the uncontrolled and challenging orchard environment. Comparisons with representative detection methods (SSD, FCOS, Faster RCNN, YOLOv4, YOLOv5s, Variant YOLOv5s, YOLOXs and YOLOv7) show that the proposed MTYOLOX achieves the highest AP50 of 0.834 and AR50 of 0.933 with encouraging detection speed, total parameters, flops and model size. The competitive performance indicated that the proposed method is feasible for apple inflorescences detection of the whole tree. On the basis of detection results from the MTYOLOX, the tree-level inflorescences density mapping is implemented for the potential commercial application. This study is expected to provide a solution reference for precision horticulture.