Deep convolutional neural networks and Swin transformer-based frameworks for individual date palm tree detection and mapping from large-scale UAV images

Abstract

Timely and reliable mapping of individual date palm trees is essential for their monitoring, health and risk assessment, pest control, and sustainable management of the date palm industry. This study presents an instance segmentation framework for large-scale detection and mapping of date palm trees using unmanned aerial vehicle (UAV)-based images. First, a data conversion framework is created to convert UAV image tiles and ground-truth vector data into annotation format of Common Objects in Context. Second, this study examines the efficacy of various instance segmentation models, namely, mask region convolutional neural network (Mask R-CNN), Mask Scoring R-CNN, You Only Look At CoefficientTs, Point-based Rendering, Segmenting Objects by Locations (SOLO), and SOLOv2) with varying residual learning networks (ResNets) in detecting and delineating individual date palm trees. Furthermore, the performance of two variants of Swin Transformer networks with a feature pyramid network (FPN) (Swin-small-FPN and Swin-tiny-FPN) as Mask R-CNN network backbones was also evaluated. Third, we assess the generalizability of the evaluated instance segmentation models and backbones on different testing datasets with varying spatial resolutions. Results show that Mask R-CNN models based on Swin Transformers backbones outperform those with ResNets in the detection and segmentation of date palm trees with mAP50 of 92% and 91% and F-measures of 94% and 93%. Moreover, the Mask scoring R-CNN-based ResNet-50 and Mask R-CNN with a Swin-small-FPN backbone outperform the evaluated models and demonstrate great generalizability in different datasets with diverse spatial resolutions. The proposed instance segmentation framework provides an efficient tool for date palm tree mapping from multi-scale UAV-based images and is valuable and suitable for individual tree crown delineations and other earth-related applications.