A transformer-based image detection method for grassland situation of alpine meadows

Abstract

As a vital role in climate regulation, water conservation, and maintenance of ecological balance, the alpine meadow grassland is facing the threat of degradation. Detecting grassland topography, phytomass, and grassland damage are important for improving the alpine meadow situation. This study reports a Transformer-CNN method for detecting alpine meadows situations using UnmannedAerial Vehicle (UAV) - based RGB (Red, Green, and Blue) data. This method combines Oriented FAST and Rotated BRIEF (ORB) and brute force feature matching to complete image stitching and then uses the proposed model Am-mask to complete the image segmentation task. The result shows that ORB feature matching is more stable and fast than SIFT and SURF for alpine meadow image stitching. In addition, Transformer has great application potential in grassland image detection and introducing task prefix and sparse in pre-training enhances the model’s robustness. The AP value of the Am-mask model with Transformer was as high as 95.4%, about 10% higher than that of the original CNN models. In the experiment with unstitched images, the average precision of the eight trials was 95.16%, the average recall was 95.13%, and the average F1 value was 95.14%. For stitched images, the average precision, recall, and F1 value of the eight trials were 91.83%, 91.81%, and 91.82%, respectively. It was proved that the proposed method could save the inference cost of the model under the condition of ensuring the detection effect. This study may contribute to grassland environmental protection in alpine meadows.