Sub-alpine shrub classification using UAV images: Performance of human observers vs DL classifiers

Abstract

In recent years, the automatic analysis of natural environment images acquired with unmanned aerial vehicles (UAV) has rapidly gained popularity. UAVs are specially important in mountainous forests where access is difficult and large areas need to be surveyed. In Zao mountains in northeastern Japan, regenerated fir saplings are competing with sub-alpine vegetation shrubs after a severe fir tree mortality caused by bark beetle infestation. A detailed survey of vegetation distribution is key to improve our understanding of species succession and the influence of climate change in that process. To that end, we evaluated the suitability of deep-learning-based automatic image classification of UAV images in order to map sub-alpine vegetation succession in large areas and the potential of fir regeneration.In order to assess the contribution of this technology in this research field, we first conducted an observer study to assess the difficulty for humans of the task of classifying vegetation from images. Afterwards, we compared the observers' accuracy to four state-of-the art deep learning networks for automatic image classification. The best observer accuracy of 55% demonstrates the limitations of species classification using only images. Furthermore, a detailed analysis of the sources of error showed that even though humans could differentiate between deciduous and evergreen species with an accuracy of 96%, identifying the correct species within each group proved much more challenging. In contrast, deep learning networks achieved accuracy values in the range of 70–80% for species classification, clearly demonstrating capabilities beyond human experts. Our experiments also indicated that the performance of these networks was significantly influenced by the similarity between the datasets used to fine-tune them and evaluate them. This fact highlights the importance of building publicly available images databases to further improve the results.Nevertheless, the results presented in this paper show that the analysis of UAV-acquired with deep learning networks can usher in a new type of large-scale study, spanning tenths or even hundreds of hectares with high spatial resolution (of a few cms per pixel), providing the ability to assess challenging vegetation dynamics problems that go beyond the ability of conventional fieldwork methodologies.