Potential evaluation of visible-thermal UAV image fusion for individual tree detection based on convolutional neural network

Abstract

• One of the advantages of the U-Net convolution network is having a large receptive field. • The U-Net network has no fully connected layer which reduces the need for training data. • Using the fusing visible and thermal images, trees in shady areas can be identified with high accuracy. • Isolation of trees from tall boxwoods and also detection of small trees using the normalized digital surface model. • The proposed algorithm can accurately estimate tree crown parameters with high accuracy. Unmanned aerial vehicles (UAVs) outfitted with thermal and visible sensors are already a popular platform in precision agriculture thanks to recent advances in remote sensing. Many researchers have studied integrating data from sensors with different spectral characteristics to achieve higher-level properties and, consequently, detect the trees accurately. In this research, visible and thermal images, as well as normalized digital surface models resulting from UAVs with high spatial resolution, are employed to accurately extract trees from two studied urban areas with complex backgrounds. In the thermal image, trees can be detected in hidden areas based on their brightness temperature difference compared to other features. In contrast, the visible image has a higher spatial resolution, and fusing this data with thermal images can resolve the complexity of the problem. In the proposed method, first, a deep learning network based on visible-thermal data is evaluated in terms of detecting trees with various data approaches. These evaluations include comparison tests on four types of data input to the convolutional network of the visible images, thermal images, fusing visible-thermal images, and also fusing visible-thermal- normalized digital surface model images. Results of evaluation parameters indicate maximum precision in the fourth approach (intersection-over-union = 91.72, F-score = 95.67). Then, the output binary map with the highest accuracy approach and Canny edge detection operator is utilized to accurately identify tree boundaries, count, and estimate the area and diameter of the tree canopy. Finally, the findings revealed the root mean square error (RMSE) first and second areas are 0.21 m 2 , 0.08 m and 0.24 m 2 , 0.11 m respectively for the area and diameter of the tree crown.