Patch-U-Net: tree species classification method based on U-Net with class-balanced jigsaw resampling

Abstract

ABSTRACT Automatic tree species classification based on remote-sensing images can significantly improve the efficiency of tree species investigation and save considerable cost of human labor. A fully convolutional network (FCN) can automatically extract tree species-related features to achieve higher classification performance. However, this kind of method needs a large quantity of training data. Due to few samples and unbalanced sample distribution of tree species remote-sensing images, directly applying FCN to tree species image classification task could not achieve good results. We proposed Patch-U-Net to tackle the above problem. Our method adopts the class-balanced jigsaw resampling strategy to explicitly balance inter-class distribution and augment data in patch-wise. Besides, it extracts multi-scale information of each patch by combining the encoder–decoder and skip connection structure. We compared Patch-U-Net with six existing methods, and Patch-U-Net achieved the best performance. Specifically, Pixel Accuracy (PA), Mean Intersection over Union (MIoU), and Frequency Weighted Intersection over Union (FWIoU) of Patch-U-Net are 80.33%, 57.46%, and 67.37%, which are 14.3%, 33.16%, and 17.81% higher than those of the baseline model U-Net, respectively. The results show that Patch-U-Net can improve the performance of remote-sensing tree species classification by solving the problem of unbalanced samples, which is more suitable for the remote-sensing image classification of tree species with imbalance species.