3D-CNN with Multi-Scale Fusion for Tree Crown Segmentation and Species Classification

Abstract

Natural secondary forests play a crucial role in global ecological security, climate change mitigation, and biodiversity conservation. However, accurately delineating individual tree crowns and identifying tree species in dense natural secondary forests remains a challenge. This study combines deep learning with traditional image segmentation methods to improve individual tree crown detection and species classification. The approach utilizes hyperspectral, unmanned aerial vehicle laser scanning data, and ground survey data from Maoershan Forest Farm in Heilongjiang Province, China. The study consists of two main processes: (1) combining semantic segmentation algorithms (U-Net and Deeplab V3 Plus) with watershed transform (WTS) for tree crown detection (U-WTS and D-WTS algorithms); (2) resampling the original images to different pixel densities (16 × 16, 32 × 32, and 64 × 64 pixels) and inputting them into five 3D-CNN models (ResNet10, ResNet18, ResNet34, ResNet50, VGG16). For tree species classification, the MSFB combined with the CNN models were used. The results show that the U-WTS algorithm achieved a recall of 0.809, precision of 0.885, and an F-score of 0.845. ResNet18 with a pixel density of 64 × 64 pixels achieved the highest overall accuracy (OA) of 0.916, an improvement of 0.049 over the original images. After incorporating MSFB, the OA improved by approximately 0.04 across all models, with only a 6% increase in model parameters. Notably, the floating-point operations (FLOPs) of ResNet18 + MSFB were only one-eighth of those of ResNet18 with 64 × 64 pixels, while achieving similar accuracy (OA: 0.912 vs. 0.916). This framework offers a scalable solution for large-scale tree species distribution mapping and forest resource inventories.