Deer survey from drone thermal imagery using enhanced faster R-CNN based on ResNets and FPN

Abstract

Deer surveys play an important role in the estimation of local ecological balance. In the Chitwan National Park of Nepal, the dense tree canopies and tall vegetation often obscure the presence of wild deer, which has a negative effect on the accurate population surveys of wild deer. UAVs equipped with infrared sensors have been increasingly used to monitor wild deer by capturing a lot of images. How to automatically recognize and obtain the number of deer objects from thermal images is becoming an important research topic. Due to the difference between thermal images and true-color images, as well as the variations in deer object sizes in these two types of images, current ready-to-use object detection models, designed for true-color imagery, are ill-suited for the task of detecting small deer objects within thermal imagery. In this paper, an enhanced Faster R-CNN was constructed to detect small deer objects from thermal images, in which a Feature Pyramid Network (FPN) based on a residual network is used to improve feature extraction for small deer objects and multi-scale feature map constrution for the subsequent region proposals searching, bounding box regression, and regions of interest (RoIs) classification. In addition, small-scaled anchor boxes and a multi-scale feature map selection criterion are devised to improve the detection accuracy of small objects. Finally, based on Faster R-CNN, FPN, and different residual networks including ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152, we constructed five object detection models, and evaluated their detection performance by using COCO evaluation matrix. Under the condition of IoU≥0.5, the integration of Faster R-CNN, FPN, and ResNet18 demonstrated to perform better than others. Specifically, The COCO evaluation results revealed an Average Precision (AP) score of 91.6% for all deer objects. Small deer objects (area ≤ 200 pixels) achieved an AP score of 73.6%, medium deer objects (200 < area ≤ 400 pixels) demonstrated an AP score of 93.4%, and large deer objects (area > 400 pixels) achieved the highest AP score of 94.3%. Our research is helpful for effective wild deer monitoring and conservation and can be a valuable reference for the exploration of small object detection from low-resolution thermal images.