Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network

Jakaria Rabbi,Subir Chowdhury,Matthias Schubert,Nilanjan Ray,Dennis Chao

doi:10.3390/rs12091432

Abstract

The detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement performance, but reconstructed images usually miss high-frequency edge information. Therefore, object detection performance showed degradation for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we applied a new edge-enhanced super-resolution GAN (EESRGAN) to improve the quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss was backpropagated into the EESRGAN to improve the detection performance. We proposed an architecture with three components: ESRGAN, EEN, and Detection network. We used residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we used a faster region-based convolutional network (FRCNN) (two-stage detector) and a single-shot multibox detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) dataset and another self-assembled (oil and gas storage tank) satellite dataset showed superior performance of our method compared to the standalone state-of-the-art object detectors.

Highlights

We aim to improve the detection performance of small objects on remote sensing imagery
Inspired by the architecture of enhanced super-resolution GAN (ESRGAN), we remove batch normalization (BN) layers to increase the performance of the generator G and to reduce the computational complexity
Our method outperformed the standalone state-of-the-art methods such as single-shot multibox detector (SSD) or faster R-convolutional neural network (CNN) when implemented in low-resolution remote sensing imagery

Summary

Introduction

There are many methods for detecting and locating objects from images, which are captured using satellites or drones. Detection performance is not satisfactory for noisy and low-resolution (LR) images, especially when the objects are small [4]. Even on high-resolution (HR) images, the detection performance for small objects is lower than that for large objects [5]. Current state-of-the-art detectors have excellent accuracy on benchmark datasets, such as ImageNet [6] and Microsoft common objects in context (MSCOCO) [7]. These datasets consist of everyday natural images with distinguishable features and comparatively large objects

Objectives

Methods

Results

Discussion

Conclusion