Physics Guided Remote Sensing Image Synthesis Network for Ship Detection

Abstract

Automatic detection and localization of objects in remote sensing images are of great significance for remote sensing systems. Existing frameworks usually train an object detection network using collected remote-sensing images. However, these models usually perform poorly due to the lack of large-scale training datasets, which is often the case for special remote sensing scenarios, e.g., the detection of ships in the open sea. Although image synthesis is a common strategy to alleviate the issue of data insufficiency, the trained model still performs poorly when being tested on real-world scenes. Aimed at this, a novel sensor-related image synthesis framework, dubbed as RS-ISP, is developed to address the lack of on-orbit remote sensing images. Specifically, our RS-ISP introduces two novel designs to ensure the distribution consistency between the generated images and the real images. 1) The first is a novel pipeline for modeling the physical process of noise production during image capture using specific sensors. 2) The second is the design of a detection-oriented image harmonization model. Similar to the existing design, our model first produces coarse synthetic images by copy-paste operation, on which the proposed harmonization process is used to reduce the variation of the pasted foreground and background. By incorporating these two designs into a unified framework, our RS-ISP is designed and used to produce large-scale synthetic images used to train the object detection model for detecting ships in remote sensing images. Comparative experiments demonstrated that RS-ISP increased the AP@.50 from 0.148 to 0.498 for the ship detection task. Code will be publicly available.