Generating Natural Adversarial Remote Sensing Images

Abstract

Over the last years, remote sensing image (RSI) analysis has started resorting to using deep neural networks to solve most of the commonly faced problems, such as detection, land cover classification, or segmentation. As far as critical decision-making can be based upon the results of RSI analysis, it is important to clearly identify and understand potential security threats occurring in those machine learning algorithms. Notably, it has recently been found that neural networks are particularly sensitive to carefully designed attacks, generally crafted given the full knowledge of the considered deep network. In this article, we consider the more realistic but challenging case where one wants to generate such attacks in the case of a black-box neural network. In this case, only the prediction score of the network is accessible, on a specific dataset. Examples that lure away the network’s prediction, while being perceptually similar to real images, are called natural or unrestricted adversarial examples. We present an original method to generate such examples based on a variant of the Wasserstein generative adversarial network. We demonstrate its effectiveness on natural adversarial hyperspectral image generation and image modification for fooling a state-of-the-art detector. Among others, we also conduct a perceptual evaluation with human annotators to better assess the effectiveness of the proposed method. Our code is available for the community: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/PythonOT/ARWGAN</uri> .