Novelty Detection in Remote Sensing Imagery

Abstract

Object detection and classification in remote sensing imagery have been studied for decades, and has had a resurgence recently with significant improvements from deep learning. Most approaches follow the standard target recognition paradigm by assuming a fixed set of known object classes. The detector/classifier is trained on these, and attempts to disregard everything else. However, the real-world is complicated and unpredictable; often, there are new, interesting objects that are similar to known classes, but sufficiently different such that the system will (correctly) ignore them. The goal of novelty detection is to detect instances of new object types rather than misclassifying them as known types or background, while continuing to correctly classify instances of known object types. The primary challenge in novelty detection is determining how different a new image should be in order to be novel, vs. a new condition or variant of a known class. To address this, our method performs novelty detection in imagery using extreme value theory (EVT) operating in a CNN-based feature space. EVT characterizes the distribution of outliers in long-tailed distributions to identify novelties. We conducted experiments on the xView dataset for object detection and classification in satellite imagery, reducing it to a classification dataset by using its annotated bounding boxes on objects and holding out a set of 18 of its 60 classes as novelties. Our results indicate that EVT is effective at distinguishing novel from known object classes, even when novel classes are similar to known ones.