An online continual object detector on VHR remote sensing images with class imbalance

Abstract

It is a great challenge for traditional offline detectors to learn from continuous data streams, remember previous tasks and adapt to new-coming tasks in dynamic environments. To meet the challenge, online continual learning has recently attracted increasing attention, while the overwhelming majority of works focus only on classification with a balanced class distribution assumption. In this paper, we propose a replay-based approach called an online continual object detector (OCOD) for very-high-resolution (VHR) remote sensing images. First, we find that rehearsal imbalance is ubiquitous, and has more important impact on experimental results than class imbalance, which is contrary to the situation of offline learning (due to the limited memory). Here, rehearsal imbalance refers to significant difference among the number of images pertaining to various classes. Second, entropy is used to measure the degree of rehearsal imbalance in the memory, and an entropy reservoir sampling (ERS) strategy is proposed to maintain rehearsal balance in the online memory. Finally, a rehearsal-balancing priority assignment network (RBPAN) is proposed to adaptively select images from the memory for a rehearsal-balancing replay procedure. The experimental results obtained on three publicly available VHR satellite images from the NWPU VHR-10, DIOR and DOTA datasets, highlight the effectiveness and practicality of developed method.