Cuepervision: self-supervised learning for continuous domain adaptation without catastrophic forgetting

Abstract

Perception systems, to a large extent, rely on neural networks. Commonly, the training of neural networks uses a finite amount of data. The usual assumption is that an appropriate training dataset is available, which covers all relevant domains. This abstract will follow the example of different lighting conditions in autonomous driving scenarios. In real-world datasets, a single source domain, such as day images, often dominates the dataset composition. This poses a risk to overfit on specific source domain features within the dataset, and implicitly breaches the assumption of full or relevant domain coverage. While applying the model to data outside of the source domain, the performance drops, posing a significant challenge for data-driven methods. A common approach is supervised retraining of the model on additional data. Supervised training requires the laborious acquisition and labeling of an adequate amount of data and often becomes infeasible when data augmentation strategies are not applicable. Furthermore, retraining on additional data often causes a performance drop in the source domain, so-called catastrophic forgetting. In this paper, we present a self-supervised continuous domain adaptation method. A model trained supervised on the source domain (day) is used to generate pseudo labels on the samples of an adjacent target domain (dawn). The pseudo labels and samples enable to fine-tune the existing model, which, as a result, is adapted into the intermediate domain. By iteratively repeating these steps, the model reaches the target domain (night). The results, of the novel method, on the MNIST dataset and its modification, the continuous rotatedMNIST dataset demonstrate a domain adaptation of 86.2%, and a catastrophic forgetting of only 1.6% in the target domain. The work contributes a hyperparameter ablation study, analysis, and discussion of the new learning strategy.