Incremental few-shot object detection via knowledge transfer

Abstract

As a challenging problem in machine learning, incremental few-shot object detection (iFSD) [1] aims to incrementally detect novel classes with few examples, while keeping the previous knowledge without revisiting base classes. Here, we propose two models based on the observation that when new memories come, new connections will be created between memory cells in the brain [2]. The first one, which is called the multi-class head (MCH) model, simulates how humans add new memory-connections that every time novel classes come, a classification branch is added to predict novel classification. And the second model, called the bi-path multi-class head (BPMCH) model, adds a new backbone, which is initialized with the weight of the base class backbone, to transfer more knowledge of the base class to the novel class. Considering accuracy and speed, we choose the Fully Convolutional One-Stage Object Detection (FCOS) [3] + Adaptive Training Sample Selection (ATSS) [4] detector as our baseline. Our models are first trained on the base classes with abundant examples and then finetuned on novel classes with few examples, which not only maintain the knowledge learned from the base class but also transfer the knowledge to the novel class. Extensive experiments show that our models outperform the state-of-the-art model ONCE [1] on the COCO [5] and PASCAL VOC [6] by a large margin.