Bridging the Gap Between Cumbersome and Light Detectors via Layer-Calibration and Task-Disentangle Distillation in Remote Sensing Imagery

Abstract

With urgent application requirements such as satellite in-orbit processing and unmanned aerial vehicle tracking, knowledge distillation (KD) following the teacher-student teaching mechanism has shown great potential to obtain lightweight detectors. However, compact students have limited accuracy due to the interference of large scale variations and blurred boundaries in remote sensing objects. Specifically, previous methods mostly force teacher-student responses from the layer of same depth and scale to align. Stereotyped manual inter-layer associations may cause discriminative features of multi-scale objects to be incorrectly bundled. Further, the regression branch follows the identical distillation paradigm as the classification branch, resulting in ambiguous object bounding box deviations. To solve the above two issues, we propose an effective KD framework called layer-calibration and task-disentangle distillation (LTD). First, cross-layer calibration distillation (CCD) structure is innovatively proposed. It adaptively binds a student layer with several related target layers, rather than a fixed layer in the teacher model. Appropriate and clear knowledge of large and small objects is transmitted. Since the CCD structure requires explicit global inner product computation between multiple layers, local implicit calibration (LIC) module is further proposed to reduce distilled convergence difficulty. Second, task-aware spatial disentangle distillation (TASD) structure is devised to transfer task-decoupled semantics and localization knowledge in a divide-and-conquer manner, alleviating objects localization imprecision. Experiments demonstrate that our LTD achieves state-of-the-art performance on several datasets, and is a plug-and-play approach to most detectors. The code will be available soon.