GraphSKT: Graph-guided Structured Knowledge Transfer for Domain Adaptive Lesion Detection.

Abstract

Adversarial-based adaptation has dominated the area of domain adaptive detection over the past few years. Despite their general efficacy for various tasks, the learned representations may not capture the intrinsic topological structures of the whole images and thus are vulnerable to distributional shifts especially in real-world applications, such as geometric distortions across imaging devices in medical images. In this case, forcefully matching data distributions across domains cannot ensure precise knowledge transfer and are prone to result in the negative transfer. In this paper, we explore the problem of domain adaptive lesion detection from the perspective of relational reasoning, and propose a Graph-Structured Knowledge Transfer (GraphSKT) framework to perform hierarchical reasoning by modeling both the intra- and inter-domain topological structures. To be specific, we utilize cross-domain correspondence to mine meaningful foreground regions for representing graph nodes and explicitly endow each node with contextual information. Then, the intra- and inter-domain graphs are built on the top of instance-level features to achieve a high-level understanding of the lesion and whole medical image, and transfer the structured knowledge from source to target domains. The contextual and semantic information is propagated through graph nodes methodically, enhancing the expressive power of learned features for the lesion detection tasks. Extensive experiments on two types of challenging datasets demonstrate that the proposed GraphSKT significantly outperforms the state-of-the-art approaches for detection of polyps in colonoscopy images and of mass in mammographic images.