Instance-Aware Feature Alignment for Cross-Domain Cell Nuclei Detection in Histopathology Images

Abstract

Robust nuclei detection is crucial prerequisite for histologic characteristics of nuclei that can assist various clinical tasks such as disease diagnosis and cancer grading. Despite of their success, most existing nuclei detection methods ignore the case where the testing (target) domain has different data distribution with the training (source) domain, which is known as the problem of domain shift. In fact, the domain shift problem is prevalent in histopathology images due to various reasons such as different staining procedures and organ specific nuclear morphology. Thus, the performance of a nuclei detection model in the source domain will be hurt if it is directly applied to the target domain. To address this problem, we propose a novel instance-aware domain adaption framework for nuclei detection in histopathology images, which includes both image-level alignment (IMA) and instance-level alignment (INA) components to minimize the domain shift. Especially, INA component extracts instance-level features by using nuclei locations as the guidance and effectively aligns the instance-level features via adversarial training. Furthermore, to facilitate instance-level feature alignment, a Temporal Ensembling based Nuclei Localization (TENL) module is introduced in INA component to automatically generate candidate nuclei locations in the target domain. We evaluate the proposed method on different benchmark settings and obtain remarkable improvements compared to existing methods on the challenging problem of cross-domain cell nuclei detection.