Deconfounded multi-organ weakly-supervised semantic segmentation via causal intervention

Abstract

In weakly-supervised semantic segmentation, obtaining the class activation maps for pseudo masks is crucial. Since multiple organs appear in the same medical image, it is reasonable to obtain the activation maps of each organ by the organ-level features instead of the image-level features. The image-level features are decomposed into the organ-level features, yet the prior anatomical knowledge makes a spurious association between the image-level and organ-level features. To this end, we apply the causal intervention to cut off the spurious association and propose a novel deconfounded multi-organ weakly-supervised semantic segmentation (DeMos) method. Based on the original class activation mapping (CAM) method, the model is retrained to learn the deconfounded features of each organ via cross-attention, and we approximate the expectation of the intervention instead of the traditional likelihood. When the model converges, we extract the activation maps by CAM. Our method not only generates high-quality pseudo masks on the CHAOS, ACDC and ProMRI datasets, but is also applicable to other CAM variants. Furthermore, with the refinement, DeMos achieves the dice similarity coefficient of 93.26% on the task of the left ventricle segmentation, which outperforms the state-of-the-art methods.