Two-stream collaborative network for multi-label chest X-ray Image classification with lung segmentation

Abstract

Automated chest X-ray (CXR) image analysis is often subject to serious disruption and misguided by its imaging artifacts and noise regions. To minimize such negative effects, many state-of-the-art works have made great efforts in the precise segmentation of lung fields. Based on these works, some image-based features can be extracted directly from lung filed to provide clues for many types of lung diseases such as lung nodule, cardiomegaly, pneumothorax, or emphysema. In this paper, we propose a novel two-stream collaborative network call TSCN for multi-label CXR image classification based on lung segmentation. Specifically, we first train a robust lung segmentor with U-Net and apply it to capture the lung filed from the original CXR image. Then we perform a two-stream feature fusion operation to aggregate the contextual information in both the global image and lung field for complementary feature learning. By taking advantage of the two-stream deep structures and two types of image inputs, a novel self-adaptive weighted fusion scheme is designed to jointly learn these two feature streams in the end-to-end training phase, in order to realize adaptive two-stream feature subsets selection and optimization. Extensive experiments on the ChestX-ray14 dataset demonstrate the effectiveness of the proposed method as compared with the state-of-the-art baselines.