A Hard Knowledge Regularization Method with Probability Difference in Thorax Disease Images

Abstract

The computer-aided thorax disease diagnosis suffers from the existing noisy labels in large-scale datasets. Especially, the fine-grained thorax images also show high inter-similarity, which leads to the classification task being more challenging. Previous works select the small-loss samples to optimize the deep networks but neglect the knowledge of the hard positive examples with large losses. To address the above problems, in this paper, we propose a novel Probability Difference Regularization method (PDR) to explore more knowledge from the hard positive examples for thorax disease diagnosis. PDR regularizes the deep network with the predicted probability discrepancy between the ground truth and the other classes. We first define a Probability Difference score as the difference between the predicted probability corresponding to the ground truth and the largest probability according to the other categories. Then the PDR regularizes the network by the negative of the PD score. Consequently, PDR benefits the classification in two folds. First, it could enlarge the predicted probability gap between the ground truth and other classes. Second, it enhances the learning of hard examples by introducing extra penalties for incorrect predictions during training. PDR is complimented for both multi-class and multi-label classification problems. We conduct experiments to validate the effectiveness of the proposed method on the ChestX-ray2017 dataset (which easily introduces noisy labels) and the ChestX-ray14 dataset (a real noisy dataset). Experimental results demonstrate that PDR consistently improves the performance of multi-class and multi-label thorax disease classification tasks.