Joint Learning of Binary Classifiers and Pairwise Label Correlations for Multi-label Image Classification

Abstract

Typical approaches to multi-label image classification learn a binary classifier for each label, including the binary cross-entropy (CE) loss function in convolutional neural network (CNN) based methods, which learns each label with independent binary logistic regression at the output layer. While these approaches have achieved sufficient success due to the strong learning capability of CNN, further improvements are limited for their neglect of the data imbalance and failure in exploiting explicit label correlations. In this paper, we deal with this problem by jointly learning the binary classifiers and pairwise label correlations (JBP) in an end-to-end manner. For pairwise learning, we introduce the strategy of online hard sample mining to focus on distinguishing confusing label pairs. we also investigate an imbalance aware cross-entropy (ICE) loss function by incorporating cost-sensitivity into the existing cross-entropy loss. Experiments on three popular datasets demonstrate the effectiveness of our proposed method.