Multi-label out-of-distribution detection via exploiting sparsity and co-occurrence of labels

Abstract

Although Out-Of-Distribution (OOD) detection has been extensively studied, the OOD detection under multi-label settings, which is closer to the real world, is still in its infancy. The pioneer work ignores some unique properties of multi-label images, such as the sparsity and co-occurrence of labels. Here, we empirically observe that these properties readily distinguish OOD and in-distribution data. Motivated by this observation, we propose a novel multi-label OOD detection approach named Sparse Label Co-occurrence Scoring (SLCS) to exploit the sparsity and co-occurrence information of labels. SLCS follows conventions and deems the logits outputted by the penultimate layer of the trained multi-label image classification model as the prediction confidences of a sample to categories in the training label set. A logit sparse filtering process is employed to filter out the low-confidence logits for avoiding the interference of low-confidence predictions while preserving the high-confidence logits to obtain the label sparsity. Then, the label co-occurrence pairs are counted for each sample based on its predicted categories and the label co-occurrence matrix constructed on the training set. Finally, the preserved logits are weighted by the label co-occurrence information and accumulated to produce the OOD detection score for each sample. Extensive experimental results on three well-known multi-label image datasets demonstrate the discriminating power of SLCS, which achieves greatly improved performances compared with the only multi-label OOD detection approach — JointEnergry and the state-of-the-art single-label OOD detection approaches. The performance improvements of SLCS over JointEnergy in FPR95 are 12.85%, 12.41%, and 9.50% on MS-COCO, VOC 2012, and NUS-WIDE datasets respectively.