Exploiting classifier inter-level features for efficient out-of-distribution detection

Abstract

Deep learning approaches have achieved state-of-the-art performance in a wide range of applications. Most often, however, it is falsely assumed that samples at inference follow a similar distribution as the training data. This assumption impairs models' ability to handle Out-of-Distribution (OOD) data during deployment. While several OOD detection approaches mostly focus on outputs of the last layer, we propose a novel mechanism that exploits features extracted from intermediate layers of a deep classifier. Specifically, we train an off-the-shelf auxiliary network using features of early layers to learn distinctive representations that improve OOD detection. The proposed network can be appended to any classification model without imposing any modification to its original architecture. Additionally, the mechanism does not require access to OOD data during training. We evaluate the performance of the mechanism on a variety of backbone architectures and datasets for near-OOD and far-OOD scenarios. The results demonstrate improvements in OOD detection compared to other state-of-the-art approaches. In particular, our proposed mechanism improves AUROC by 14.2% and 8.3% in comparison to the strong OOD baseline method, and by 3.2% and 3.9% in comparison to the second-best performing approach, on CIFAR-10 and CIFAR-100 datasets respectively.