BFAct: Out-of-Distribution Detection with Butterworth Filter Rectified Activations

Abstract

AbstractOut-of-Distribution (OOD) Detection has drawn a lot of attention recently due to it being an essential building block for safely deploying neural network models in real-life applications. The challenge in this field is that modern neural network tends to produce overconfident predictions on OOD data, which work against the principles of OOD detection techniques. To overcome this challenge, we propose Butterworth Filter rectified Activations (BFAct), a technique to rectify activations and drastically alleviate the overconfidence predictions on OOD data. Our work is motivated by an analysis of a neural network’s internal activation and proved to be a surprisingly effective post hoc method for the OOD Detection task. The advantage of using Butterworth Filter is that the passband of the filter has a smooth and monotonically decreasing frequency response, which helps to correct the abnormal activations to normal distribution. BFAct is not only able to generalize on various neural network architectures but also compatible with various OOD score functions. Our main experiments are evaluated on a large-scale OOD Detection benchmark based on ImageNet-1k, which is closer to practical application scenarios. We also conduct experiments on CIFAR-10 and CIFAR-100. The results illustrate that our method outperforms the state-of-the-art on both large-scale and common benchmarks.KeywordsOut-of-distribution detectionNeural networkActivationsButterworth filter