Uncertainty quantification for multilabel text classification

Abstract

AbstractDeep neural networks have recently achieved impressive performance on multilabel text classification. However, the uncertainty in multilabel text classification tasks and their application in the model are often overlooked. To better understand and evaluate the uncertainty in multilabel text classification tasks, we propose a general framework called Uncertainty Quantification for Multilabel Text Classification framework. Based on the prediction results produced by traditional neural networks, the aleatory uncertainty of each classification label and the epistemic uncertainty of the prediction result can further be obtained by this framework. We design experiments to characterize the properties of aleatory uncertainty and epistemic uncertainty from the data characteristics and model features. The experimental results show that this framework is reasonable. Furthermore, we demonstrate how this framework allows us to define the model optimization criterion to identify policies that balance the expected training cost, model performance, and uncertainty sensitivity.This article is categorized under: Algorithmic Development > Bayesian Models