Capturing Model Uncertainty with Data Augmentation in Deep Learning

Abstract

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2022 SIAM International Conference on Data Mining (SDM)Capturing Model Uncertainty with Data Augmentation in Deep LearningWenming Jiang, Ying Zhao, Yihan Wu, and Haojia ZuoWenming Jiang, Ying Zhao, Yihan Wu, and Haojia Zuopp.271 - 279Chapter DOI:https://doi.org/10.1137/1.9781611977172.31PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Neural network models have been widely used in many fields and achieved many successes. Quantifying model uncertainty, which is able to show the reliability degree of predictions, has attracted more and more researchers' attention. Bayesian neural networks are well known in this category, as they could provide the distributions of predictions, but it takes a prohibitive computational cost to train them. In this paper, we develop a novel way to quantify the model uncertainty of the models trained with data augmentation, i.e., performing data transformation such as adding Gaussian noise to input data before every forward pass of model training and inference. We show that data augmentation is equivalent to performing a corresponding transformation on model weights for some data augmentation methods. We also show that training with Gaussian noise approximates Bayesian inference in Gaussian processes. The experiments on both regression and classification tasks demonstrate that the proposed data augmentation models achieve better predictive performance than baseline models. For all four datasets, the calculated predictive uncertainty can be used as an uncertainty function in selective prediction to reject high risk predictions effectively. Previous chapter Next chapter RelatedDetails Published:2022eISBN:978-1-61197-717-2 https://doi.org/10.1137/1.9781611977172Book Series Name:ProceedingsBook Code:PRDT22Book Pages:1-737