Distribution Based Upper Lower Bound Estimation In Deep Neural Nets

Abstract

A crucial part of forecasting is quantifying the uncertainties involved during prediction. Neural networks are great at making points estimates. However, the lack of uncertainty quantification through a predictive distribution has constrained their application in sensitive areas. In this paper we propose a simple distribution based upper lower bound estimation algorithm. We mathematically demonstrate a distribution based coverage probability and interval width assessment method. Our approach encompasses the quality metrics, prediction interval coverage probability (PICP) and mean prediction interval width (MPIW) in its bound estimation. Then, we developed a customized loss function with adaptive hyperparameter that balances the needed coverage probability in relation to the prediction interval. Finally, we evaluated the performance of our approach on a UCI regression data using the recent Quality Driven (QD) bound estimation method as a benchmark. The experiment has revealed the algorithm offers a simple tuning and a stable evaluation for a predictive distribution regardless of the network settings. In the simulation for the same 95% coverage probability, we observed the algorithm achieved the minimum prediction interval. Although, we assumed a logistically approximated Gaussian distribution for the algorithm derivation, its has a robust response to data with asymmetric distribution as well. Hence, the algorithm can be used as an alternative method to provide a predictive distribution for neural networks.