Uncertainty Quantification via Deep Ensembles in Missile Performance Prediction

Abstract

Advances in computational technologies have enabled engineers to obtain large amounts of data efficiently. One of the most widely used techniques is the regression model and numerous engineers who are interested in the reliability of their predictions have used the Gaussian process regression. However, time and memory complexities of Gaussian process regression hinder it from being efficiently exploited in data-intensive engineering disciplines where high-dimensional or large training datasets are frequently dealt with. In this regard, deep neural networks have recently attracted the attention of engineers owing to their universal approximation capabilities. Despite these advantages, they have a critical drawback: the predictions from them are overconfident and therefore miscalibrated. Due to Gaussian process regression’s inefficiency in large datasets and deep neural networks’ inability to estimate uncertainty, Bayesian neural networks are extensively adopted in the decision-making process under predictive uncertainty. Though their advantages make them attractive, they are known to be impractical for engineering applications due to complex and cumbersome training algorithms along with prohibitive computational costs. Accordingly, simple but scalable methods for approximate Bayesian inference have been proposed recently. Among them, deep ensembles approach provides not only accurate predictions, but also reliable and practically useful uncertainty estimates on a variety of datasets and architectures. In this study, this state-of-the-art approximate Bayesian inference approach is investigated. The purpose of this manuscript is to introduce and examine the applicability of the DE approach to the multi-output regression task in aerodynamic discipline. Especially, the effects of the number of network members in the ensemble are investigated, which have been overlooked in most previous studies. Their results demonstrate that though the predictions from DE model become accurate as the number of members increases, the trend that DE model becomes underconfident is observed. Finally, we proposed to apply a simple and straightforward post-processing uncertainty calibration method to address these pitfalls of DE, and its effectiveness is presented in a quantitative manner.