Prediction and structural uncertainty analyses of artificial neural networks using hierarchical Bayesian model averaging

Abstract

This study adopts a hierarchical Bayesian model averaging (HBMA) method to analyze prediction uncertainty resulted from uncertain components in artificial neural networks (ANNs). The HBMA is an ensemble method for prediction and is used to segregate the sources of model structure uncertainty in ANNs and investigate their variance contributions to total prediction variance. Specific sources of uncertainty considered in ANNs include the uncertainty in neural network weights and biases (model parameters), uncertainty of selecting an activation function for the hidden layer, and uncertainty of selecting a number of hidden layer nodes (model structure). Prediction uncertainties due to uncertain inputs and ANN model parameters are represented by within-model variance. Prediction uncertainties due to uncertain activation function and uncertain number of nodes for the hidden layer are represented by between-model variance. The method is demonstrated through a study that employs ANNs to predict fluoride concentration in the aquifers of the Maku area, Azarbaijan, Iran. The results show that uncertain inputs and ANN model parameters produces the most prediction variance, followed by prediction variances from uncertain number of hidden layer nodes and uncertain activation function.