Optimal design of artificial neural networks by a multi-objective strategy: groundwater level predictions

Abstract

Currently, environmental modelling is frequently conducted with the aid of artificial neural networks (ANNs) in an effort to achieve greater accuracy in simulation and forecasting beyond that typically obtained when using solely linear models. For the design of an ANN, modellers must contend with two key issues: (a) the selection of model input and (b) the determination of the number of hidden neurons. A novel approach is introduced to address the optimal design of ANNs based on a multi-objective strategy that enables the user to find a set of feasible ANNs, determined as optimal trade-off solutions between model simplicity and accuracy. This is achieved in a multi-objective fashion by simultaneously minimizing three different cost functions: the model input dimension, the hidden neuron number and the generalization error computed on a validation set of data. The multi-objective approach is based on the Pareto dominance criterion and an evolutionary strategy has been employed to solve the combinatorial optimization problem. From a theoretical perspective, the choice of a multi-objective approach marks an attempt to account for, and overcome, the “curse of dimensionality” and to circumvent the drawbacks of “overfitting” that are inherent in ANNs. Moreover, it is demonstrated that the strategy renders the choice of the ANN more robust, as is evident by “unseen data” in the testing stage, since structure determination is not merely based on the statistical evaluation of the generalization performance. The methodology is tested and the results are reported in a case study relating groundwater level predictions to total monthly rainfall.