Evaluation of artificial intelligence tool performance and uncertainty for predicting sewer structural condition

Abstract

The implementation of a risk-informed asset management system by a wastewater infrastructure utility requires information regarding the probability and the consequences of component failures. This paper focuses on the former, evaluating the performance of artificial intelligence tools, namely artificial neural networks (ANNs) and support vector machines (SVMs), in predicting the structural condition of sewers. The performance of these tools is compared with that of logistic regression on the case study of the wastewater infrastructures of SANEST — Sistema de Saneamento da Costa do Estoril (Costa do Estoril Wastewater System). The uncertainty associated to ANNs and SVMs is quantified and the results of a trial and error approach and the use of optimization algorithms to develop SVMs are compared. The results highlight the need to account for both the performance and the uncertainty in the process of choosing the best model to estimate the sewer condition, since the ANNs present the highest average performance (78.5% correct predictions in the test sample) but also the highest dispersion of performance results (73% to 81% correct predictions in the test sample), whereas the SVMs have lower average performance (71.1% without optimization and 72.6% with the parameters optimized using the Covariance Matrix Adaptation Evolution Strategy) but little variability.