The use of receiver operating characteristic curves and precision-versus-recall curves as performance metrics in unsupervised structural damage classification under changing environment

Abstract

The development of long-term structural health monitoring systems is recently receiving a growing scientific interest in the field of Civil Engineering. In the context of unsupervised learning processes, deviations of dynamic parameters from their normal conditions can allow damage detection. However, due to the fact that modal properties are highly sensitive to environmental and operational factors, it is extremely important to remove such effects in order to obtain suitable damage sensitive features. In this regard, the selection of a proper multivariate statistical data analysis method for removing environmental effects is a key issue, heavily affecting the distribution of the residuals, the control chart and therefore the damage detection. However, specific criteria guiding the optimal definition of such statistical methods are yet to be established. In order to bridge this research gap, an original methodology is proposed in the present paper, based on Receiver Operating Characteristic (ROC) curves in combination with Precision-versus-Recall (PR) curves. Specifically, ROC and PR curves are computed and compared for a variety of statistical methods for data normalization and different damage scenarios and the model selection strategy is formulated as an optimization problem. The proposed approach is exemplified by application in two case studies of continuously monitored structures: the Z24 Bridge in Switzerland and the Consoli Palace, a mediaeval masonry building in Italy. The results highlight that the combined use of both ROC and PR curves represents a suitable tool for defining the most effective statistical data analysis method and the optimal damage threshold, in order to minimize the occurrence of false alarms and missing alarms.