Confidence metric for signal classification in Non Destructive Evaluation

Abstract

Quantitative assessment of the reliability of defect classification is critical in non-destructive evaluation (NDE) applications. Particularly in automated data analysis systems, such a measure enables the system to monitor its own performance and automatically flag indications where operator intervention is required. Apart from inherent ambiguity of non-discriminative features and inadequate training samples, noisy measurement is a primary reason underlying the classifier's unreliable decisions. In this paper, we have developed a framework to incorporate the major sources of classification errors into a single quantitative measure. By bootstrapping and weighting Bayes posterior probability with estimated noise distribution, effect of noise in NDE measurements is embedded in the resultant confidence measure. The effectiveness of the proposed method is first demonstrated on synthetic dataset from an eddy current simulation model. It is then used to analyze confidence of classifying experimental data from eddy current inspection of defects in steam generator tubes.