Energy Program Evaluation: Uses, Methods, and Results (Proceedings of the 1991 International Energy Program Evaluation Conference), 636 pages. National Energy Program Evaluation Conference, Chicago.

Abstract

In the probabilistic context, validation metric for models with multiple responses is essentially used to measure the difference between joint statistical distributions resulting from simulation predictions and experimental observations respectively. Considering both uncertainty and correlation, existing validation metrics either ignore correlations among responses or have a relatively huge computational cost. In this paper, by extending the concept of “area metric” and “u-pooling method” developed for validating a scalar response, two new metrics are proposed to validate models with multiple correlated responses using Mahalanobis distance (MD). One new metric is the MD area metric for validating multi-responses at a single validation site. The other is the MD-pooling metric, and it allows for pooling the evidence from all relevant data of multi-response over the intended validation domain into a scalar measure to assess the global predictive capability of computational models. The proposed metrics are applicable to validation for models with multiple correlated responses. Their several favorable properties include objectiveness, affordability, unboundedness, and determinacy of the results. Compared with the existing validation metrics, the feasibility, effectiveness and efficiency of our proposed two metrics are illustrated by a numerical test example and an engineering example.