Abstract
ABSTRACT The calibration of complex computer codes using uncertainty quantification (UQ) methods is a rich area of statistical methodological development. When applying these techniques to simulators with spatial output, it is now standard to use principal component decomposition to reduce the dimensions of the outputs in order to allow Gaussian process emulators to predict the output for calibration. We introduce the “terminal case,” in which the model cannot reproduce observations to within model discrepancy, and for which standard calibration methods in UQ fail to give sensible results. We show that even when there is no such issue with the model, the standard decomposition on the outputs can and usually does lead to a terminal case analysis. We present a simple test to allow a practitioner to establish whether their experiment will result in a terminal case analysis, and a methodology for defining calibration-optimal bases that avoid this whenever it is not inevitable. We present the optimal rotation algorithm for doing this, and demonstrate its efficacy for an idealized example for which the usual principal component methods fail. We apply these ideas to the CanAM4 model to demonstrate the terminal case issue arising for climate models. We discuss climate model tuning and the estimation of model discrepancy within this context, and show how the optimal rotation algorithm can be used in developing practical climate model tuning tools. Supplementary materials for this article are available online.
Highlights
The design and analysis of computer experiments, part of a wider cross-disciplinary endeavour called ‘Uncertainty Quantification’ or ‘uncertainty quantification (UQ)’, has a rich history in statistical methodological development as far back as the landmark paper by Sacks et al (1989)
We showed that even when the prior assessment of model discrepancy is not inconsistent with the ability of the simulator, dimension reduction of spatial output using the ensemble-derived principal components often leads to a terminal case analysis
We proposed a rotation of the singular value decomposition (SVD) basis to highlight and incorporate important lowsignal patterns that may be contained in the original SVD basis, giving a new basis that avoids the terminal case when this is possible
Summary
The design and analysis of computer experiments, part of a wider cross-disciplinary endeavour called ‘Uncertainty Quantification’ or ‘UQ’, has a rich history in statistical methodological development as far back as the landmark paper by Sacks et al (1989). Climate model parameters are often explored individually and tuning done by hand and eye, with the parameters changed, and the new run either accepted or rejected based on heuristic comparison with the current ‘best’ integration Different descriptions of these processes are offered by Mauritsen et al (2012); Williamson et al (2017); Hourdin et al (2017).
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
