Abstract
Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnostics to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, fluctuations, and equilibrium gradients against observation. The utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak [J. L. Luxon, Nucl. Fusion 42, 614 (2002)], as part of a multi-year transport model validation activity.
Highlights
All models are wrong, but some are useful. —George E
Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnostics to enable quantitatively meaningful comparisons between simulation and experiment
The utility of metrics based upon these comparisons is demonstrated by applying them to gyrokinetic predictions of turbulent transport in a variety of discharges performed on the DIII-D tokamak [J
Summary
Given the importance of rigorous validation studies in many fields, it is no surprise that there is a broad and well-established literature in the field, ranging from guidelines and best practices prescribed by professional societies[1,5] to journal reviews[6,7,8,9,10,11] and textbooks,[12,13] in addition to the numerous articles and reports detailing the outcomes of individual studies. A common feature of both these reviews and the broader literature is the identification of validation metrics as key components of any serious, robust validation study While both the Terry et al and Greenwald reviews discuss these metrics in some detail, including potential mathematical formulations, both emphasize the need for further work. Of these issues, this paper will use the formulation of validation metrics relevant for turbulent transport in MFE plasmas as a type of worked example.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have