Representation Of Model Uncertainty Research Articles

A Combined Scheme based on the Multiscale Stochastic Perturbed Parameterization Tendencies and Perturbed Boundary Layer Parameterization for a Global Ensemble Prediction System

Abstract Stochastic representations of model uncertainties are of great importance for the performance of ensemble prediction systems (EPSs). The stochastically perturbed parameterization tendencies (SPPT) scheme with a single-scale random pattern has been used in the operational global EPS of China Meteorological Administration (CMA-GEPS) since 2018. To deal with deficiencies in this operational single-scale SPPT scheme, a combined scheme based on the multiscale SPPT (mSPPT) scheme and the stochastically perturbed parameterization for the planetary boundary layer (SPP-PBL) scheme is developed. In the combined scheme, the mSPPT component aims to expand model uncertainties characterized by SPPT at mesoscale, synoptic scale, and planetary scale. The SPP-PBL component with six vital parameters is used to capture uncertainties in PBL processes, which is underrepresented by SPPT for the tapering treatment within PBL. Comparisons between the operational SPPT scheme and the mSPPT scheme reveal that the mSPPT scheme can generate more improvements in both ensemble reliability and forecast skills mainly in tropics. Besides, additional benefits from SPP-PBL on top of mSPPT are shown to be primarily distributed in tropics at the lower layers below 850 hPa and surface. Furthermore, the combined scheme of mSPPT and SPP-PBL is suggested to yield better spread–error relationships and forecast skills than the operational SPPT scheme in terms of objective verification scores for standard upper-air variables and surface parameters. A case study for the extreme precipitation event on 20 July 2021 in Henan Province of China also demonstrates the better ability of the combined scheme in forecasting the precipitation intensity and location. Significance Statement A comprehensive and reasonable representation of model uncertainties helps to improve the performance of ensemble prediction systems (EPSs). Despite the popular usage in simulating model uncertainties, the stochastically perturbed parameterization tendencies (SPPT) scheme possesses several shortcomings. To overcome this, a combined scheme based on the multiscale SPPT (mSPPT) scheme and the stochastically perturbed parameterization for the planetary boundary layer (SPP-PBL) scheme is proposed. Based on the global EPS of China Meteorological Administration (CMA-GEPS), additional benefits from the independent usage of mSPPT and SPP-PBL are disclosed. And the combined scheme can inherit the merits of mSPPT and SPP-PBL and generate more improvements on ensemble performance than the original single-scale SPPT scheme. This research provides a guideline for future upgradation of CMA-GEPS.

A novel approach to a multi-model ensemble for climate change models: Perspectives on the representation of natural variability and historical and future climate

This study developed a novel approach that integrated climate model selection and multi-model ensemble (MME) construction to effectively represent model uncertainties and, consequently, improve consistency in the evaluation of changes to extreme rainfall in different scenarios. Our focus was to combine 10 regional climate model (RCM) simulations, forced by two global climate models (GCM), especially for estimation of design rainfall in a changing climate. We hypothesized that the natural variability and statistically higher moment attributes in the extreme rainfall simulations from RCMs were not fully preserved. Therefore, the MME approach could be more effective in climate change studies, largely due to the use of multiple climate models. First, an experimental study was proposed to validate the efficacy of the proposed modeling framework approach adopting L-moments to quantify relative importance among climate models and their use for representing natural variability in the MME construction. The proposed approach was then applied to climate change scenarios collected from multiple RCMs for the Han-River watershed during both historical (1981–2005) and future (2006–2 100) periods. The results showed that the climate model selection informed by natural variability demonstrated better performance, representing nearly identical distribution to the observed annual maximum rainfall (AMR) in the Han River watershed. The range of the selected scenario was relatively narrower than that of all the scenarios, and the change rate was more consistent with the limited zero crossing, reflecting improvement in both model performance and consistency over historical and future periods, respectively. The change rate of the MME under RCP8.5 appeared to be an approximately 20% increase for the near (2011–2040) and far (2071–2 100) future, and the degree of increase in the rate for the mid-future (2041–2070) was slightly lower than that for the other periods, with increase of approximately 10%.

Uncertainty growth and forecast reliability during extratropical cyclogenesis

Abstract. In global numerical weather prediction, the strongest contribution to ensemble variance growth over the first few days is at synoptic scales. Hence it is particularly important to ensure that this synoptic-scale variance is reliable. Here we focus on wintertime synoptic-scale growth in the North Atlantic storm track. In the 12 h background forecasts of the Ensemble of Data Assimilations (EDA) from the European Centre for Medium-Range Weather Forecasts (ECMWF), we find that initial variance growth at synoptic scales tends to be organized in particular flow situations, such as during the deepening of cyclones (cyclogenesis). Both baroclinic and diabatic aspects may be involved in the overall growth rate. However, evaluation of reliability through use of an extended error–spread equation indicates that the ECMWF ensemble forecast, which is initialized from the EDA but with additional singular vector perturbations, appears to have too much variance at a lead time of 2 d and that this over-spread is associated with cyclogenesis situations. Comparison of variance growth rates and reliability with other forecast systems within The International Grand Global Ensemble (TIGGE) archive indicates some sensitivity to the model or its initialization. For the ECMWF ensemble forecast, sensitivity experiments suggest that a large part of the total day-2 spread in cyclogenesis cases is associated with the growth of EDA uncertainty, but up to 25 % can be associated with the additional singular vector perturbations to the initial conditions and up to 25 % with the representation of model uncertainty. The sensitivities of spread to resolution, the explicit representation of convection, and the assimilation of local observations are also considered. The study raises the question of whether the EDA now successfully represents initial uncertainty (and the enhanced growth rates associated with cyclogenesis) to the extent that singular vector perturbations could be reduced in magnitude to improve storm track reliability. This would leave a more seamless forecast system, allowing short-range diagnostics to better help improve the model and model-uncertainty representation, which could be beneficial throughout the forecast range.

A new approach to represent model uncertainty in the forecasting of tropical cyclones: The orthogonal nonlinear forcing singular vectors

AbstractTropical cyclone (TC) track forecasting has been considerably improved in recent decades, while TC intensity forecasting remain challenging. In this study, orthogonal nonlinear forcing singular vectors (O‐NFSVs) for emulating the impact of model uncertainties are used to conduct TC ensemble forecasting experiments with the Weather Research and Forecasting (WRF) model, with a focus on improving TC intensity forecasting skill. The O‐NFSVs approach is compared with the traditional stochastic kinetic‐energy backscatter (SKEB) and stochastically perturbed parametrization tendency (SPPT) schemes. The results demonstrate that the O‐NFSVs ensembles generally provide a better representation of the model uncertainties affecting TC intensification, with much better deterministic and probabilistic skills. These results also extend to the ability to forecast TC track, although the perturbations have not been optimized for that specific purpose. The O‐NFSVs are therefore appropriate perturbation structures for describing the uncertainties of the TC intensity and track forecasting and are also favourable for recognizing the rapid intensification process.

Scaling up uncertainties in allometric models: How to see the forest, not the trees

Quantifying uncertainty in forest assessments is challenging because of the number of sources of error and the many possible approaches to quantify and propagate them. The uncertainty in allometric equations has sometimes been represented by propagating uncertainty only in the prediction of individuals, but at large scales with large numbers of trees uncertainty in model fit is more important than uncertainty in individuals. We compared four different approaches to representing model uncertainty: a formula for the confidence interval, Monte Carlo sampling of the slope and intercept of the regression, bootstrap resampling of the allometric data, and a Bayesian approach. We applied these approaches to propagating model uncertainty at four different scales of tree inventory (10 to 10,000 trees) for four study sites with varying allometry and model fit statistics, ranging from a monocultural plantation to a multi-species shrubland with multi-stemmed trees. We found that the four approaches to quantifying uncertainty in model fit were in good agreement, except that bootstrapping resulted in higher uncertainty at the site with the fewest trees in the allometric data set (48), because outliers could be represented multiple times or not at all in each sample. The uncertainty in model fit did not vary with the number of trees in the inventory to which it was applied. In contrast, the uncertainty in predicting individuals was higher than model fit uncertainty when applied to small numbers of trees, but became negligible with 10,000 trees. The importance of this uncertainty source varied with the forest type, being largest for the shrubland, where the model fit was most poor. Low uncertainties were observed where model fit was high, as was the case in the monoculture plantation and in the subtropical jungle where hundreds of trees contributed to the allometric model. In all cases, propagating uncertainty only in the prediction of individuals would underestimate allometric uncertainty. It will always be most correct to include both uncertainty in predicting individuals and uncertainty in model fit, but when large numbers of individuals are involved, as in the case of national forest inventories, the contribution of uncertainty in predicting individuals can be ignored. When the number of trees is small, as may be the case in forest manipulation studies, both sources of allometric uncertainty are likely important and should be accounted for.

An attention-based domain spatial-temporal meta-learning (ADST-ML) approach for PM2.5 concentration dynamics prediction

Because PM2.5 causes serious health impacts, accurately predicting PM2.5 concentration is important for air pollution mitigation. The problem is to capture its dynamic evolution and continuously predict changes based on historical observations. However, the task of prediction is extremely challenging due to the spatial-temporal correlation of PM2.5 (i.e., dynamic changes over time and geographic location) and spatial-temporal uncertainty (i.e., it is affected by multiple factors in temporal and spatial distribution). We propose a novel approach of attention-based domain spatial-temporal meta-learning (ADST-ML), which can train the distribution of PM2.5 prediction tasks in an area through historical PM2.5 data, meteorological data, and temporal feature data, thereby learning a strategy that can quickly adapt to tasks such as predicting the unobservable process by which the concentration of PM2.5 changes. Our prediction model has the following innovations: (1) ADST-ML has a novel network architecture design, using convolutional neural networks (CNN), an attention mechanism, and a long short-term memory (LSTM) network model to adaptively extract the spatial-temporal dependence of PM2.5 and other multivariate factors; (2) ADST-ML captures the dynamic changes of PM2.5 prediction tasks through variational inference; (3) ADST-ML designs meta-training and testing algorithms based on Bayesian meta-learning to represent model uncertainty and knowledge transfer from training data. When conducting experiments on a dataset containing Beijing air quality inspection records, our ADST-ML outperforms baselines in terms of prediction accuracy. Since it is a regional forecasting model, it breaks through some of the limitations of forecasting based on the original site, and offers the possibility of accurate forecasting in urban areas.

Accelerating Bayesian Neural Networks via Algorithmic and Hardware Optimizations

Bayesian neural networks (BayesNNs) have demonstrated their advantages in various safety-critical applications, such as autonomous driving or healthcare, due to their ability to capture and represent model uncertainty. However, standard BayesNNs require to be repeatedly run because of Monte Carlo sampling to quantify their uncertainty, which puts a burden on their real-world hardware performance. To address this performance issue, this article systematically exploits the extensive structured sparsity and redundant computation in BayesNNs. Different from the unstructured or structured sparsity in standard convolutional NNs, the structured sparsity of BayesNNs is introduced by Monte Carlo Dropout and its associated sampling required during uncertainty estimation and prediction, which can be exploited through both algorithmic and hardware optimizations. We first classify the observed sparsity patterns into three categories: channel sparsity, layer sparsity and sample sparsity. On the algorithmic side, a framework is proposed to automatically explore these three sparsity categories without sacrificing algorithmic performance. We demonstrated that structured sparsity can be exploited to accelerate CPU designs by up to 49 times, and GPU designs by up to 40 times. On the hardware side, a novel hardware architecture is proposed to accelerate BayesNNs, which achieves a high hardware performance using the runtime adaptable hardware engines and the intelligent skipping support. Upon implementing the proposed hardware design on an FPGA, our experiments demonstrated that the algorithm-optimized BayesNNs can achieve up to 56 times speedup when compared with unoptimized Bayesian nets. Comparing with the optimized GPU implementation, our FPGA design achieved up to 7.6 times speedup and up to 39.3 times higher energy efficiency.

Using Stochastically Perturbed Parameterizations to Represent Model Uncertainty. Part I: Implementation and Parameter Sensitivity

Abstract Accurately representing model-based sources of uncertainty is essential for the development of reliable ensemble prediction systems for NWP applications. Uncertainties in discretizations, algorithmic approximations, and diabatic and unresolved processes combine to influence forecast skill in a flow-dependent way. An emerging approach designed to provide a process-level representation of these potential error sources, stochastically perturbed parameterizations (SPP), is introduced into the Canadian operational Global Ensemble Prediction System. This implementation extends the SPP technique beyond its typical application to free parameters in the physics suite by sampling uncertainty both within the dynamical core and at the formulation level using “error models” when multiple physical closures are available. Because SPP perturbs components within the model, internal consistency is ensured and conservation properties are not affected. The full SPP scheme is shown to increase ensemble spread to keep pace with error growth on a global scale. The sensitivity of the ensemble to each independently perturbed “element” is then assessed, with those responsible for the bulk of the response analyzed in more detail. Perturbations to surface exchange coefficients and the turbulent mixing length have a leading impact on near-surface statistics. Aloft, a tropically focused error model representing uncertainty in the advection scheme is found to initiate growing perturbations on the subtropical jet that lead to forecast improvements at higher latitudes. The results of Part I suggest that SPP has the potential to serve as a reliable representation of model uncertainty for ensemble NWP applications. Significance Statement Ensemble systems account for the negative impact that uncertainties in prediction models have on forecasts. Here, uncertain model parameters and algorithms are subjected to perturbations representing impact on forecast errors. By initiating error growth within the model calculations, the equally skillful members of the ensemble remain physically realistic and self-consistent, which is not guaranteed by other depictions of model error. This “stochastically perturbed parameterization” technique (SPP) comprises many small error sources, each analyzed in isolation. Each source is related to a limited set of processes, making it possible to determine how the individual perturbations affect the forecast. We conclude that SPP in the Canadian Global Ensemble Forecasting System produces realistic estimates of the impact of model uncertainties on forecast skill.

Using Stochastically Perturbed Parameterizations to Represent Model Uncertainty. Part II: Comparison with Existing Techniques in an Operational Ensemble

Abstract The ability of a stochastically perturbed parameterization (SPP) approach to represent uncertainties in the model component of the Canadian Global Ensemble Prediction System was demonstrated in Part I of this investigation. The goal of this second step in SPP evaluation is to determine whether the scheme represents a viable alternative to the current operational combination of a multiphysics configuration and stochastically perturbed parameterization tendencies (SPPT). An assessment of the impact of each model uncertainty estimate in isolation reveals that, although the multiphysics configuration is highly effective at generating ensemble spread, it is often the result of differing biases rather than a reflection of flow-dependent error growth. Moreover, some of the members of the multiphysics ensemble suffer from large errors on regional scales as a result of suboptimal configurations. The SPP scheme generates a greater diversity of member solutions than the SPPT scheme in isolation, and it has an impact on forecast performance that is similar to that of current operational uncertainty estimates. When the SPP framework is combined with recent upgrades to the model physics suite that are only applicable in the stochastic perturbation context, the quality of global ensemble guidance is significantly improved. Significance Statement The stochastically perturbed parameterization (SPP) technique was introduced in Part I to represent model uncertainties in forecasts generated by an operational global ensemble prediction system. We focus here on the viability of this technique as a replacement for the system’s current uncertainty estimates: multiphysics and stochastic perturbations of physics tendencies. Despite the practical success of this combination, it suffers from physical inconsistencies and poor conservation properties. The adoption of SPP allows the ensemble to benefit from a recent set of model updates that couple with this new representation of model uncertainty to yield significant improvements in the quality of forecasts generated by the system.

A Riemannian stochastic representation for quantifying model uncertainties in molecular dynamics simulations

A Riemannian stochastic representation of model uncertainties in molecular dynamics is proposed. The approach relies on a reduced-order model, the projection basis of which is randomized on a subset of the Stiefel manifold characterized by a set of linear constraints defining, e.g., Dirichlet boundary conditions in the physical space. We first show that these constraints are, indeed, preserved through Riemannian pushforward and pullback actions to, and from, the tangent space to the manifold at any admissible point. This fundamental property is subsequently exploited to derive a probabilistic model that leverages the multimodel nature of the atomistic setting. The proposed formulation offers several advantages, including a simple and interpretable low-dimensional parameterization, the ability to constraint the Fréchet mean on the manifold, and ease of implementation and propagation. The relevance of the proposed modeling framework is finally demonstrated on various applications including multiscale simulations on graphene-based systems.

Stochastically Perturbed Parameterizations for the Process-Level Representation of Model Uncertainties in the CMA Global Ensemble Prediction System

Stochastically Perturbed Parameterizations for the Process-Level Representation of Model Uncertainties in the CMA Global Ensemble Prediction System

A stochastic and non‐linear representation of model uncertainty in a convective‐scale ensemble prediction system

AbstractAccurately addressing model uncertainties with a consideration of the enhanced effect of non‐linearities in a high‐resolution convective‐scale system is a crucial issue for performing convection‐allowing ensemble prediction systems (CAEPSs). In this study, a conditional non‐linear–stochastic perturbation method is developed to simultaneously consider both a stochastic and a non‐linear representation of model uncertainties associated with physics parameterization in the Global and Regional Assimilation and Prediction Enhanced System (GRAPES)‐CAEPS with a horizontal resolution of 3 km. The non‐linear forcing singular vector (NFSV) for a non‐linear representation of model uncertainties and the Stochastically Perturbed Parameterization Tendencies (SPPT) scheme for a stochastic representation of model uncertainties, are applied. Two experiments were carried out over South China for a month (May 1–30, 2020), one with a SPPT scheme and the other with a non‐linear–stochastic perturbation using a combination of SPPT and NFSV schemes. The combination of SPPT and NFSV schemes is compared with the SPPT scheme alone to investigate whether the conditional non‐linear–stochastic perturbation method that combines non‐linear and stochastic schemes can represent model uncertainty better than the traditional stochastic SPPT approach. The results show that combining the NFSV and SPPT schemes improves the overall probabilistic skill and has an advantage over the SPPT scheme, which may imply that adding additional state‐independent non‐linear noise contributes to a more comprehensive characterization of model error for representing model uncertainties in CAEPSs. This discovery sheds light on the design and development of model perturbation strategies for convective‐scale ensembles in the future.

A Nonlinear Representation of Model Uncertainty in a Convective-Scale Ensemble Prediction System

A Nonlinear Representation of Model Uncertainty in a Convective-Scale Ensemble Prediction System

Model Uncertainty Representation in a Convection-Permitting Ensemble—SPP and SPPT in HarmonEPS

Abstract The stochastically perturbed parameterizations scheme (SPP) is here implemented and tested in HarmonEPS—the convection-permitting limited area ensemble prediction system by the international research program High Resolution Limited Area Model (HIRLAM) group. SPP introduces stochastic perturbations to values of chosen closure parameters representing efficiencies or rates of change in parameterized atmospheric (sub)processes. The impact of SPP is compared to that of the stochastically perturbed parameterization tendencies scheme (SPPT). SPP in this first version in HarmonEPS perturbs 11 parameters, active in different atmospheric processes and under various weather conditions. The main motivation for this study is the lack of variability seen in cloud products in HarmonEPS, as reported by duty forecasters. SPP in this first version is able to increase variability in a range of weather variables, including the cloud products. However, for some weather variables the root-mean-squared error of the ensemble mean is increased and the mean bias is impacted, especially in winter. This indicates that (some) parameter perturbation distributions are not optimal in the current configuration, and a further sensitivity analysis is required. SPPT resulted in a smaller increase in variability in the ensemble, but the impact was almost completely masked out when combined with perturbations of the initial state, lateral boundaries, and surface properties. An in-depth investigation into this lack of impact from SPPT is here presented through examining, among other things, accumulated tendencies from the model physics. Significance Statement Small inaccuracies, simplifications, or errors in any part of a complex and nonlinear system like a weather model can amplify and in a short time become significant. We wanted to introduce a physically consistent way of representing these uncertainties in a model that is used in several European countries. To do this we introduce variations in a few parameters that are used in the model description, and that we know are uncertain. By doing this we were able to increase the variability of the cloud products as desired. We see this as a promising approach for capturing the possibilities of fog occurring or not in this model. Further refinements are needed before it can be used in operational weather forecasts.

Process‐oriented stochastic perturbations applied to the parametrization of turbulence and shallow convection for ensemble prediction

AbstractWeather forecasting nowadays often requires some estimation of uncertainties associated with the output of meteorological models, in order to better inform decision making, especially in the context of intense weather events. Ensemble prediction systems provide such information through sets of possible scenarios which are designed to represent various uncertainty sources, including model uncertainties. A wide variety of methods have been proposed to estimate model uncertainties, among which perturbation methods targeting uncertain processes are a promising research field. In this study, we focus on the representation of small‐scale variability by process‐oriented perturbation schemes applied to two key physical processes, namely turbulence and shallow convection. The perturbations are applied to a single‐column version of the convection‐permitting AROME model, in three idealized boundary‐layer cases. Large‐eddy simulations (LESs) of the same cases serve as a reference for the subgrid variability that has to be represented, and the results are also compared to those given by the Stochastically Perturbed Parametrization Tendencies (SPPT) method, which is a method commonly used by weather forecast centres to represent model uncertainty. The spread produced by our process‐oriented perturbations of turbulence and shallow convection does not represent all the small‐scale variability implied by the LESs for temperature and humidity. However, it is of a similar order of magnitude for the wind, thanks to perturbations generated by the stochastic turbulence scheme. The dispersion is structurally different from what is obtained with SPPT. It is non‐negligible in the lower levels, where SPPT perturbations are usually suppressed because of numerical instabilities, indicating a possible complementarity between the schemes.

Ordering and inequalities for mixtures on risk aggregation

AbstractAggregation sets, which represent model uncertainty due to unknown dependence, are an important object in the study of robust risk aggregation. In this paper, we investigate ordering relations between two aggregation sets for which the sets of marginals are related by two simple operations: distribution mixtures and quantile mixtures. Intuitively, these operations “homogenize” marginal distributions by making them similar. As a general conclusion from our results, more “homogeneous” marginals lead to a larger aggregation set, and thus more severe model uncertainty, although the situation for quantile mixtures is much more complicated than that for distribution mixtures. We proceed to study inequalities on the worst‐case values of risk measures in risk aggregation, which represent conservative calculation of regulatory capital. Among other results, we obtain an order relation on VaR under quantile mixture for marginal distributions with monotone densities. Numerical results are presented to visualize the theoretical results and further inspire some conjectures. Finally, we provide applications on portfolio diversification under dependence uncertainty and merging p‐values in multiple hypothesis testing, and discuss the connection of our results to joint mixability.

Representing Model Uncertainty in Multiannual Predictions

AbstractThe most prominent way to account for model uncertainty is through the pragmatic combination of simulations from individual climate models into a multimodel ensemble (MME). However, alternative approaches to represent intrinsic model errors within single‐model ensembles (SMEs) using stochastic parameterizations have proven beneficial in numerical weather prediction. Nevertheless, stochastic parameterizations are not included in most current decadal prediction systems. Here, the effect of the stochastically perturbed physical tendency (SPPT) scheme is examined in 28‐month predictions using ECMWF's forecast model and contrasted with a MME constructed from current decadal prediction systems. Compared to SMEs, SPPT improves the skill and reliability of tropical sea surface temperature forecasts during the first 18 months (similar to the MME). Thus, stochastic schemes can be an effective and low‐cost alternative to be used separately or in conjunction with the multimodel combination to improve the reliability of climate predictions on multiannual time scales.

Revision of the Stochastically Perturbed Parametrisations model uncertainty scheme in the Integrated Forecasting System

AbstractThe Stochastically Perturbed Parametrisations scheme (SPP) represents model uncertainty in numerical weather prediction by introducing stochastic perturbations into the physical parametrisation schemes. The perturbations are constructed in such a way that the internal consistency of the physical parametrisation schemes is preserved. We developed a revised version of SPP for the Integrated Forecasting System of the European Centre for Medium‐Range Weather Forecasts (ECMWF). The revised version introduces perturbations to additional quantities and modifies the probability distributions sampled by the scheme. Medium‐range ensemble forecasts with the revised SPP are considerably more skilful than ensemble forecasts with the original implementation of SPP. The revised version of SPP is similar, in terms of forecast skill, to the Stochastically Perturbed Parametrisation Tendency scheme (SPPT), which is currently used to represent model uncertainty in the operational ECMWF ensemble forecasts.

What Does a Convection-Allowing Ensemble of Opportunity Buy Us in Forecasting Thunderstorms?

AbstractThe High Resolution Ensemble Forecast v2.1 (HREFv2.1), an operational convection-allowing model (CAM) ensemble, is an “ensemble of opportunity” wherein forecasts from several independently designed deterministic CAMs are aggregated and postprocessed together. Multiple dimensions of diversity in the HREFv2.1 ensemble membership contribute to ensemble spread, including model core, physics parameterization schemes, initial conditions (ICs), and time lagging. In this study, HREFv2.1 forecasts are compared against the High Resolution Rapid Refresh Ensemble (HRRRE) and the Multiscale data Assimilation and Predictability (MAP) ensemble, two experimental CAM ensembles that ran during the 5-week Spring Forecasting Experiment (SFE) in spring 2018. The HRRRE and MAP are formally designed ensembles with spread achieved primarily through perturbed ICs. Verification in this study focuses on composite radar reflectivity and updraft helicity to assess ensemble performance in forecasting convective storms. The HREFv2.1 shows the highest overall skill for these forecasts, matching subjective real-time impressions from SFE participants. Analysis of the skill and variance of ensemble member forecasts suggests that the HREFv2.1 exhibits greater spread and more effectively samples model uncertainty than the HRRRE or MAP. These results imply that to optimize skill in forecasting convective storms at 1–2-day lead times, future CAM ensembles should employ either diverse membership designs or sophisticated perturbation schemes capable of representing model uncertainty with comparable efficacy.

Ensemble PM2.5 Forecasting During the 2018 Camp Fire Event Using the HYSPLIT Transport and Dispersion Model

AbstractBiomass burning releases a vast amount of aerosols into the atmosphere, often leading to severe air quality and health problems. Prediction of the air quality effects from biomass burning emissions is challenging due to uncertainties in fire emission, plume rise calculation, and other model inputs/processes. Ensemble forecasting is increasingly used to represent model uncertainties. In this paper, an ensemble forecast was conducted to predict surface PM2.5 during the 2018 California Camp Fire event using the National Oceanic and Atmospheric Administration (NOAA) HYSPLIT dispersion model at 0.1° horizontal resolution. Different combinations of four satellite‐based fire emission data sets (FEER, FLAMBE, GBBEPx, and GFAS), two plume rise schemes (Briggs and Sofiev), various meteorology inputs, and model setup options were used to create the forecast ensemble, for a total of 112 experiments. The performance of each ensemble member and the ensemble mean were evaluated using ground‐based observations, with four statistical metrics and an overall rank. The ensemble spread of the 112 members reached 1,000 μg/m3, highlighting the large uncertainty in wildfire forecast. The ensemble mean displayed the best performance. Each fire emission product contributed to one or more members among the top 10 performers, revealing the forecasting dependence on both the quality of fire emissions data and model representation of emission, transport, and removal processes. In addition, an ensemble size reduction technique was introduced. With the help of this technique, the ensemble size was reduced from 112 to 28 members and still produced an ensemble mean that yielded comparable or even better performance to that of the full ensemble.