Model Forecast Errors Research Articles

Examination of Nudging Schemes in the Simulation of Meteorology for Use in Air Quality Experiments: Application in the Great Lakes Region

AbstractHigh levels of ozone have been observed along the shores of Lake Michigan for the last 40 years. Models continue to struggle in their ability to replicate ozone behavior in the region. In the retrospective way in which models are used in air quality regulation development, nudging or four-dimensional data assimilation (FDDA) of the large-scale environment is important for constraining model forecast errors. Here, paths for incorporating large-scale meteorological conditions but retaining model mesoscale structure are evaluated. For the July 2011 case studied here, iterative FDDA strategies did not improve mesoscale performance in the Great Lakes region in terms of diurnal trends or monthly averaged statistics, with overestimations of nighttime wind speed remaining as an issue. Two vertical nudging strategies were evaluated for their effects on the development of nocturnal low-level jets (LLJ) and their impacts on air quality simulations. Nudging only above the planetary boundary layer, which has been a standard option in many air quality simulations, significantly dampened the amplitude of LLJ relative to nudging only above a height of 2 km. While the LLJ was preserved with nudging only above 2 km, there was some deterioration in wind performance when compared with profiler networks above the jet between 500 m and 2 km. In examining the impact of nudging strategies on air quality performance of the Community Multiscale Air Quality model, it was found that performance was improved for the case of nudging above 2 km. This result may reflect the importance of the LLJ in transport or perhaps a change in mixing in the models.

Assessing the Performance of Separate Bias Kalman Filter in Correcting the Model Bias for Estimation of Soil Moisture Profiles

The performance of separate bias Kalman filter (SepKF) in correcting the model bias for the improvement of soil moisture profiles is evaluated by assimilating the near-surface soil moisture observations into a land surface model (LSM). First, an observing system simulation experiment (OSSE) is carried out, where the true soil moisture is known, two types of model bias (i.e., constant and sinusoidal) are specified, and the bias error covariance matrix is assumed to be proportional to the model forecast error covariance matrix with a ratio λ. Second, a real assimilation experiment is carried out with measurements at a site over Northwest China. In the OSSE, the soil moisture estimation with the SepKF is improved compared with ensemble Kalman filter (EnKF) without the bias filter, because SepKF can properly correct the model bias, especially in the situation with a large model bias. However, the performance of SepKF becomes slightly worse if the constant model bias increases or temporal variability of the sinusoidal model bias becomes large. It is suggested that the ratio λ should be increased (decreased) in order to improve the soil moisture estimation if temporal variability of the sinusoidal model bias becomes high (low). Finally, the assimilation experiment with real observations also shows that SepKF can further improve the estimation of soil moisture profiles compared with EnKF without the bias correction.

Common dimensioning of frequency restoration reserve capacities for European load-frequency control blocks: An advanced dynamic probabilistic approach

This paper advances the probabilistic modelling approach for dynamic dimensioning of Frequency Restoration Reserve (FRR) capacities of [1]. The approach is based on the convolution of forecast error distributions of generation and load sources. The improvements include 1. the replacement of normal distributions with better fitting intermittency distributions to model forecast error distributions of wind and photovoltaic generation and load, 2. the adjustments for facilitating the dynamic and static dimensioning for 20 European load-frequency control (LFC) blocks, 3. the dimensioning of FRR capacities not only for a historical year (2016) but also for a scenario year 2050 with 100% renewable generation and 4. the implementation of a common dimensioning of several LFC blocks in order to determine the saving potential of an implied integration of European LFC blocks.

A Multi-Model Consensus Forecast Technique for Tropical Cyclone Intensity Based on Model Output Calibration

ABSTRACT We analyzed the errors associated with forecasts of tropical cyclone (TC) intensity from 2010-2012 in the western North Pacific region made by seven operational numerical weather prediction models. The results show that the forecast error is significantly related to the initial error as well as the initial TC intensity, size, and translation speed. Other factors highly related to the forecast error include the environmental sea surface pressure, vertical wind shear and maximum potential intensity. We used stepwise regression to set up model forecast error estimation equations, which were used to calibrate the model output. Independent experiments showed that the calibrated model forecasts have significant skill compared to the original model output. Finally, a multimodel consensus forecast technique for TC intensity was developed based on the calibrated model output; this technique has 28% (15-20%) skill at 12 h (24-72 h) compared to the climatology and persistence forecasts of TC intensity. This consensus technique has greater skill than the consensus forecast based on the original model output and therefore it has the potential to be applied in operation.

Measuring connectedness of euro area sovereign risk

We introduce a method for measuring the default risk connectedness of euro zone sovereign states using credit default swap (CDS) and bond data. The connectedness measure is based on an out-of-sample variance decomposition of model forecast errors. Due to its predictive nature, it can respond to crisis occurrences more quickly than common in-sample techniques. We determine the sovereign default risk connectedness using both CDS and bond data in order to obtain a more comprehensive picture of the system. We find evidence that there are several observable factors that drive the difference between CDS and bonds, but both data sources still contain specific information for connectedness spill-overs. In general, we can identify countries that impose risk on the system and the respective spill-over channels. Our empirical analysis covers the years 2009–2014, such that the recovery paths of countries exiting EU and IMF financial assistance schemes and the responses to the ECB’s unconventional policy measures can be analyzed.

A Predicative Estimation of Supremum of the Model's Forecast Error Resulting from a Conceptual Dataset Shift. on the Example of the Meme-gram-model

A Predicative Estimation of Supremum of the Model's Forecast Error Resulting from a Conceptual Dataset Shift. on the Example of the Meme-gram-model

Improving PM2.5 Air Quality Model Forecasts in China Using a Bias-Correction Framework

Chinese cities are experiencing severe air pollution in particular, with extremely high PM2.5 levels observed in cold seasons. Accurate forecasting of occurrence of such air pollution events in advance can help the community to take action to abate emissions and would ultimately benefit the citizens. To improve the PM2.5 air quality model forecasts in China, we proposed a bias-correction framework that utilized the historic relationship between the model biases and forecasted and observational variables to post-process the current forecasts. The framework consists of four components: (1) a feature selector that chooses the variables that are informative to model forecast bias based on historic data; (2) a classifier trained to efficiently determine the forecast analogs (clusters) based on clustering analysis, such as the distance-based method and the classification tree, etc.; (3) an error estimator, such as the Kalman filter, to predict model forecast errors at monitoring sites based on forecast analogs; and (4) a spatial interpolator to estimate the bias correction over the entire modeling domain. One or more methods were tested for each step. We applied five combinations of these methods to PM2.5 forecasts in 2014–2016 over China from the operational AiMa air quality forecasting system using the Community Multiscale Air Quality (CMAQ) model. All five methods were able to improve forecast performance in terms of normalized mean error (NME) and root mean square error (RMSE), though to a relatively limited degree due to the rapid changing of emission rates in China. Among the five methods, the CART-LM-KF-AN (a Classification And Regression Trees-Linear Model-Kalman Filter-Analog combination) method appears to have the best overall performance for varied lead times. While the details of our study are specific to the forecast system, the bias-correction framework is likely applicable to the other air quality model forecast as well.

Verification of 24-h Quantitative Precipitation Forecasts over the Pacific Northwest from a High-Resolution Ensemble Kalman Filter System

Abstract Environment and Climate Change Canada (ECCC) has recently developed an experimental high-resolution EnKF (HREnKF) regional ensemble prediction system, which it tested over the Pacific Northwest of North America for the first half of February 2011. The HREnKF has 2.5-km horizontal grid spacing and assimilates surface and upper-air observations every hour. To determine the benefits of the HREnKF over less expensive alternatives, its 24-h quantitative precipitation forecasts are compared with those from a lower-resolution (15 km) regional ensemble Kalman filter (REnKF) system and to ensembles directly downscaled from the REnKF using the same grid as the HREnKF but with no additional data assimilation (DS). The forecasts are verified against rain gauge observations and gridded precipitation analyses, the latter of which are characterized by uncertainties of comparable magnitude to the model forecast errors. Nonetheless, both deterministic and probabilistic verification indicates robust improvements in forecast skill owing to the finer grids of the HREnKF and DS. The HREnKF exhibits a further improvement in performance over the DS in the first few forecast hours, suggesting a modest positive impact of data assimilation. However, this improvement is not statistically significant and may be attributable to other factors.

Targeted ocean sampling guidance for tropical cyclones

AbstractA 3‐D variational ocean data assimilation adjoint approach is used to examine the impact of ocean observations on coupled tropical cyclone (TC) model forecast error for three recent hurricanes: Isaac (2012), Hilda (2015), and Matthew (2016). In addition, this methodology is applied to develop an innovative ocean observation targeting tool validated using TC model simulations that assimilate ocean temperature observed by Airborne eXpendable Bathy Thermographs and Air‐Launched Autonomous Micro‐Observer floats. Comparison between the simulated targeted and real observation data assimilation impacts reveals a positive maximum mean linear correlation of 0.53 at 400–500 m, which implies some skill in the targeting application. Targeted ocean observation regions from these three hurricanes, however, show that the largest positive impacts in reducing the TC model forecast errors are sensitive to the initial prestorm ocean conditions such as the location and magnitude of preexisting ocean eddies, storm‐induced ocean cold wake, and model track errors.

Power Consumption Short Term Forecast Using Signal Auto Regressive Method

Short term forecast of power consumption plays a critical role in optimal exploiting of power system. Economic performance and reliability of a network significantly depends to load forecast's accuracy. This forecast is used for load management and planning for unit commitment (UC) and it has especial complexity due to influence of multiple nonlinear relations between daily periodic variations and load consumption changes. 30–60 minutes forecasts are employed extensively in power distribution network. This study aims to perform short term forecasts by using linear auto regressive (AR) modelling. In this paper, collected data from a 63 kV distribution station is modelled by a aforementioned method. It is demonstrated that AR method can model data properly using auto correlation functions residual error and sum of squares criteria. Since data is non-stationary, performance of auto regressive integrated moving average (ARIMA) is investigated and optimal rank of model and the best data length to perform modelling are presented. Regarding to forecasts, appropriate model's rank for AR and ARIMA are 20 and (1, 1, 0) respectively. In AR model forecast error is ±10% which is equal to 38.1 and in ARIMA method forecast mse is ±1.21% and equals to 0.7277 and mse value for neural network method is 0.952 that totally ARIMA method shows better performance than the other.

Hybrid 4DVAR with a Local Ensemble Tangent Linear Model: Application to the Shallow-Water Model

Abstract An ensemble-based tangent linear model (TLM) is described and tested in data assimilation experiments using a global shallow-water model (SWM). A hybrid variational data assimilation system was developed with a 4D variational (4DVAR) solver that could be run either with a conventional TLM or a local ensemble TLM (LETLM) that propagates analysis corrections using only ensemble statistics. An offline ensemble Kalman filter (EnKF) is used to generate and maintain the ensemble. The LETLM uses data within a local influence volume, similar to the local ensemble transform Kalman filter, to linearly propagate the state variables at the central grid point. After tuning the LETLM with offline 6-h forecasts of analysis corrections, cycling experiments were performed that assimilated randomly located SWM height observations, based on a truth run with forced bottom topography. The performance using the LETLM is similar to that of the conventional TLM, suggesting that a well-constructed LETLM could free 4D variational methods from dependence on conventional TLMs. This is a first demonstration of the LETLM application within a context of a hybrid-4DVAR system applied to a complex two-dimensional fluid dynamics problem. Sensitivity tests are included that examine LETLM dependence on several factors including length of cycling window, size of analysis correction, spread of initial ensemble perturbations, ensemble size, and model error. LETLM errors are shown to increase linearly with correction size in the linear regime, while TLM errors increase quadratically. As nonlinearity (or forecast model error) increases, the two schemes asymptote to the same solution.

The Square Root Information Increment Ensemble Filter

Abstract A new type of ensemble filter is developed, one that stores and updates its state information in an efficient square root information filter form. It addresses two shortcomings of conventional ensemble Kalman filters: the coarse characterization of random forecast model error effects and the overly optimistic approximation of the estimation error statistics. The new filter uses an assumed a priori covariance approximation that is full rank but sparse, possibly with a dense low-rank increment. This matrix can be used to develop a nominal square root information equation for the system state and uncertainty. The measurements are used to develop an additional low-rank square root information equation. New algorithms provide forecasts and analyses of these increments at a computational cost comparable to that of existing ensemble Kalman filters. Model error effects are implicit in the a priori covariance time history, thereby obviating one of the reasons for including an inflation operation. The use of an a priori full-rank covariance allows the analysis operations to improve the state estimate without the need for a localization adjustment. This new filter exhibited worse performance than a typical covariance square root ensemble Kalman filter when operating on the Lorenz-96 problem in a chaotic regime. It excelled on a version of the Lorenz-96 problem where nonlinearities in the forecast model were weak, where the state vector uncertainty lay predominantly in a small subspace, and where the observations were spatially sparse. Such a problem might be representative of ionospheric space weather data assimilation where forcing variability can dominate the state uncertainty and where remote sensing data coverage can be sparse.

Ensemble Prediction of Atmospheric Refractivity Conditions for EM Propagation

AbstractAn ensemble forecast system has been developed at the Naval Research Laboratory to improve the analyses and forecasts of atmospheric refractivity for electromagnetic (EM) propagation with the intention of accounting for uncertainties in model forecast errors. Algorithms for a matrix of ensemble statistics have been developed to analyze the probability, location, intensity, and structure of ducting of various types. Major parameters of ducting layers and their ensemble statistics are calculated from the ensemble forecasts. Their relationships to the large-scale and mesoscale environment are also investigated. The Wallops Island field experiment from late April to early May 2000 is selected to evaluate the system. During the spring season, this coastal region maintains a strong sea surface temperature gradient between cold shelf waters and the warm Gulf Stream, where the boundaries between land, the coastal water, and the Gulf Stream have a strong influence on marine boundary layer structures and the formation of ducting layers. Sounding profiles during the field experiment are used in the study to further understand the structures of the ducting layers and also to validate the ensemble forecast system. While some advantages of the ensemble system over the deterministic forecast for atmospheric refractivity prediction in the boundary layer are studied and demonstrated in this study, the weaknesses of the current ensemble system are revealed for future improvement of the system.

Greenhouse gas simulations with a coupled meteorological and transport model: the predictability of CO<sub>2</sub>

Abstract. A new model for greenhouse gas transport has been developed based on Environment and Climate Change Canada's operational weather and environmental prediction models. When provided with realistic posterior fluxes for CO2, the CO2 simulations compare well to NOAA's CarbonTracker fields and to near-surface continuous measurements, columns from the Total Carbon Column Observing Network (TCCON) and NOAA aircraft profiles. This coupled meteorological and tracer transport model is used to study the predictability of CO2. Predictability concerns the quantification of model forecast errors and thus of transport model errors. CO2 predictions are used to compute model–data mismatches when solving flux inversion problems and the quality of such predictions is a major concern. Here, the loss of meteorological predictability due to uncertain meteorological initial conditions is shown to impact CO2 predictability. The predictability of CO2 is shorter than that of the temperature field and increases near the surface and in the lower stratosphere. When broken down into spatial scales, CO2 predictability at the very largest scales is mainly due to surface fluxes but there is also some sensitivity to the land and ocean surface forcing of meteorological fields. The predictability due to the land and ocean surface is most evident in boreal summer when biospheric uptake produces large spatial gradients in the CO2 field. This is a newly identified source of uncertainty in CO2 predictions but it is expected to be much less significant than uncertainties in fluxes. However, it serves as an upper limit for the more important source of transport error and loss of predictability, which is due to uncertain meteorological analyses. By isolating this component of transport error, it is demonstrated that CO2 can only be defined on large spatial scales due to the presence of meteorological uncertainty. Thus, for a given model, there is a spatial scale below which fluxes cannot be inferred simply due to the fact that meteorological analyses are imperfect. These unresolved spatial scales correspond to small scales near the surface but increase with altitude. By isolating other components of transport error, the largest or limiting error can be identified. For example, a model error due to the lack of convective tracer transport was found to impact transport error on the very largest (wavenumbers less than 5) spatial scales. Thus for wavenumbers greater than 5, transport model error due to meteorological analysis uncertainty is more important for our model than the lack of convective tracer transport.

MODEL-BASED VS. PROFESSIONAL FORECASTS: IMPLICATIONS FOR MODELS WITH NOMINAL RIGIDITIES

We compare model forecast error statistics with forecast error statistics of professional forecasts. We look at a standard sticky-prices–wages model, concluding that it delivers too strong a theoretical forecastability of the variables under scrutiny, at odds with the data (professional forecasts). We argue that the lack of compatibility between the model and professional forecasts results from trying to fit inflation (which is probably nonstationary) to a model that assumes inflation is stationary. A modified version of the model, one with a varying inflation target, delivers a better fit in terms of forecastability.

Retrieval of Land Surface Model State Variables through Assimilating Screen Level Humidity and Temperature Measurements

This study demonstrates successful variational retrieval of land surface states by assimilating screen level atmospheric measurements of specific humidity and air temperature. To this end, the land surface scheme is first validated against the Oklahoma Atmospheric Surface Layer Instrumentation System measurements with necessary refinements to the forward model implemented. The retrieval scheme involves a 1D land surface-atmosphere model, the corresponding adjoint codes, and a cost function that measures residuals between observed and modeled screen level atmospheric temperature and specific humidity. The retrieval scheme is robust when subjected to observational errors with magnitudes comparable to instrument accuracy and for initial guess errors larger than typical model forecast errors. Using varying assimilation window lengths centered on different periods of a day, the sampling strategy is assessed. The daytime observations are more informative compared to nocturnal observations. An assimilation window as narrow as four hours, if centered on local noon, contains comparable information to an expanded window covering the whole day. There exists an optimal assimilation window length resulting from the contest between degrading forecast accuracy and increasing information content. For an assimilation window less than two days, the “optimal” assimilation window length is inversely proportional to the data ingesting frequency.

How Analysts and Whisperers Use Fundamental Accounting Signals to Make Quarterly EPS Forecasts

We examine the relative efficiency of whisperers’ and analysts’ forecasts of one-quarter-ahead earnings per share (EPS) and identify commonalities and differences in their use of fundamentals to forecast earnings. Results suggest that (a) fundamentals that focus on sales and cost of sales are relevant in explaining one-quarter-ahead EPS changes; (b) whisperers focus on cash flow fundamentals and accrual-based earnings measures in their one-quarter-ahead forecasts, whereas analysts focus on only cash flow fundamentals; and (c) although neither analysts nor whisperers fully incorporate information contained in fundamentals and accrual-based earnings measures in their forecasts, whisperers’ earnings forecast model (forecast errors model) exhibits higher (lower) explanatory power than that of analysts. We also examine robustness of our results by reestimating the models using a two-way random-effects panel data estimator. Although our conclusions remain the same, more statistically significant fundamentals emerge in panel regression results. Evidence presented in this article is consistent with (a) whisperers being different from analysts and (b) whisper forecasts containing unique incremental information beyond that of analysts’ forecasts. Market participants may want to consider using both forecasts when making investment decisions.

Instrument Choice, Carbon Emissions, and Information

This Article examines the consequences of a previously unrecognized difference between pollutant cap-and-trade schemes and pollution taxes. Implementation of cap-and-trade relies on a forecast of future emissions, while implementation of a pollution tax does not. Realistic policy designs using either regulatory instrument almost always involve a phase-in over time to avoid economic disruption. Cap-and-trade accomplishes this phase-in via a limit on emissions that falls gradually below the forecast of future pollutant emissions. Emissions taxation accomplishes the same via a gradually increasing levy on pollution. Because of the administrative complexity of establishing an emissions trading market, cap-and-trade programs typically require between three and five years lead time before imposing obligations on emitters. In this Article, I present new evidence showing that forecast error over this timeframe for United States energy- related carbon dioxide emissions from the Department of Energy’s energy model—the model used for policy design by Congress and EPA—is biased and imprecise to such a degree as to make its use impractical. The forecasted emissions are insufficiently accurate to allow for creation of a reliable or predictable market signal to incentivize emission reductions. By contrast, carbon taxes, because they do not depend upon a baseline emissions forecast, create a relatively clear level of policy stringency. This difference matters because policies that end up weaker than intended face low odds for strengthening, while those that end up stronger than intended are likely to be weakened. The political asymmetry combined with actual model forecast errors leads to bias in favor of suboptimal, weak, policies for cap-and-trade. This is a serious concern if, as is usually the case, a cap is set based on political bargaining rather than on an optimal balancing of abatement costs and avoided climate damage. By contrast, the same model bias would lead to more environmentally effective than forecast carbon taxes but without the political consequences created by price volatility, were such programs to be implemented in the United States. Thus, while theory tells us that cap-and-trade and carbon taxes can be equivalent, imperfect information leads to suboptimal environmental performance of emissions trading, relative to carbon taxation policies. Policymakers should weigh these practical, information-related concerns when considering approaches to controlling emissions of greenhouse gases.

Representing Forecast Error in a Convection-Permitting Ensemble System

Abstract Ensembles provide an opportunity to greatly improve short-term prediction of local weather hazards, yet generating reliable predictions remain a significant challenge. In particular, convection-permitting ensemble forecast systems (CPEFSs) have persistent problems with underdispersion. Representing initial and or lateral boundary condition uncertainty along with forecast model error provides a foundation for building a more dependable CPEFS, but the best practice for ensemble system design is not well established. Several configurations of CPEFSs are examined where ensemble forecasts are nested within a larger domain, drawing initial conditions from a downscaled, continuously cycled, ensemble data assimilation system that provides state-dependent initial condition uncertainty. The control ensemble forecast, with initial condition uncertainty only, is skillful but underdispersive. To improve the reliability of the ensemble forecasts, the control ensemble is supplemented with 1) perturbed lateral boundary conditions; or, model error representation using either 2) stochastic kinetic energy backscatter or 3) stochastically perturbed parameterization tendencies. Forecasts are evaluated against stage IV accumulated precipitation analyses and radiosonde observations. Perturbed ensemble forecasts are also compared to the control forecast to assess the relative impact from adding forecast perturbations. For precipitation forecasts, all perturbation approaches improve ensemble reliability relative to the control CPEFS. Deterministic ensemble member forecast skill, verified against radiosonde observations, decreases when forecast perturbations are added, while ensemble mean forecasts remain similarly skillful to the control.

A Study of Aerosol Impacts on Clouds and Precipitation Development in a Large Winter Cyclone

Abstract Aerosols influence cloud and precipitation development in complex ways due to myriad feedbacks at a variety of scales from individual clouds through entire storm systems. This paper describes the implementation, testing, and results of a newly modified bulk microphysical parameterization with explicit cloud droplet nucleation and ice activation by aerosols. Idealized tests and a high-resolution, convection-permitting, continental-scale, 72-h simulation with five sensitivity experiments showed that increased aerosol number concentration results in more numerous cloud droplets of overall smaller size and delays precipitation development. Furthermore, the smaller droplet sizes cause the expected increased cloud albedo effect and more subtle longwave radiation effects. Although increased aerosols generally hindered the warm-rain processes, regions of mixed-phase clouds were impacted in slightly unexpected ways with more precipitation falling north of a synoptic-scale warm front. Aerosol impacts to regions of light precipitation, less than approximately 2.5 mm h−1, were far greater than impacts to regions with higher precipitation rates. Comparisons of model forecasts with five different aerosol states versus surface precipitation measurements revealed that even a large-scale storm system with nearly a thousand observing locations did not indicate which experiment produced a more correct final forecast, indicating a need for far longer-duration simulations due to the magnitude of both model forecast error and observational uncertainty. Last, since aerosols affect cloud and precipitation phase and amount, there are resulting implications to a variety of end-user applications such as surface sensible weather and aircraft icing.