Forecast Error Reduction Research Articles

Year-round impact of winter sea ice thickness observations on seasonal forecasts

Abstract. Nowadays many seasonal forecasting centres provide dynamical predictions of sea ice. While initializing sea ice by assimilating sea ice concentration (SIC) is common, constraining initial conditions of sea ice thickness (SIT) is only in its early stages. Here, we make use of the availability of Arctic-wide winter SIT observations covering 2011–2016 to constrain SIT in the ECMWF (European Centre for Medium-Range Weather Forecasts) ocean–sea-ice analysis system with the aim of improving the initial conditions of the coupled forecasts. The impact of the improved initialization on the predictive skill of pan-Arctic sea ice for lead times of up to 7 months is investigated in a low-resolution analogue of the currently operational ECMWF seasonal forecasting system SEAS5. By using winter SIT information merged from CS2 and SMOS (CS2SMOS: CryoSat-2 Soil Moisture and Ocean Salinity), substantial changes in sea ice volume and thickness are found in the ocean–sea-ice analysis, including damping of the overly strong seasonal cycle of sea ice volume. Compared with the reference experiment, which does not use SIT information, forecasts initialized using SIT data show a reduction of the excess sea ice bias and an overall reduction of seasonal sea ice area forecast errors of up to 5 % at lead months 2 to 5. Change in biases is the main forecast impact. Using the integrated ice edge error (IIEE) metric, we find significant improvement of up to 28 % in the September sea ice edge forecast started in April. However, sea ice forecasts for September started in spring still exhibit a positive sea ice bias, which points to a melting that is too slow in the forecast model. A slight degradation in skill is found in the early freezing season sea ice forecasts initialized in July and August, which is related to degraded initial conditions during these months. Both ocean reanalyses, with and without SIT constraint, show strong melting in the middle of the melt season compared to the forecasts. This excessive melting related to positive net surface radiation biases in the atmospheric flux forcing of the ocean reanalyses remains and consequently degrades analysed summer SIC. The impact of thickness initialization is also visible in the sea surface and near-surface temperature forecasts. While positive forecast impact is seen in near-surface temperature forecasts of early freezing season (September–October–November) initialized in May (when the sea ice initial conditions have been observationally constrained in the preceding winter months), negative impact is seen for the same season when initialized in the month of August when the sea ice initial conditions are degraded. We conclude that the strong thinning by CS2SMOS initialization mitigates or enhances seasonally dependent forecast model errors in sea ice and near-surface temperatures in all seasons. The results indicate that the memory of SIT in the spring initial conditions lasts into autumn, influencing forecasts of the peak summer melt and early freezing seasons. Our results demonstrate the usefulness of new sea ice observational products in both data assimilation and forecasting systems, and they strongly suggest that better initialization of SIT is crucial for improving seasonal sea ice forecasts.

Using payments data to nowcast macroeconomic variables during the onset of Covid-19

The spread of COVID-19 has caused large-scale loss of life and economic damage. This pandemic has had a swift effect on the macroeconomy, posing a new and different shock to the Canadian economy. Governments have responded in many ways, including through public health measures, fiscal stimulus and monetary policy. Policy-makers seek to understand the current state of the economy for their COVID-19 support to be effective. However, major economic indicators are released with a substantial delay. This problem is usually dealt with by using a linear model of past economic variables. But this is not the best approach presently, given the large and nonlinear effects of COVID-19 on the economy. In this paper, we develop a model to predict the current state of the economy—known as nowcasting—using retail payments system data and machine learning. The Canadian retail payments data aid in understanding the current state of the economy because they include many types of transactions and are available daily. These data features are ideal for macroeconomic nowcasting during a crisis. The flexibility of machine learning can help capture the large and nonlinear effects of the COVID-19 shock. We find that our model—compared with a benchmark model—has a significant increase in prediction accuracy, measured as a 15 to 45 percent reduction in forecast error.

One-week-ahead electricity price forecasting using weather forecasts, and its application to arbitrage in the forward market: an empirical study of the Japan Electric Power Exchange

Although forecasting one-week-ahead average electricity prices is necessary for decision-making such as evaluating forward contracts, its modeling has not been sufficiently studied. Therefore, to find a suitable forecasting approach, this study constructs and compares multiple parsimonious models using widely published weekly weather forecasts and then applies them to arbitrage trading in the forward market. In particular, we clarify the following empirical results using the data from Japan Electric Power Exchange. First, instead of using forecasted temperature directly as an explanatory variable, the two-step forecasting method using measured temperature as an intermediate variable is more likely to reduce forecast errors. Second, quantile regression has better density forecast accuracy than the generalized autoregressive conditional heteroscedasticity model. Third, the logarithmic conversion for prices tends to improve forecast accuracy. Fourth, one-week-ahead weather forecasts can significantly improve both the price forecast accuracy and the arbitrage profit. The proposed arbitrage strategy can be used by many participants because it can be flexibly changed according to the player’s risk tolerance. In addition, our forecasting/trading method, based on published weather forecasts, has wide applicability in that it can be constructed even in markets where system information is not sufficiently disclosed.

A Brief Assessment of the Impact of Nearly 40 Years of Assimilated Observations Over the Amazon Basin

AbstractAdding a Forecast Sensitivity‐based Observation Impact component to Version 2 of the Modern Era Retrospective‐analysis for Research and Applications, the present study provides an assessment of the impact of nearly 40 years of observations on short‐range (24‐hr) forecasts over the Amazon basin. Under self‐verification, forecast errors are found to slightly increase from the early data‐sparse days to the more recent years, when data dramatically increase. Throughout the reanalysis, satellite radiances dominate in volume, but only before 1999 they dominate the impacts. Beyond 1999, over 50% of forecast error reduction is associated with conventional observations (radiosondes). Atmospheric Motion Vectors are also found to be large contributors to error reduction, but their contribution reduces in dry periods. In opposition to Atmospheric Motion Vectors, satellite radiances tend to contribute more in the dry season. Results provide motivation for additional conventional observations and the use of all‐sky treatment of radiances.

Impact of model upgrades on diabatic processes in extratropical cyclones and downstream forecast evolution

AbstractModels are continuously developed at numerical weather prediction (NWP) centres to improve forecast skill, with new operational model configurations adopted every few years. The parametrizations of diabatic processes are probably the most frequently updated part of NWP models as they are crucial for accurate weather predictions and contain uncertainties in their formulation. The impact of model developments is assessed here in forecasts from the Met Office's weather forecast model initialised throughout the North Atlantic Waveguide and Downstream Impact Experiment field campaign period in autumn 2016. Planned model parametrization developments are considered, together with an ‘inexpensive coupled’ forecast with daily updating of the sea surface temperature and sea‐ice fraction. Forecasts produced from the coupled system have, on average, indistinguishable skill from the control forecasts, suggesting the benefits of coupled atmosphere–ocean NWP systems can be small. In contrast, a reduction in forecast error (∼4%) is identified in forecasts produced using an upgraded convection scheme. Periods of low forecast skill during the study period are shown to be associated with the onset and decay of blocking events and increased diabatic heating of air masses reaching the upper troposphere. In forecasts of a specific block development case that was not accurately predicted in any of the experiments or in the operational ensemble forecast from the Met Office, the representation of diabatic heating in the warm conveyor belt of an upstream cyclone is shown to moderate the subsequent block development; forecasts in which the heating is stronger generally have a more‐amplified blocking ridge and amplified heating contributions from all parametrizations as diagnosed using diabatic tracers. Hence, we demonstrate that plausible changes to the representation of several different diabatic processes in models can impact forecast block development via changes within upstream cyclones.

Atmospheric River Reconnaissance Observation Impact in the Navy Global Forecast System

Abstract Atmospheric rivers, often associated with impactful weather along the west coast of North America, can be a challenge to forecast even on short time scales. This is attributed, at least in part, to the scarcity of eastern Pacific in situ observations. We examine the impact of assimilating dropsonde observations collected during the Atmospheric River (AR) Reconnaissance 2018 field program on the Navy Global Environmental Model (NAVGEM) analyses and forecasts. We compare NAVGEM’s representation of the ARs to the observations, and examine whether the observation–background difference statistics are similar to the observation error variance specified in the data assimilation system. Forecast sensitivity observation impact is determined for each dropsonde variable, and compared to the impacts of the North American radiosonde network. We find that the reconnaissance soundings have significant beneficial impact, with per observation impact more than double that of the North American radiosonde network. Temperature and wind observations have larger total and per observation impact than moisture observations. In our experiment, the 24-h global forecast error reduction from the reconnaissance soundings can be comparable to the reduction from the North American radiosonde network for the field program dates that include at least two flights.

A Novel Fitted Rolling Horizon Control Approach for Real-Time Policy Making in Microgrid

In recent years, rolling horizon control (RHC) approaches have attracted growing attention due to its feature of reducing forecast errors for real-time/online operation and optimization. However, the performance of the existing RHC approach degrades if the intra-day forecast data is unavailable or missing due to Internet or cloud service provider outages, software/hardware failures, and many other factors. In this paper, we propose a new fitted-RHC approach to overcome this challenge. The proposed fitted-RHC framework is designed with a regression algorithm which utilizes the empirical knowledge to make the real-time decisions whenever the intra-day forecast data is unavailable. The regression algorithm utilizes a statistical relative probability method to calculate the relative probability for each decision vector, and output the proper optimization policy. In addition, we adopt a modified version of exogenous information transition function that is more suitable to conduct the simulations in a real-time environment. Simulation results in microgird show that the proposed fitted-RHC approach can achieve the optimal policy for the deterministic case even with the missing data, and perform efficiently with the uncertain environment in stochastic case study. In comparison, the proposed fitted-RHC approach outperforms several other optimization techniques.

Automatic Time Series Modelling and Forecasting: A Replication Case Study of Forecasting Real GDP, the Unemployment Rate, and the Impact of Leading Economic Indicators

We test and report on time series modelling and forecasting using several U.S. Leading Economic Indicators (LEI) as an input to forecasting real U.S. GDP and the unemployment rate. These time series have been addressed before, but our results are more statistically significant using more recently developed time series modelling techniques and software. Montgomery, Zarnowitz, Tsay, and Tiao (1998) modeled the U.S. unemployment rate as a function of the weekly unemployment claims time series, 1948 – 1992. In this replication case study, we apply the Hendry and Doornik automatic time series PC-Give (AutoMetrics) methodology to the well-studied macroeconomics series, U.S. real GDP and the unemployment rate. The Autometrics system substantially reduces regression sum of squares measures relative to traditional variations on the random walk with drift model. The LEI are a statistically significant input to real GDP. A similar conclusion is found for the impact of the LEI and weekly unemployment claims series leading the unemployment rate series. We tested the forecasting ability of best univariate and best bivariate models over 60- and 120-period rolling windows and report considerable forecast error reductions. The adaptive averaging autoregressive model forecast ADA-AR and the adaptive learning forecast, ADL, produced the smallest root mean square errors and lowest mean absolute errors.

Forecasting dengue and influenza incidences using a sparse representation of Google trends, electronic health records, and time series data.

Dengue and influenza-like illness (ILI) are two of the leading causes of viral infection in the world and it is estimated that more than half the world’s population is at risk for developing these infections. It is therefore important to develop accurate methods for forecasting dengue and ILI incidences. Since data from multiple sources (such as dengue and ILI case counts, electronic health records and frequency of multiple internet search terms from Google Trends) can improve forecasts, standard time series analysis methods are inadequate to estimate all the parameter values from the limited amount of data available if we use multiple sources. In this paper, we use a computationally efficient implementation of the known variable selection method that we call the Autoregressive Likelihood Ratio (ARLR) method. This method combines sparse representation of time series data, electronic health records data (for ILI) and Google Trends data to forecast dengue and ILI incidences. This sparse representation method uses an algorithm that maximizes an appropriate likelihood ratio at every step. Using numerical experiments, we demonstrate that our method recovers the underlying sparse model much more accurately than the lasso method. We apply our method to dengue case count data from five countries/states: Brazil, Mexico, Singapore, Taiwan, and Thailand and to ILI case count data from the United States. Numerical experiments show that our method outperforms existing time series forecasting methods in forecasting the dengue and ILI case counts. In particular, our method gives a 18 percent forecast error reduction over a leading method that also uses data from multiple sources. It also performs better than other methods in predicting the peak value of the case count and the peak time.

The tradeoff between relevance and comparability in segment reporting

The rule change for segment reporting in 1998 has arguably made segment reporting more relevant through the adoption of the management approach. Meanwhile, the management approach has resulted in a decrease in the comparability of segment income. We introduce firm-specific measures of changes in relevance and comparability due to the rule change. Our treatment firms experienced an increase in the relevance of segment reporting but a large decrease in the comparability of segment income; our benchmark firms barely experienced any changes in relevance and comparability. We examine earnings forecasts before vs. after the rule change issued by financial analysts—a major user group of segment reporting. Relative to benchmark firms, treatment firms’ analyst forecast error reductions around the segment disclosure event are not significantly different after the rule change than before the rule change, but treatment firms’ forecast dispersion reductions around the segment disclosure event are significantly larger after the rule change than before the rule change. These results suggest that despite the decrease in comparability, the new segment reporting rule has increased the decision usefulness of segment information by decreasing disagreement among analysts.

Efectos de la comunicación sobre los errores de pronóstico de inflación: evidencia para Colombia para el período 2008-2016

El objetivo de este trabajo es examinar los efectos de la comunicación del banco central sobre los errores de pronóstico de inflación para Colombia en el período 2008 – 2016. La metodología empleada consiste en un análisis econométrico compuesto por: i) La estimación de un modelo EGARCH; ii) El uso de Vector Auto regresivos (VAR); y iii) Un análisis de descomposición de varianza. Los resultados del trabajo muestran que los anuncios de política monetaria generan efectos importantes sobre los errores de pronóstico. La principal recomendación de política es que la comunicación es una herramienta que tiene el banco central para reducir los errores de pronóstico. En particular, la comunicación tiene el potencial para lograr una convergencia entre las expectativas de inflación y los objetivos perseguidos por el banco central. La principal limitación está relacionada con la falta de información más desagregada sobre las expectativas de inflación que no permiten analizar su reacción ante diferentes noticias macroeconómicas. La originalidad del trabajo consiste en analizar los errores de pronóstico de inflación con base en la inflación compensada de los títulos de deuda pública.

Forecast Sensitivity Observation Impact in the 4DVAR and Hybrid-4DVAR Data Assimilation Systems

AbstractIn this study, the observation impacts on 24-h forecast error reduction (FER), based on the adjoint method in the four-dimensional variational (4DVAR) data assimilation (DA) and hybrid-4DVAR DA systems coupled with the Unified Model, were evaluated from 0000 UTC 5 August to 1800 UTC 26 August 2014. The nonlinear FER in hybrid-4DVAR was 12.2% greater than that in 4DVAR due to the use of flow-dependent background error covariance (BEC), which was a weighted combination of the static BEC and the ensemble BEC based on ensemble forecasts. In hybrid-4DVAR, the observation impacts (i.e., the approximated nonlinear FER) for most observation types increase compared to those in 4DVAR. The increased observation impact from using hybrid-4DVAR instead of 4DVAR changes depending on the analysis time and regions. To calculate the ensemble BEC in hybrid-4DVAR, analyses at 0600 and 1800 UTC (0000 and 1200 UTC) used 3-h (9-h) ensemble forecasts. Greater observation impact was obtained when 3-h ensemble forecasts were used for the ensemble BEC at 0600 and 1800 UTC, than with 9-h ensemble forecasts at 0000 and 1200 UTC. Different from other observations, the atmospheric motion vectors (AMVs) deduced from geostationary satellite are more frequently observed in the same area. When the ensemble forecasts with longer integration times were used for the ensemble BEC in hybrid-4DVAR, the observation impact of the AMVs decreased the most in East Asia. This implies that the observation impact of AMVs in East Asia shows the highest sensitivity to the integration time of the ensemble members used for deducing the flow-dependent BEC in hybrid-4DVAR.

A dynamic Nelson–Siegel model with forward-looking macroeconomic factors for the yield curve in the US

This paper employs a factor-augmented dynamic Nelson–Siegel (FADNS) model to predict the yield curve in the US that relies on a large data set of mostly forward-looking macroeconomic variables. FADNS models significantly improve interest rate forecasts relative to many extant models in the literature. For longer horizons, it outperforms autoregressive alternatives, with reductions in mean absolute forecast error of up to 18% using quarterly data and of up to 40% at higher frequencies. For shorter horizons, it is still competitive against autoregressive forecasts, outclassing them for 7- and 10-year yields. The out-of-sample analysis reveals that the forward-looking nature of the indicators we employ is crucial for improving forecasting performance. Including them indeed reduces the mean absolute error with respect to specifications based on backward-looking macroeconomic indicators for any model we consider.

A Kalman filter-based bottom-up approach for household short-term load forecast

Renewable energy sources are now being used with buildings like PV panels. Consequently, short-term household load forecast plays an important role in managing distributed energy generation, local consumption, and grid-building integration. Forecasting household load, however, can be an intractable problem. These loads are characterized by large uncertainty and variations, leaving much room to improve accuracy. To improve the household load forecast accuracy, this paper advocates a Kalman filter-based bottom-up approach. First, using a deep learning model and a persistence model on public datasets, the authors verified the advantage of the bottom-up approach through granularity analysis at the appliance, room, house levels. Employing the Symmetric Mean Absolute Percentage Error, the authors compared two strategies: (1) the conventional strategy, which forecasts the load directly at the household level, and (2) the bottom-up strategy, which aggregates the forecasts made at the room or appliance level. Experimental results on public datasets demonstrated that the bottom-up approach holds great promise. Second, as the bottom-up approach is often criticized for the cost, the authors designed a recontextualized Kalman filter model to efficiently forecast appliance energy usages. Using two strategies, the authors compared the Kalman filter-based bottom-up approach with deep-learning models. They found the bottom-up approach reduced forecast errors 49% more than the deep-learning models and 47% more than the conventional strategy. Finally, the authors concluded that a Kalman filter-based bottom-up approach could efficiently improve household load forecast accuracy. The findings could help give fast and accurate load forecasts for building energy management and predictive controls.

The Influence of Tropical Forecast Errors on Higher Latitude Predictions

AbstractThe atmospheric response to variations in tropical latent heating extends well beyond its source region, and therefore it is thought that a reduction of tropical forecast errors should also benefit subsequent forecasts over the extratropics. This relationship is evaluated using a conditional skill analysis applied to subseasonal reforecasts from the National Centers for Environmental Prediction Coupled Forecast System and the European Centre for Medium‐Range Weather Forecasts Integrated Forecast System. It is shown that there is enhanced or attenuated skill in Northern Hemisphere Weeks 2–4 forecasts when tropical short range precipitation forecasts are “good” or “poor,” respectively. This conditional skill is modulated by both El Ni o Southern Oscillation and the Madden and Julian Oscillation, particularly in the Integrated Forecast System. The results presented here indicate that midlatitude Weeks 2–4 predictive skill would benefit from improvements in Week 1 tropical performance, particularly for the National Centers for Environmental Prediction system.

Effect of introducing weather parameters on the accuracy of milk production forecast models

The objective of this study was to analyze the effect of adding meteorological data to the training process of two milk production forecast models. The two models chosen were the nonlinear auto-regressive model with exogenous input (NARX) and the multiple linear regression (MLR) model. The accuracy of these models were assessed using seven different combinations of precipitation, sunshine hours and soil temperature as additional model training inputs. Lactation data (daily milk yield and days in milk) from 39 pasture-based Holstein-Friesian Irish dairy cows were selected to compare to the model outputs from a central database. The models were trained using historical milk production data from three lactation cycles and were employed to predict the total daily milk yield of a fourth lactation cycle for each individual cow over short (10-day), medium (30-day) and long-term (305-day) forecast horizons. The NARX model was found to provide a greater prediction accuracy when compared to the MLR model when predicting annual individual cow milk yield (kg), with R2 values greater than 0.7 for 95.5% and 14.7% of total predictions, respectively. The results showed that the introduction of sunshine hours, precipitation and soil temperature data improved the prediction accuracy of individual cow milk prediction for the NARX model in the short, medium and long-term forecast horizons. Sunshine hours was shown to have the largest impact on milk production with an improvement of forecast accuracy observed in 60% and 70% of all predictions (for all 39 test cows from both groups). However, the overall improvement in accuracy was small with a maximum forecast error reduction of 4.3%. Thus, the utilization of meteorological parameters in milk production forecasting did not have a substantial impact on forecast accuracy.

Multilayer perceptron for short-term load forecasting: from global to local approach

Many forecasting models are built on neural networks. The key issues in these models, which strongly translate into the accuracy of forecasts, are data representation and the decomposition of the forecasting problem. In this work, we consider both of these problems using short-term electricity load demand forecasting as an example. A load time series expresses both the trend and multiple seasonal cycles. To deal with multi-seasonality, we consider four methods of the problem decomposition. Depending on the decomposition degree, the problem is split into local subproblems which are modeled using neural networks. We move from the global model, which is competent for all forecasting tasks, through the local models competent for the subproblems, to the models built individually for each forecasting task. Additionally, we consider different ways of the input data encoding and analyze the impact of the data representation on the results. The forecasting models are examined on the real power system data from four European countries. Results indicate that the local approaches can significantly improve the accuracy of load forecasting, compared to the global approach. A greater degree of decomposition leads to the greater reduction in forecast errors.

Accuracy gains from conservative forecasting: Tests using variations of 19 econometric models to predict 154 elections in 10 countries.

ProblemDo conservative econometric models that comply with the Golden Rule of Forecasting provide more accurate forecasts?MethodsTo test the effects of forecast accuracy, we applied three evidence-based guidelines to 19 published regression models used for forecasting 154 elections in Australia, Canada, Italy, Japan, Netherlands, Portugal, Spain, Turkey, U.K., and the U.S. The guidelines direct forecasters using causal models to be conservative to account for uncertainty by (I) modifying effect estimates to reflect uncertainty either by damping coefficients towards no effect or equalizing coefficients, (II) combining forecasts from diverse models, and (III) incorporating more knowledge by including more variables with known important effects.FindingsModifying the econometric models to make them more conservative reduced forecast errors compared to forecasts from the original models: (I) Damping coefficients by 10% reduced error by 2% on average, although further damping generally harmed accuracy; modifying coefficients by equalizing coefficients consistently reduced errors with average error reductions between 2% and 8% depending on the level of equalizing. Averaging the original regression model forecast with an equal-weights model forecast reduced error by 7%. (II) Combining forecasts from two Australian models and from eight U.S. models reduced error by 14% and 36%, respectively. (III) Using more knowledge by including all six unique variables from the Australian models and all 24 unique variables from the U.S. models in equal-weight “knowledge models” reduced error by 10% and 43%, respectively.OriginalityThis paper provides the first test of applying guidelines for conservative forecasting to established election forecasting models.UsefulnessElection forecasters can substantially improve the accuracy of forecasts from econometric models by following simple guidelines for conservative forecasting. Decision-makers can make better decisions when they are provided with models that are more realistic and forecasts that are more accurate.

The Tradeoff between Relevance and Comparability in Segment Reporting

The rule change for segment reporting in 1998 has arguably made segment reporting more relevant through the adoption of the management approach. Meanwhile, the management approach has resulted in a decrease in the comparability of segment income. We introduce firm-specific measures of changes in relevance and comparability due to the rule change. Our treatment firms experienced an increase in the relevance of segment reporting but a large decrease in the comparability of segment income; our benchmark firms barely experienced any changes in relevance and comparability. We examine earnings forecasts before vs. after the rule change issued by financial analysts—a major user group of segment reporting. Relative to benchmark firms, treatment firms’ analyst forecast error reductions around the segment disclosure event are not significantly different after the rule change than before the rule change, but treatment firms’ forecast dispersion reductions around the segment disclosure event are significantly larger after the rule change than before the rule change. These results suggest that despite the decrease in comparability, the new segment reporting rule has increased the decision usefulness of segment information by decreasing disagreement among analysts.

Multiscale Global Atmosphere Model SL-AV: the Results of Medium-range Weather Forecasts

Development of the multiscale version of the global atmosphere model SL-AV required many improvements in the dynamical core, replacement or refinement of parameterization algorithms and complex tuning of the model. These modifications were initially tested with the experiments on modern climate simulation and then incorporated into the model configuration for medium-range numerical weather prediction. The impact of these model improvements on forecast quality is studied in this paper. The increase in accuracy of model climate characteristics has led to the reduction of forecast errors. The comparison of quality for numerical forecasts starting from the initial data of Hydrometcenter of Russia and ECMWF is carried out. The effect of replacing the initial data turned out to be comparable to the effect of multi-year works on model development. This shows the importance and necessity of development and improvement of the Hydrometcenter of Russia data assimilation system.