Forecast Bias Research Articles

Climate Services Toolbox (CSTools) v4.0: from climate forecasts to climate forecast information

Abstract. Despite the wealth of existing climate forecast data, only a small part is effectively exploited for sectoral applications. A major cause of this is the lack of integrated tools that allow the translation of data into useful and skillful climate information. This barrier is addressed through the development of an R package. Climate Services Toolbox (CSTools) is an easy-to-use toolbox designed and built to assess and improve the quality of climate forecasts for seasonal to multi-annual scales. The package contains process-based, state-of-the-art methods for forecast calibration, bias correction, statistical and stochastic downscaling, optimal forecast combination, and multivariate verification, as well as basic and advanced tools to obtain tailored products. Due to the modular design of the toolbox in individual functions, the users can develop their own post-processing chain of functions, as shown in the use cases presented in this paper, including the analysis of an extreme wind speed event, the generation of seasonal forecasts of snow depth based on the SNOWPACK model, and the post-processing of temperature and precipitation data to be used as input in impact models.

Development of New Observation Operators for Assimilating GOES-R Geostationary Lightning Mapper Flash Extent Density Data Using GSI EnKF: Tests with Two Convective Events over the United States

Abstract In a prior study, GOES-R Geostationary Lightning Mapper (GLM) flash extent density (FED) data were assimilated using ensemble Kalman filter into a convection-allowing model for a mesoscale convective system (MCS) and a supercell storm. The FED observation operator based on a linear relation with column graupel mass was tuned by multiplying a factor to avoid large FED forecast bias. In this study, new observation operators are developed by fitting a third-order polynomial to GLM FED observations and the corresponding FED forecasts of graupel mass of the MCS and/or supercell cases. The new operators are used to assimilate the FED data for both cases, in three sets of experiments called MCSFit, SupercellFit, and CombinedFit, and their performances are compared with the prior results using the linear operator and with a reference simulation assimilating no FED data. The new nonlinear operators reduce the frequency biases (root-mean-square innovations) in the 0–4-h forecasts of the FED (radar reflectivity) relative to the results using the linear operator for both storm cases. The operator obtained by fitting data from the same case performs slightly better than fitting to data from the other case, while the operator obtained by fitting forecasts of both cases produce intermediate but still very similar results, and the latter is considered more general. In practice, a more general operator can be developed by fitting data from more cases. Significance Statement Prior studies found that assimilation of satellite lightning observation can benefit storm forecasts for up to 4 h. A linear lightning observation operator originally developed for assimilating pseudo-satellite lightning observations was tuned earlier through sensitivity experiments when assimilating real lightning data. However, the linear relation does not fit the model and observational data well and significant bias can exist. This study develops new lightning observation operators by fitting a high-order polynomial to satellite lightning observations and model-predicted quantities that directly relate to lightning. The new operator was found to reduce the frequency biases and root-mean-square innovations for lightning and radar reflectivity forecasts, respectively, up to several hours relative to the linear operator. The methodology can be applied to larger data samples to obtain a more general operator for use in operational data assimilation systems.

Forecasts of the Warm-Sector Heavy Rainfall With a Warm Shear Pattern Over Coastal Areas of the Yangtze–Huaihe River in a Regional Business Forecast Model

Based on the observational hourly precipitation data and the ERA5 reanalysis datasets, the short-term forecasts of the warm-sector heavy rainfall with warm-type shear line (WRWS) events over the coastal areas of the Yangtze–Huaihe River (YHR) are investigated in the regional business model Precision Weather Analysis and Forecasting System (PWAFS). Evaluations and diagnoses are carried out via objective estimations and composition analyses for the rainy season of 2017. Results show that the short-term forecasts of PWAFS are characterized by considerable skills for WRWS events in the coastal areas of YHR in view of the object-based diagnostic evaluation, which, however, tend to generate the rain belts with northeast shift phases and weaker intensities. Meantime, the threat score results for WRWS-associated processes show that the model forecasting skill declines sharply as the precipitation intensity increases. Moreover, composition differences of the synoptic-scale thermodynamic characteristics between observations and forecast results are diagnosed to reveal the possible mechanisms of the short-term forecast biases toward WRWS. The zonal westerlies are overestimated in the model, while the southerlies are underestimated in the lower troposphere over coastal areas of YHR, leading to the northeastward shifted shear line and the absent moisture channel associated with the East China Sea at the boundary layer. Attributed to these atmospheric circulation biases, the accumulated warm and moist energy is weaker at the boundary layer, and hence, the short-term forecasts of the rain-belt location for WRWS over the YHR coastal areas have northeast shifting phases with weaker intensities of precipitation in forecasts of the regional business model PWAFS.

Design and Analytic Features for Reducing Biases in Skill-Building Intervention Impact Forecasts

Despite policy relevance, longer-term evaluations of educational interventions are relatively rare. A common approach to this problem has been to rely on longitudinal research to determine targets for intervention by looking at the correlation between children’s early skills (e.g., preschool numeracy) and medium-term outcomes (e.g., first-grade math achievement). However, this approach has sometimes over—or under—predicted the long-term effects (e.g., 5th-grade math achievement) of successfully improving early math skills. Using a within-study comparison design, we assess various approaches to forecasting medium-term impacts of early math skill-building interventions. The most accurate forecasts were obtained when including comprehensive baseline controls and using a combination of conceptually proximal and distal short-term outcomes (in the nonexperimental longitudinal data). Researchers can use our approach to establish a set of designs and analyses to predict the impacts of their interventions up to 2 years post-treatment. The approach can also be applied to power analyses, model checking, and theory revisions to understand mechanisms contributing to medium-term outcomes.

Surprisingness and Occupational Engagement Influence Affective Forecasting in Career-Relevant Contexts.

People tend to misestimate their future emotions. This phenomenon is thought to be associated with information accessibility. However, few studies have demonstrated the impact of context-specific information accessibility on affective forecasting. This research investigated the effects of information accessibility on affective forecasting in career context (i.e., occupational engagement was seen as information accessibility), during which surprise or not surprise context was played simultaneously. We found that affective forecasting appeared stably across emotional response types. Specifically, there was an underestimation in interest appraisals and an overestimation in satisfaction. These biases were influenced by occupational engagement, which only worked in career interest appraisals. High occupational engagement made people estimate their future emotions more accurately and overcome their forecasting bias. Surprisingness was then manipulated further to explain whether it could impact the effect of occupational engagement on affective forecasting. The emotional responses in both prediction and experience were affected by surprisingness, thus causing no affective forecasting biases. These results suggest the role of occupational engagement in affective forecasting and provide evidence supporting the information accessibility model about the mechanism in affective forecasting.

Firm life cycle, expectation errors and future stock returns

I study the return predictability of firm life cycle, originally documented by Dickinson (2011). I show that a hedge portfolio strategy going long on mature firms and short on introduction firms generates a significant hedge portfolio return of 1.29% per month in return-weighted portfolios and 0.72% in value-weighted portfolios. The returns to firm life cycle are related to investors’ and analysts’ expectation errors, are driven by market-wide investor sentiment, and are more pronounced among stocks with low institutional ownership and high idiosyncratic volatility. Quantile regressions show that introduction firms have considerably greater uncertainty and skewness in future earnings growth outcomes than mature firms, such that analysts are better able to justify optimistically biased forecasts for introduction firms compared to mature firms.

Snow Representation over Siberia in Operational Seasonal Forecasting Systems

Seasonal forecasting systems still have difficulties predicting temperature over continental regions, while their performance is better over some maritime regions. On the other hand, the land surface is a substantial source of (sub-)seasonal predictability. A crucial land surface component in focus here is the snow cover, which stores water and modulates the surface radiation balance. This paper’s goal is to attribute snow cover seasonal forecasting biases and lack of skill to either initialization or parameterization errors. For this purpose, we compare the snow representation in five seasonal forecasting systems (from DWD, ECMWF, Météo-France, CMCC, and ECCC) and their performances in predicting snow and 2-m temperature over a Siberian region against ERA5 reanalysis and station data. Although all systems use similar atmospheric and land initialization approaches and data, their snow and temperature biases differ in sign and amplitude. Too-large initial snow biases persist over the forecast period, delaying and prolonging the melting phase. The simplest snow scheme (used in DWD’s system) shows too-early and fast melting in spring. However, systems including multi-layer snow schemes (Météo-France and CMCC) do not necessarily perform better. Both initialization and parameterization are causes of snow biases, but, depending on the system, one can be more dominant.

Multi model forecast biases of the diurnal variations of intense rainfall in the Beijing-Tianjin-Hebei region

Multi model forecast biases of the diurnal variations of intense rainfall in the Beijing-Tianjin-Hebei region

LEARNING ABOUT REGIME CHANGE

AbstractTotal factor productivity (TFP) and investment specific technology (IST) growth both exhibit regime‐switching behavior, but the regime at any given time is difficult to infer. We build a rational expectations real business cycle model where the underlying TFP and IST regimes are unobserved. We develop a general perturbation solution algorithm for a wide class of models with unobserved regime‐switching. Using our method, we show learning about regime‐switching fits the data, affect the responses to regime shifts and intraregime shocks, increase asymmetries in the responses, generate forecast error bias even with rational agents, and raise the welfare cost of fluctuations.

The Equatorial Pacific Cold Tongue Bias in CESM1 and Its Influence on ENSO Forecasts

Abstract The mean-state bias and the associated forecast errors of the El Niño–Southern Oscillation (ENSO) are investigated in a suite of 2-yr-lead retrospective forecasts conducted with the Community Earth System Model, version 1, for 1954–2015. The equatorial Pacific cold tongue in the forecasts is too strong and extends excessively westward due to a combination of the model’s inherent climatological bias, initialization imbalance, and errors in initial ocean data. The forecasts show a stronger cold tongue bias in the first year than that inherent to the model due to the imbalance between initial subsurface oceanic states and model dynamics. The cold tongue bias affects not only the pattern and amplitude but also the duration of ENSO in the forecasts by altering ocean–atmosphere feedbacks. The predicted sea surface temperature anomalies related to ENSO extend to the far western equatorial Pacific during boreal summer when the cold tongue bias is strong, and the predicted ENSO anomalies are too weak in the central-eastern equatorial Pacific. The forecast errors of pattern and amplitude subsequently lead to errors in ENSO phase transition by affecting the amplitude of the negative thermocline feedback in the equatorial Pacific and tropical interbasin adjustments during the mature phase of ENSO. These ENSO forecast errors further degrade the predictions of wintertime atmospheric teleconnections, land surface air temperature, and rainfall anomalies over the Northern Hemisphere. These mean-state and ENSO forecast biases are more pronounced in forecasts initialized in boreal spring–summer than other seasons due to the seasonal intensification of the Bjerknes feedback.

Forecasting at capacity: the bias of unconstrained forecasts in model evaluation

Forecasting at capacity: the bias of unconstrained forecasts in model evaluation

The Accuracy and Informativeness of Management Earnings Forecasts: A Review and Unifying Framework*

ABSTRACTThis paper synthesizes the literature on management earnings forecasts (MFs) and adaption mechanisms, combines existing theories into a unifying framework, and discusses the primary determinants of MF accuracy and informativeness. The proposed model refines existing theories by emphasizing the dynamics and multiperiod interactions among firm management, financial analysts, and investors, thereby simplifying the assessment of the complex relations within the forecast cycle. Furthermore, we analyze when and to what extent financial analysts and investors anticipate bias and misleading information. Overall, the literature review provides strong support for a positive correlation between the extent and credibility of MFs, on the one hand, and stock returns, share liquidity, and analyst coverage, on the other hand. Earnings forecasts tend to be optimistically biased, with a positive correlation with forecast uncertainty, earnings flexibility, financial distress, investor sentiment, and the share price dependency of managers' remuneration. Firm growth, legal liability, and litigation risk are significantly associated with forecast pessimism. We also find that MF accuracy increases with previous forecast accuracy, firm size, analyst coverage, analyst agreement, management qualifications, and corporate governance level. Moreover, investors do not anticipate the full extent of predictable forecast bias, leading to systematic share price drifts after the announcement of earnings forecasts and actual earnings. The study's results have substantial implications for researchers, firm managers, investors, financial analysts, and regulators. Although managers may enhance their forecasts' credibility by providing precise, bundled, and disaggregated forecasts, external stakeholders should carefully analyze forecast antecedents and characteristics to assess the direction and magnitude of expected MF bias.

A seven-parameter Bernoulli-Gamma-Gaussian model to calibrate subseasonal to seasonal precipitation forecasts

Subseasonal to seasonal (S2S) precipitation forecasts can fill the gap between weather and seasonal forecasts. While raw forecasts from S2S models are informative, calibration is necessary to correct systematic biases and quantify forecast uncertainty to facilitate applications of S2S forecasts. This paper develops a seven-parameter Bernoulli-Gamma-Gaussian model to calibrate S2S precipitation forecasts. The Bernoulli distribution characterizes the occurrence of zero versus non-zero values of precipitation, the Gamma distribution accounts for the distribution of non-zero precipitation amounts and the bivariate Gaussian distribution formulates the relationship between raw forecasts and observations. A case study of the East River catchment in South China is devised for the S2S forecasts provided by the European Centre for Medium-Range Weather Forecasts (ECWMF). The Bernoulli-Gamma-Gaussian model is used to calibrate raw forecasts at the daily time step and the Schaake Shuffle is used to instill the temporal patterns to form ensemble time series forecasts. The results show that raw S2S forecasts generally suffer from considerable forecast bias and unreliable ensemble spread and can exhibit negative forecast skill. Compared to the quantile mapping, the Bernoulli-Gamma-Gaussian model with the Schaake Shuffle is effective in correcting biases, improving reliability and yielding forecast skill for daily and accumulated precipitation. Furthermore, a sliding window of 15 days tends to be suitable to pool samples to deal with sampling variability and precipitation seasonality for the purpose of calibration of S2S forecasts. Overall, the Bernoulli-Gamma-Gaussian model can serve as an effective tool to calibrate S2S precipitation forecasts for hydrological modeling and water resources management.

Deep neural networks for spatiotemporal PM2.5 forecasts based on atmospheric chemical transport model output and monitoring data

Reliable long-horizon PM2.5 forecasts are crucial and beneficial for health protection through early warning against air pollution. However, the dynamic nature of air quality makes PM2.5 forecasts at long horizons very challenging. This study proposed a novel machine learning-based model (MCNN-BP) that fused multiple convolutional neural networks (MCNN) with a back-propagation neural network (BPNN) for making spatiotemporal PM2.5 forecasts for the next 72 h at 74 stations covering the whole Taiwan simultaneously. Model configuration involved an ensemble of massive hourly air quality and meteorological monitoring datasets and the existing publicly-available PM2.5 simulated (forecasted) datasets from an atmospheric chemical transport (ACT) model. The proposed methodology collaboratively constructed two CNNs to mine the observed data (the past) and the forecasted data from ACT (the future) separately. The results showed that the MCNN-BP model could significantly improve the accuracy of spatiotemporal PM2.5 forecasts and substantially reduce the forecast biases of the ACT model. We demonstrated that the proposed MCNN-BP model with effective feature extraction and good denoising ability could overcome the curse of dimensionality and offer satisfactory regional long-horizon PM2.5 forecasts. Moreover, the MCNN-BP model has considerably shorter computational time (5 min) and lower computational load than the compute-intensive ACT model. The proposed approach hits a milestone in multi-site and multi-horizon forecasting, which significantly contributes to early warning against regional air pollution.

The association between current earnings surprises and the ex post bias of concurrently issued management forecasts

The vast majority of managers’ earnings forecasts are issued concurrently (i.e., bundled) with their firm’s current earnings announcement. We document a predictable bias in these forecasts—the forecasts fail to fully reflect the persistence of the current earnings surprise. Specifically, we find that managers issue (1) optimistically biased forecasts alongside negative earnings surprises and (2) pessimistically biased forecasts alongside large positive earnings surprises. Bayesian updating implies this bias could be unintentional, but we find that the bias is stronger when managers have greater incentives and fewer constraints to issue biased forecasts, suggesting that, to some extent, the bias might be intentional. Relatedly, although managers typically have better information about their firm’s earnings than analysts, we show that analyst reliance on these biased management forecasts represents a mechanism (and an alternative interpretation) for a similar analyst underreaction to current earnings attributed in the literature to analysts’ cognitive bias. We also find that, on average, investors do not appear to initially understand the bias in these forecasts but do unravel it over longer windows. However, investors more quickly unravel the bias when the manager has a history of issuing biased forecasts and when the firm has more sophisticated investors. Overall, we document that managers’ forecasts appear to repeatedly underweight the persistence of current earnings surprises, are biased in ways that improve investors’ perceptions of managers’ ability, and that this behavior concentrates in subsamples where outsiders have a harder time recognizing any bias.

Quantile mapping correction of analog ensemble forecast for solar irradiance

Probabilistic forecasts of solar irradiance are a key technology for integrating solar power into power grids. However, ensemble forecasts, the conventional probabilistic forecasting method suffers from bias inherent in numerical weather prediction. The recently proposed Analog Ensemble (AnEn) method is expected to have less bias. We therefore construct an experimental AnEn system for day-ahead solar irradiance forecasts and investigate its performance. The AnEn method finds analogs by a nearest-neighbor search, so its members can be ranked by the order found in the search. We find that the probability distribution function (PDF) for predicted irradiance deduced from members with large orders are heavily deformed from the desired PDF, namely, that of the observed solar irradiance. This induces bias in AnEn forecasts limiting their performance. The bias can be ascribed to the limited size of data available in the AnEn forecast system. In many practical cases, this limitation is unavoidable. We propose a correction scheme based on quantile mapping to deal with this situation. By enhancing forecast reliability, this scheme provides better continuous ranked probability scores compared with the basic AnEn method. However, PDF estimation in the quantile mapping process can be unstable. We therefore propose another scheme that introduces constraints to mitigate this uncertainty thereby achieving further performance gains.

Projecting current feelings into the past and future: Better current relationship quality reduces negative retrospective bias and increases positive forecasting bias

We examine bias in how people perceive their romantic relationship over time. Participants appraised their relationship 6 months and 1 year ago on average more negatively than they had done at the time (retrospective bias) but showed no significant mean-level forecasting bias. Higher relationship quality at the time of appraisal was linked to less negative retrospective bias but to more positive forecasting bias (Study 1). Similarly, participants who were experimentally manipulated to focus on the high relationship quality aspects in their relationship (vs. considering challenging aspects of their relationship) showed more forecasting bias and less retrospective bias (Study 2). This pattern occurred due to participants’ projecting positive current feelings onto predicted relationship quality in the future and remembered relationship quality in the past. This projection reduced the overall negative bias in recalled relationship quality for those currently perceiving higher relationship quality but increased positive bias in forecasted relationship quality.

Verifying operational intra-day solar forecasts from ECMWF and NOAA

Global horizontal irradiance (GHI) forecasting at intra-day horizons of up to 12-h ahead is vital to grid integration of solar photovoltaics, but has been fundamentally difficult for all methods that do not involve numerical weather prediction (NWP), since non-NWP methods are unable to extrapolate the data to a horizon beyond a fews hours. The European Centre for Medium-Range Weather Forecasts (ECMWF) and the National Oceanic and Atmospheric Administration (NOAA) are the most representative weather centers in Europe and America, respectively. To understand their operational impact and value to grid integration, the ECMWF’s High Resolution (HRES) model and two models from NOAA, namely, Rapid Refresh (RAP) and High-Resolution Rapid Refresh (HRRR), are validated through the Murphy–Winkler distribution-oriented verification framework, over the year 2020, at seven locations. All forecasts are retrieved at native horizontal resolutions—9 km for HRES, 13 km for RAP, and 3 km for HRRR; it depicts the “off-the-shelf” scenario if these forecasts are to be utilized by end users. Three simple linear correction methods, each being statistically optimal in its own respect, are used to post-process the raw forecasts. It was found that 1–12-h-ahead ECMWF’s HRES forecasts have a significantly lower root mean square error (14.0–33.7%) as compared to NOAA’s HRRR (19.0–53.2%) and RAP (19.2–45.9%). Even after the large biases in HRRR and RAP forecasts are removed, those post-processed versions are still inferior to the raw HRES forecasts.

Inside the black box of institutional investors in an emerging market: the role of sell side analyst outputs

PurposeIn the backdrop of emerging market setting of India, this study aims to attempt to identify how Institutional investors use sell side analyst outputs for their decision-making processes in light of inherent biases in their forecasts and recommendations. The study also conceptualizes the role of internal buy side teams in the process and try to figure out the key attributes and services provided by sell side analysts, which provide maximum value to the investors.Design/methodology/approachThe study is centered upon in-depth semi-structured interviews of ten institutional investors from top Indian asset management companies covering a wide range of topics tied back to theoretical explanations. The data collected was transcribed, coded and analyzed using content analysis to ensure a systematic synthesis of point of view.FindingsThe findings show that internal analyst teams of institutional investors play a dominant role in terms of validation of sell side analysts’ outputs (given the inherent biases in sell side analyst forecasts). Further, the engagement of sell side analysts by the investors are determined not only through profitable recommendations but also on the basis of soundness of the investment rationale along with other services provided. Finally, this study puts into perspective, the critical role of analyst industry knowledge and access to company management (as opposed to analyst pedigree and forecast accuracy) for institutional investors decision-making.Practical implicationsThe findings of the paper have profound implications for various stakeholders such as companies, sell side analysts, policy makers, researchers and students of finance in terms of detailed understanding of investment processes of institutional investors in the context of emerging markets like India, which have a different legal and regulatory set-up compared to developed markets. The authors also provide a critical perspective through an intriguing paradox that exists between finance theory and its relevance for actual practitioners.Originality/valueTo the best of the authors’ knowledge, this is the first study in India which look inside the “black box” of institutional investors and their decision-making process, especially with respect to how they use sell side outputs.

Do independent fiscal institutions cause better fiscal outcomes in the European Union?

This paper explores the budgetary implications of the independent fiscal institutions (IFIs) in the European Union (EU). We employ a dynamic panel model for the period 2000–2019 and find that these fiscal watchdogs have a positive and significant influence not only on government budget balance for the EU member states, resulting in smaller government budget deficits, but also on countries’ compliance with fiscal rules, results that hold across alternative fiscal balances. IFIs appear to have a beneficial impact on fiscal performance and compliance with numerical targets in countries with poorly designed fiscal responsibility norms but weaker influence when fiscal rules are less binding (well-designed fiscal rules). The findings remain significant regardless the year of accession to the EU (old vs. new members) or euro-area status (euro-area vs. non-euro-area members). However, we document that IFIs play a larger role in countries that established these monitoring bodies before 2013, indicating that experience matters in IFI performance. Also, our findings show that the influence of IFIs remains if we take into account institutional reforms in which their mandates were extended with different powers and tasks, which has a positive and significant effect on fiscal balances. Moreover, we find that, under the circumstance of systemic and banking crises, these institutions are associated with improved fiscal outcomes, reflecting their increased concern about the path of public finances and their role in reducing budgetary forecasting biases. Our results are robust to a variety of specifications and models, including alternative measures of the government budget balance and after controlling for a set of institutional characteristics and for potential endogeneity in the estimations.