Forecast Bias Research Articles

Investigation of model forecast biases and skilful prediction for Assam heavy rainfall 2022

Extreme rainfall events (ERE) during the summer monsoon season have been occurring over most parts of India resulting in flooding and immense socio-economic loss. These extremes are becoming a frequent norm in the hilly and mountainous regions of the country such as Assam. Assam received one of the most historical EREs from 14–June 17, 2022. The present study analyses the performance of a suite of high-resolution ensemble model forecasts for this extreme event in terms of its intensity, and distribution with a lead time of up to 96 h. Furthermore, the 36 numerical experiments are carried out using two different land use and land cover (LULC) data sets (i.e. ISRO and USGS) and three different sets of parameterization schemes (i.e. planetary boundary layer, cumulus, and microphysics).Rainfall distributions in the case of USGS LULC are relatively less coherent and underestimated (60–260 mm/day) against IMD (80–300 mm/day) including the rainfall categories heavy (HR), very heavy (VHR), and extremely heavy (EHR) rainfall throughout the day-1 to day-4. Among all the ensembles (E1-E10), USGS (E6 - E10) has underestimated rainfall (140–260 mm/day) compared to ISRO (150–280 mm/day), specifically in MR and HR categories over the upper Assam (UAD) and lower Assam (LAD) divisions. Further, the Bias Correction Ensemble (BCE) technique is applied to minimize the forecast errors. A rigorous statistical analysis in terms of frequency distribution, Taylor diagram, and benchmark skill scores is carried out to elucidate the model biases. The set of the model ensembles using ISRO (E1- E5) and USGS (E6- E10) reasonably captured the HR, VHR, and EHR. In addition, throughout the forecast hour, BCE E5 (E10) is noted with the distinct realistic (underestimated) representation of model bias (5–20 %) (10–30 %) over all the subdivisions of Assam. Our results suggest that the combined efforts of ensembles of physical parameterization schemes, along with proper LULC, and the BCE approach are required to overcome challenges to improve the skills of rainfall events, particularly over complex terrains such as Assam.

Improving the atmospheric dispersion forecasts over Washington, D.C. using UrbanNet observations: A study with HYSPLIT model

UrbanNet data have been used to enhance the predictions of the Weather Research and Forecasting (WRF) model through observational nudging to improve temperature and wind predictions. HYSPLIT was utilized to understand the impact of using the observationally nudged WRF fields in dispersion modeling. The meteorological observations collected from the National Weather Service monitoring stations located at two major airports in Washington metropolitan area were also assimilated into WRF modeling. The results showed that observational nudging successfully adjusted WRF wind fields towards the observations and significantly reduced the forecast temperature bias at nighttime. The comparison of HYSPLIT simulations with and without the enhancement of the WRF model using UrbanNet and airport data showed significant differences in the pattern and direction of the dispersion plume especially during the early morning hours. Furthermore, ingesting the data from the closer airport to the downtown area in WRF provided HYSPLIT simulations very similar to the ones using UrbanNet data. This provides strong evidence that local data are essential to adjust weather prediction models routinely used to drive dispersion models. There was also evidence of increased mixing height when using local data collected in the downtown, mainly resulting from the increased surface heating in the city.

Digital Technology Innovation, Surplus Management and Analyst Forecast Bias

With the rapid development of digital economy, digital technology, as its technical support, has an important impact on market information environment and analysts' forecast deviation. Using the data of listed companies from 2007 to 2021, this paper investigates the impact of corporate digital technology innovation on analyst forecast bias. We find that enterprise digital technology innovation has a restraining effect on analyst forecast bias, and this conclusion is still valid after a series of robustness tests. Heterogeneity analysis shows that this effect is more obvious in firms with low media attention, investor attention and institutional ownership. Mechanism analysis shows that digital technology innovation inhibits firms' earnings management behavior, thus reducing analyst forecast bias. These results are consistent with the logic that digital technology innovation enables managers to use visual earnings management to suppress earnings manipulation and reduce analyst forecast bias. The conclusion of this paper expands the research scope of the economic effect of technological innovation in the digital era, and has certain significance for enterprises to optimize digital innovation strategy and improve market transparency.

ESG rating disagreement and idiosyncratic return volatility: Evidence from China

Following the increasing importance of sustainable development and the popularity of ESG investing activities, ESG ratings have grown to be crucial references for investors’ decision-making. However, there are substantial disagreements among different rating agencies. This study examines the impact of ESG rating disagreement on idiosyncratic return volatility using data from five prominent rating agencies: SynTao Green Finance, Huazheng, Hexun, Bloomberg, and Rankins ESG Ratings. The findings suggest that ESG rating disagreement will increase idiosyncratic return volatility. This relation is driven by investor attention and noise trading. Heterogeneity tests reveal that the higher analyst coverage and greater analyst forecast bias, the more pronounced the impact of ESG rating disagreement on idiosyncratic return volatility. While firms with foreign investors and more institutional investors can alleviate the interference.

Evaluation of Wintertime Precipitation Estimates and Forecasts in the Mountains of Colorado

Abstract Wintertime precipitation poses many observational and forecasting challenges, especially in the complex topography of the western United States where radar beam blockage and difficulty siting in situ observations yields more sparse observations than in the eastern United States. Uncertainty in western U.S. winter precipitation is known to be high, so much so that some studies have found model simulated precipitation to produce similar or better large-scale estimates of annual precipitation than gridded observational products during climatologically anomalous years. This study evaluates high-resolution gridded precipitation estimates from Multi-Radar Multi-Sensor (MRMS) and Stage IV as well as forecasts from NOAA’s High-Resolution Rapid Refresh (HRRR) model in the Colorado Rocky Mountains. Gridded precipitation estimates and forecasts are compared with in situ SNOTEL measurements for two seasons of wintertime precipitation. The influence of forecast length, lead time, and model elevation on seasonal precipitation predictions from the HRRR are investigated. Additional comparisons are made with the relatively dense network of observations deployed in Colorado’s East River Watershed during the Study of Precipitation, the Lower Atmosphere and Surface for Hydrometeorology (SPLASH) campaign. Gridded products and forecasts are found to underestimate cold-season precipitation by 25%–65% relative to in situ and aircraft measurements, with longer forecast periods and lead times (6–24 h) having smaller biases (25%–30%) than shorter forecast periods and lead times (55%–65%). The assessment of multiple years of observations indicates that these biases are related more to the data and methods used to create the gridded products and forecasts than to precipitation characteristics. Significance Statement In the mountainous western United States, it is very challenging to both observe and forecast wintertime precipitation, yet snowfall plays an important role in providing the region’s annual water supply. This study aims to increase our understanding of the biases in observations and forecasts of snowfall in the Colorado Rocky Mountains, which can in turn impact forecasts of water availability for the ensuing warm season. In this study we find high-resolution gridded precipitation estimates and forecasts to underestimate cold-season precipitation when compared with in situ observing stations, with longer-range forecasts (e.g., daily) being the least biased. These findings were consistent over two years of study and have broad implications for the hydrologic modeling and water management communities.

Ensemble Forecasts of Extreme Flood Events with Weather Forecasts, Land Surface Modeling and Deep Learning

Integrating numerical weather forecasts that provide ensemble precipitation forecasts, land surface hydrological modeling that resolves surface and subsurface hydrological processes, and artificial intelligence techniques that correct the forecast bias, known as the “meteo-hydro-AI” approach, has emerged as a popular flood forecast method. However, its performance during extreme flood events across different interval basins has received less attention. Here, we evaluated the meteo-hydro-AI approach for forecasting extreme flood events from headwater to downstream sub-basins in the Luo River basin during 2010–2017, with forecast lead times up to 7 days. The proposed meteo-hydro approach based on ECMWF weather forecasts and the Conjunctive Surface-Subsurface Process version 2 land surface model with a spatial resolution of 1 km captured the flood hydrographs quite well. Compared with the ensemble streamflow prediction (ESP) approach based on initial conditions, the meteo-hydro approach increased the Nash-Sutcliffe efficiency of streamflow forecasts at the three outlet stations by 0.27–0.82, decreased the root-mean-squared-error by 22–49%, and performed better in reliability and discrimination. The meteo-hydro-AI approach showed marginal improvement, which suggested further evaluations with larger samples of extreme flood events should be carried out. This study demonstrated the potential of the integrated meteo-hydro-AI approach for ensemble forecasting of extreme flood events.

A Multi‐Model Ensemble of Baseline and Process‐Based Models Improves the Predictive Skill of Near‐Term Lake Forecasts

AbstractWater temperature forecasting in lakes and reservoirs is a valuable tool to manage crucial freshwater resources in a changing and more variable climate, but previous efforts have yet to identify an optimal modeling approach. Here, we demonstrate the first multi‐model ensemble (MME) reservoir water temperature forecast, a forecasting method that combines individual model strengths in a single forecasting framework. We developed two MMEs: a three‐model process‐based MME and a five‐model MME that includes process‐based and empirical models to forecast water temperature profiles at a temperate drinking water reservoir. We found that the five‐model MME improved forecast performance by 8%–30% relative to individual models and the process‐based MME, as quantified using an aggregated probabilistic skill score. This increase in performance was due to large improvements in forecast bias in the five‐model MME, despite increases in forecast uncertainty. High correlation among the process‐based models resulted in little improvement in forecast performance in the process‐based MME relative to the individual process‐based models. The utility of MMEs is highlighted by two results: (a) no individual model performed best at every depth and horizon (days in the future), and (b) MMEs avoided poor performances by rarely producing the worst forecast for any single forecasted period (<6% of the worst ranked forecasts over time). This work presents an example of how existing models can be combined to improve water temperature forecasting in lakes and reservoirs and discusses the value of utilizing MMEs, rather than individual models, in operational forecasts.

A Potential Vorticity Diagnosis of Tropical Cyclone Track Forecast Errors

AbstractTropical cyclone (TC) track forecasting provides essential guidance for coastal communities. However, track forecast errors still occur, highlighting the need for continued research into error sources. Piecewise potential vorticity (PV) inversion is used systematically to quantitatively diagnose errors in track forecasts in four models during the 2017 Atlantic hurricane season. The deep layer mean steering flow (DLMSF) provides a sufficient proxy for hurricane movement, and DLMSF errors are correlated with TC track errors. Analysis of track forecasts for Hurricanes Harvey, Irma, and Maria reveals that their track errors are attributed to steering errors caused by misrepresentations of specific pressure systems. Harvey's westward track error in the GFS resulted from zonal wind errors from the Continental High, while Irma's northward track error in the SHiELD gfsIC resulted from meridional wind errors in the Bermuda High and Continental High. Maria's southward track error in the IFS resulted from meridional wind errors in the Bermuda High and a misrepresentation of Jose to Maria's northwest. The mean absolute error of the DLMSF shows that the Bermuda High contributed the most to steering flow errors in the cases examined. Our results show that piecewise PV inversion can identify the sources of biases in TC track forecasts. The correction of these biases may lead to improved track forecasts. Quantitative diagnostics presented here provide useful information for future model development.

Effect of franchising on the reliability of financial analysts’ earnings forecasts: Evidence from publicly traded US restaurant firms

Unlike institutional investors, who have expertise in collecting and analyzing firm information to predict firms’ future performance, individual investors have a limited ability to conduct this task. Therefore, individual investors are known to commonly rely on financial analysts’ earnings forecasts, which are freely available to them. However, as demonstrated by numerous studies, financial analysts’ forecasts are not always accurate and free from bias. Drawing on screening theory and the innate financial and structural differences between highly franchised restaurants and their less franchised peers, the current study showed that analysts provide more accurate and less optimistically biased forecasts for the former than the latter. However, the positive impact of an additional number of analysts on the accuracy of forecasts was weaker for the former than the latter. The current study can help individual investors in the restaurant industry improve their understanding of financial analysts’ earnings forecasts and make better-informed investment decisions.

Crowdsourced Forecasts and the Market Reaction to Earnings Announcement News

ABSTRACT This study examines whether crowdsourced forecasts of earnings and revenues help investors unravel bias in earnings announcement news, which is commonly derived from analyst forecasts. Our results suggest that investors, on average, understand and price the predictive signals reflected in crowdsourced forecasts about the bias in analyst-based earnings and revenue surprises. Using the staggered addition of firms to the Estimize platform, we find that crowdsourced coverage is associated with reductions in the mispricing of forecast bias and declines in earnings announcement premia. We further find some evidence that managers use income-increasing accruals to meet the crowdsourced forecast benchmark and that they respond to crowdsourced coverage through increased downward earnings and revenue guidance. Overall, we conclude that user-generated content on crowdsourced financial information platforms helps investors discount biases in traditional equity research and thereby better process the news in earnings announcements. JEL Classifications: G14; G20; M41.

Development and Evaluation of a Short-Term Ensemble Forecasting Model on Sea Surface Wind and Waves across the Bohai and Yellow Sea

In this study, an ensemble forecasting model for in situ wind speed and wave height was developed using the Coupled Ocean–Atmosphere–Wave–Sediment Transport (COAWST) model. This model utilized four bias correction algorithms—Model Output Statistics (MOS), Back Propagation Neural Network (BPNN), Long Short-Term Memory (LSTM) neural network, and Convolutional Neural Network (CNN)—to construct ensemble forecasts. The training data were derived from the COAWST simulations of one year and observations from three buoy stations (Laohutan, Zhifudao, and Lianyungang) in the Yellow Sea and Bohai Sea. After the optimization of the bias correction model training, the subsequent evaluations on the ensemble forecasts showed that the in situ forecasting accuracy of wind speed and wave height was significantly improved. Although there were some uncertainties on bias correction performance levels for individual algorithms, the uncertainties were greatly reduced by the ensemble forecasts. Depending on the dynamic weight assignment, the ensemble forecasts presented a stable performance even when the corrected forecasts by three algorithms had an obvious negative bias. Specifically, the ensemble forecasting bias was found with a mean reduction of about 96%~99% and 91%~95% for wind speed and wave height, and a reduction of about 91%~98% and 16%~54% during the period of Typhoon “Muifa”. For the four correction algorithms, the performance of bias correction was not directly related to the algorithm complexity. However, the strategies with more complex algorithms (i.e., CNN) were more conservative, and simple algorithms (i.e., MOS) might have induced unstable performance levels despite their lower bias in some cases.

Chaebol-affiliated analysts and biases in interpreting accruals

This study investigates whether analysts exhibit accrual-related biases in forecasting earnings in the presence of conflicts of interest arising from business group affiliation. Our findings reveal that chaebol-affiliated analysts are more likely to overreact to accruals than do independent analysts by overestimating the persistence of accruals. Notably, affiliated analysts’ accrual-related biases are not reduced with analysts’ experience but intensify when analysts have stronger incentives to provide favorable forecasts for member firms. This suggests a possibility that in the presence of conflicts of interest analysts do not fully incorporate the implications of accruals into their forecasts, potentially with an intention to benefit group interests. In additional analyses, we find that the presence of affiliated analysts does not exacerbate accrual mispricing, which suggests that investors do not take affiliated analysts’ biased forecasts at face value. We also provide evidence that despite issuing less accurate forecasts than unaffiliated analysts, affiliated analysts do not necessarily encounter more career disadvantages.

Adapting subseasonal-to-seasonal (S2S) precipitation forecast at watersheds for hydrologic ensemble streamflow forecasting with a machine learning-based post-processing approach

Accurate and reliable precipitation predictions made by dynamical forecast models could provide crucial information for human socioeconomic activities by enabling hydrologic forecasts at the Subseasonal-to-Seasonal (S2S) timescale. To utilize available S2S precipitation predictions for hydrologic forecasts, post-processing techniques have been applied to adapt the raw S2S precipitation to local watersheds. However, conventional statistical-based post-processing techniques are more focused on correcting the forecast bias, but rather limited in improving the predictive skill of available S2S precipitation forecasts. In this study, we combine the Random Forest classifiers (RF) with the Bias Correction and Spatial Disaggregation (BCSD) to adapt the 10-member ensemble precipitation forecast from the NASA Goddard Earth Observing System model version 5 (GEOS5) at 4 watersheds located in the NCEI South climate region of the United States. The adapted S2S precipitation is further applied for streamflow forecast by forcing a classical lumped hydrologic model. The performance of S2S precipitation as well as the corresponding streamflow predictions are benchmarked with the randomly resampled precipitation and the corresponding Ensemble Streamflow Prediction (ESP) framework-generated streamflow predictions. Evaluation statistics of Kling-Gupta Efficiency (KGE), Continuous Ranked Probability Skill Score (CRPSS), Reliability, Resolution, and Sharpness are employed to evaluate the predictive skill of precipitation and streamflow both deterministically and probabilistically. Our results indicate that dynamical S2S precipitation after forecast adaptation leads to consistently higher deterministic skill over ESP at all forecast lead times and across study watersheds. However, at longer forecast lead times beyond 10–15 days, S2S precipitation with a limited ensemble size does not present higher probabilistic skill than ESP. Our results shows that the joint application of RF and BCSD improves the predictive skill of the raw S2S precipitation at study watersheds in contrast to BCSD. Further, the added predictive skill of S2S precipitation brought by RF propagates into streamflow predictions, predominantly at longer forecast lead times exceeding 10 days. Overall, our results highlight the potential success of future work to apply other data-driven approaches to adapt the raw precipitation to local watersheds for more accurate and reliable streamflow forecasts at the S2S timescale.

Institutional ownership and analysts’ earnings forecasts

This paper selects the data of China's A-share market from 2011 to 2022 to explore the impact of institutional shareholding on analysts’ earnings forecasts. The findings indicate the following: institutional shareholding enhances analysts’ earnings forecast accuracy and reduces analysts’ forecast optimism bias; institutions with small geodesic pits will be more helpful for analysts to improve the quality of earnings forecasts; and institutional shareholding improves the quality of analysts’ earnings forecasts by promoting the quality of firms’ information disclosure.

Statistical time-series forecast error and bias assessment using hold-out samples and a single benchmark (data-reduction) scoring metric

In Time-Series (TS) forecast modeling, we utilize a Hold-Out (HO) sample to assess a candidate model forecast error and bias, with the Mean Absolute Percentage Error (MAPE) as a single benchmark (data-reduction) for a scoring metric. The purpose of such HO sampling is to assess error rates and accuracy (unbiasedness) level whether the forecasted values of a candidate TS model is within a pre-specified (targeted) margin-of-error (MOE) rate of α-percent. For instance, for an accuracy expectation of 80% level, the MOE rate of α is tolerated to be, 20%. If the MAPE of a candidate TS model is less than the MOE rate of α, then the model is considered reasonably accurate enough. Otherwise, we select the next available candidate TS model with the smallest-possible MAPE. For an illustration, we apply such method to a TS sales-data example.

Extrapolators and Contrarians: Forecast Bias and Household Stock Trading

Extrapolators and Contrarians: Forecast Bias and Household Stock Trading

Northern Hemisphere extratropical cyclone biases in ECMWF subseasonal forecasts

Extratropical cyclones influence midlatitude surface weather directly via precipitation and wind and indirectly via upscale feedbacks on the large‐scale flow. Biases in cyclone frequency and characteristics in medium‐range to subseasonal numerical weather prediction might therefore hinder exploitation of potential predictability on these timescales. We thus, for the first time, identify and track extratropical cyclones in 20 years (2000–2020) of subseasonal ensemble reforecasts from the European Centre for Medium‐Range Weather Forecasts (ECMWF) in the Northern Hemisphere in all seasons. The reforecasts reproduce the climatology of cyclone frequency and life‐cycle characteristics qualitatively well up to six weeks ahead. However, there are significant regional biases in cyclone frequency, which can result from a complex combination of biases in cyclone genesis, size, location, lifetime, and propagation speed. Their magnitude is largest in summer, with the strongest regional deficit of cyclones of more than 30% in the North Atlantic, relatively large in spring, and smallest in winter and autumn. Moreover, the reforecast cyclones reach too‐high intensities during most seasons, although intensification rates are captured well. An overestimation of cyclone lifetime might partly but not exclusively explain this intensity bias. While the cyclone bias patterns often appear in lead‐time weeks 1 and 2, their magnitudes typically grow further at subseasonal lead times, in some cases up to weeks 5 and 6. Most of the dynamical sources of these biases thus likely appear in the early medium range, but sources on longer timescales probably contribute to the further increase of biases with lead time. Our study provides a useful basis to identify, better understand, and ultimately reduce biases in the large‐scale flow and in surface weather in subseasonal weather forecasts. Given the considerable biases during summer, when subseasonal predictions of precipitation and surface temperature will become increasingly important, this season deserves particular attention for future research.

The disutility of compartmental model forecasts during the COVID-19 pandemic.

During the COVID-19 pandemic, several forecasting models were released to predict the spread of the virus along variables vital for public health policymaking. Of these, the susceptible-infected-recovered (SIR) compartmental model was the most common. In this paper, we investigated the forecasting performance of The University of Texas COVID-19 Modeling Consortium SIR model. We considered the following daily outcomes: hospitalizations, ICU patients, and deaths. We evaluated the overall forecasting performance, highlighted some stark forecast biases, and considered forecast errors conditional on different pandemic regimes. We found that this model tends to overforecast over the longer horizons and when there is a surge in viral spread. We bolstered these findings by linking them to faults with the SIR framework itself.

Do natural disasters hinder analysts’ information production of non-affected firms?

This paper demonstrates that natural disasters increase analyst forecast bias on non-affected firms through attention distraction. The results hold in robustness checks, and the impact is enhanced in firms with worse information environment and analysts with poorer attention management capability.

Tropical Atlantic rainfall drives bias in extratropical seasonal forecasts

AbstractWe investigate the impact of seasonal forecast biases in the Tropical Atlantic on the North Atlantic. The analysis uses a novel ensemble‐based method to estimate the impact of tropical rainfall bias on forecasts of the Extratropical North Atlantic. The inter‐ensemble spread of the forecast model is used to estimate the impact of the bias in Tropical Atlantic rainfall on the North Atlantic by selecting model members that happen to produce forecast anomalies that most closely resemble the tropical rainfall bias and using these as a proxy for the model error. The Tropical Atlantic rainfall bias impacts Rossby wave sources over the Subtropical Atlantic and there is a clear Rossby wave pattern originating from this area which is comparable to the mean bias in hindcasts. We argue that Tropical Atlantic rainfall errors explain a significant amount of the bias in seasonal forecasts over the Extratropical North Atlantic.