Mean Square Skill Score Research Articles

Enhancing Forecast Skill of Winter Temperature of East Asia Using Teleconnection Patterns Simulated by GloSea5 Seasonal Forecast Model

GloSea5, a seasonal forecast system of the UK Met Office, shows reasonable skill among state-of-the-art operational seasonal forecast systems. However, the average surface temperature (T2m) in winter (December–February) of GloSea5 is particularly low in East Asia. To improve the seasonal forecast skill over East Asia, we focused on the high skill score of global teleconnection patterns simulated by GloSea5. Among the well-predicted teleconnection patterns, we selected those highly correlated with the East Asian T2m: East Atlantic (EA), Polar/Eurasia (PE), East Atlantic/Western Russia (EAWR), and West Pacific (WP) patterns. A multiple linear regression model was constructed using the selected teleconnection indices as predictors. These results are promising. The statistical skill-score evaluation of the constructed linear regression model using the anomaly correlation coefficient (ACC), root mean squared error (RMSE), and mean-squared skill score (MSSS) showed an improvement in the predicted T2m of East Asia, where the values of ACC and MSSS increased by 0.25 and 0.37, respectively, and the RMSE decreased by 0.63 compared to the dynamic forecast model results. These results suggest that a well-designed combined statistical and dynamical approach for seasonal prediction can be beneficial for some regions where the predictability of the dynamic model exhibits a low value.

Evaluation of the Dynamical–Statistical Downscaling Model for Extended Range Precipitation Forecasts in China

In order to focus on pentad-scale precipitation forecasts, we investigated the coupling relationship between 500 hPa geopotential height (Z500) anomalies and precipitation anomalies using the China Meteorological Administration Global Land Surface ReAnalysis Interim (CRA40/Land) gridded precipitation dataset from 1999 to 2018 and the National Centers for Environmental Prediction 1 reanalysis dataset for Z500. We obtained a dynamical–statistical downscaling model (DSDM) on the pentad scale and used the daily Z500 forecast product for sub-seasonal to seasonal forecasts (15–60 days) of the FGOALS-f2 model as the predictor. Our results showed that pentad-scale prediction of precipitation is the key to bridging the current deficiencies in sub-seasonal forecasts. Compared with the FGOALS-f2 model, the pentad DSDM had a higher skill for prediction of precipitation in China at lead times longer than four pentads throughout the year and of two pentads in the summer months. FGOALS-f2 had excellent precipitation predictability at lead times less than three pentads (15 days), so the proposed pentad DSDM could not perform better than FGOALS-f2 in this period. However, at lead times greater than four pentads, the precipitation prediction scores (such as the anomaly correlation coefficient (ACC), the temporal correlation coefficient (TCC) and the mean square skill score (MSSS)) of the pentad DSDM for the whole of China were higher than those of the FGOALS-f2 model. With the rate of increase ranging from 76% to 520%, the mean ACC scores of pentad DSDM were basically greater than 0.04 after a lead time of five pentads, whereas those of the FGOALS-f2 were less than 0.04. An analysis of the Zhengzhou “720” super heavy rainstorm event showed that the pentad DSDM also had better predictability for the distribution of precipitation at lead times of three pentads than the FGOALS-f2 model for the extreme precipitation event.

The skill assessment of ENSO prediction issued by JMA ensemble prediction system and CFSv2

Indonesian climate is strongly affected by El Niño-Southern Oscillation (ENSO) as one of climate-driven factor. ENSO prediction during the upcoming months or year is crucial for the government in order to design the further strategic policy. Besides producing its own ENSO prediction, BMKG also regularly releases the status and ENSO prediction collected from other climate centers, such as Japan Meteorological Agency (JMA) and National Oceanic and Atmospheric Administration (NOAA). However, the skill of these products is not well known yet. The aim of this study is to conduct a simple assessment on the skill of JMA Ensemble Prediction System (EPS) and NOAA Climate Forecast System version 2 (CFSv2) ENSO prediction using World Meteorological Organization (WMO) Standard Verification System for Long Range Forecast (SVS-LRF) method. Both ENSO prediction results also compared each other using Student's t-test. The ENSO predictions data were obtained from the ENSO JMA and ENSO NCEP forecast archive files, while observed Nino 3.4 were calculated from Centennial in situ Observation-Based Estimates (COBE) Sea Surface Temperature Anomaly (SSTA). Both ENSO prediction issued by JMA and NCEP has a good skill on 1 to 3 months lead time, indicated by high correlation coefficient and positive value of Mean Square Skill Score (MSSS). However, the skill of both skills significantly reduced for May-August target month. Further careful interpretation is needed for ENSO prediction issued on this mentioned period.

Extended-Range Runoff Forecasting Using a One-Way Coupled Climate–Hydrological Model: Case Studies of the Yiluo and Beijiang Rivers in China

Extended-range runoff forecasting is important for water resources management and energy planning. Experimental extended-range runoff was hindcasted, based on an extended-range climate model, developed by National Climate Center of the China Meteorological Administration, and semi-distributed hydrological model HBV-D. The skill of the runoff forecasts was explored using mean square skill score (MSSS), anomaly correlation coefficient (ACC), and areas under the relative operating characteristics curve (AUC) for three terciles for three experimental 51-day periods during flood season (June 1 to July 21, July 1 to August 20 and August 1 to September 20) for two rivers in China. The results revealed decreasing trends of the five indices, and varying length of the continuous longest skilful time slice from 3 days to 6 weeks depending on index, period and river location. In most cases, skilful abnormal terciles forecast occurred more often or with similar frequency to deterministic forecasts. It suggests that ensemble probability forecasting is a method with potential for extended-range river runoff forecast. Further, abnormal terciles are more skillful than normal terciles, and above normal are more skillful than below normal. In terms of a temporal mean of the MSSS and ACC, deterministic forecasts are skillful for both rivers in all three periods, but more skillful for the Beijiang River than for the Yiluo River in most cases.

Seasonal-to-Interannual Prediction Skills of Near-Surface Air Temperature in the CMIP5 Decadal Hindcast Experiments

Abstract This study explores the seasonal-to-interannual near-surface air temperature (TAS) prediction skills of state-of-the-art climate models that were involved in phase 5 of the Coupled Model Intercomparison Project (CMIP5) decadal hindcast/forecast experiments. The experiments are initialized in either November or January of each year and integrated for up to 10 years, providing a good opportunity for filling the gap between seasonal and decadal climate predictions. The long-lead multimodel ensemble (MME) prediction is evaluated for 1981–2007 in terms of the anomaly correlation coefficient (ACC) and mean-squared skill score (MSSS), which combines ACC and conditional bias, with respect to observations and reanalysis data, paying particular attention to the seasonal dependency of the global-mean and equatorial Pacific TAS predictions. The MME shows statistically significant ACCs and MSSSs for the annual global-mean TAS for up to two years, mainly because of long-term global warming trends. When the long-term trends are removed, the prediction skill is reduced. The prediction skills are generally lower in boreal winters than in other seasons regardless of lead times. This lack of winter prediction skill is attributed to the failure of capturing the long-term trend and interannual variability of TAS over high-latitude continents in the Northern Hemisphere. In contrast to global-mean TAS, regional TAS over the equatorial Pacific is predicted well in winter. This is mainly due to a successful prediction of the El Niño–Southern Oscillation (ENSO). In most models, the wintertime ENSO index is reasonably well predicted for at least one year in advance. The sensitivity of the prediction skill to the initialized month and method is also discussed.

Multivariate time series modeling of short-term system scale irrigation demand

Summary Travel time limits the ability of irrigation system operators to react to short-term irrigation demand fluctuations that result from variations in weather, including very hot periods and rainfall events, as well as the various other pressures and opportunities that farmers face. Short-term system-wide irrigation demand forecasts can assist in system operation. Here we developed a multivariate time series (ARMAX) model to forecast irrigation demands with respect to aggregated service points flows (IDCGi, ASP) and off take regulator flows (IDCGi, OTR) based across 5 command areas, which included area covered under four irrigation channels and the study area. These command area specific ARMAX models forecast 1–5 days ahead daily IDCGi, ASP and IDCGi, OTR using the real time flow data recorded at the service points and the uppermost regulators and observed meteorological data collected from automatic weather stations. The model efficiency and the predictive performance were quantified using the root mean squared error (RMSE), Nash–Sutcliffe model efficiency coefficient (NSE), anomaly correlation coefficient (ACC) and mean square skill score (MSSS). During the evaluation period, NSE for IDCGi, ASP and IDCGi, OTR across 5 command areas were ranged 0.98–0.78. These models were capable of generating skillful forecasts (MSSS ⩾ 0.5 and ACC ⩾ 0.6) of IDCGi, ASP and IDCGi, OTR for all 5 lead days and IDCGi, ASP and IDCGi, OTR forecasts were better than using the long term monthly mean irrigation demand. Overall these predictive performance from the ARMAX time series models were higher than almost all the previous studies we are aware. Further, IDCGi, ASP and IDCGi, OTR forecasts have improved the operators’ ability to react for near future irrigation demand fluctuations as the developed ARMAX time series models were self-adaptive to reflect the short-term changes in the irrigation demand with respect to various pressures and opportunities that farmers’ face, such as changing water policy, continued development of water markets, drought and changing technology.

Seasonal prediction skill of winter temperature over North India

The climatology, amplitude error, phase error, and mean square skill score (MSSS) of temperature predictions from five different state-of-the-art general circulation models (GCMs) have been examined for the winter (December–January–February) seasons over North India. In this region, temperature variability affects the phenological development processes of wheat crops and the grain yield. The GCM forecasts of temperature for a whole season issued in November from various organizations are compared with observed gridded temperature data obtained from the India Meteorological Department (IMD) for the period 1982–2009. The MSSS indicates that the models have skills of varying degrees. Predictions of maximum and minimum temperature obtained from the National Centers for Environmental Prediction (NCEP) climate forecast system model (NCEP_CFSv2) are compared with station level observations from the Snow and Avalanche Study Establishment (SASE). It has been found that when the model temperatures are corrected to account the bias in the model and actual orography, the predictions are able to delineate the observed trend compared to the trend without orography correction.

Forecasting daily reference evapotranspiration for Australia using numerical weather prediction outputs

Farmers and irrigation system operators make real-time irrigation decisions based on a range of factors including short-term weather forecasts of rainfall and temperature. The simplest and oldest statistical method for forecasting daily ET0 is to use the long-term monthly mean ET0 based on historical observations. Forecasts of reference crop evapotranspiration (ET0) can be calculated from Numerical Weather Prediction (NWP) outputs and ET0 has the advantage of being more directly relevant to crop water requirements than temperature. This paper evaluates forecasts of ET0 made using the Bureau of Meteorology's operational NWP forecasts derived from the Australian Community Climate and Earth System Simulator – Global (ACCESS-G). The forecast performance for ET0 was quantified using the root mean squared error (RMSE), coefficient of determination (R2), anomaly correlation coefficient (ACC) and mean square skill score (MSSS). Daily ET0 forecasts for lead times up to 9 days were compared against ET0 calculated using hourly observations from the 40 automatic weather stations across Australia.It was found that using NWP forecast daily ET0 was better than using the long-term monthly mean ET0 for lead times up to 6 days, beyond which the long-term monthly mean was better. The average MSSS for ET0 forecasts across all stations varied between 66% and 12% for lead times of 1–6 days, respectively. Further, it was found that forecast performance for daily ET0 was highest in autumn for tropical climates and lowest in spring for temperate climates. Errors in incoming solar radiation were the most important source of ET0 forecast error, followed by air temperature, dew point temperature and wind speed, for all lead times. Also, it was found that the forecast performances for incoming solar radiation and mean wind speed were relatively poor compared with the air and dew point temperatures

Winter North Atlantic Oscillation Hindcast Skill: 1900–2001

Abstract Recent proposed seasonal hindcast skill estimates for the winter North Atlantic Oscillation (NAO) are derived from different lagged predictors, NAO indices, skill assessment periods, and skill validation methodologies. This creates confusion concerning what is the best-lagged predictor of the winter NAO. To rectify this situation, a standardized comparison of NAO cross-validated hindcast skill is performed against three NAO indices over three extended periods (1900–2001, 1950–2001, and 1972–2001). The lagged predictors comprise four previously published predictors involving anomalies in North Atlantic sea surface temperature (SST), Northern Hemisphere (NH) snow cover, and an additional predictor, an index of NH subpolar summer air temperature (TSP). Significant (p < 0.05) NAO hindcast skill is found with May SST 1900–2001, summer/autumn SST 1950–2001, and warm season snow cover 1972–2001. However, the highest and most significant hindcast skill for all periods and all NAO indices is achieved with TSP. Hindcast skill is nonstationary using all predictors and is highest during 1972–2001 with a TSP correlation skill of 0.59 and a mean-squared skill score of 35%. Observational evidence is presented to support a dynamical link between summer TSP and the winter NAO. Summer TSP is associated with a contemporaneous midlatitude zonal wind anomaly. This leads a pattern of North Atlantic SST that persists through autumn. Autumn SSTs may force a direct thermal NAO response or initiate a response via a third variable. These findings suggest that the NH subpolar regions may provide additional winter NAO lagged predictability alongside the midlatitudes and the Tropics.