Gerrity Skill Score Research Articles

Assessing the skill of the UK Hydrological Outlook

ABSTRACTThe UK Hydrological Outlook (UKHO) is a seasonal forecast of future river flows and groundwater levels. The UKHO contains both presentations of outputs from models simulating future conditions and a high-level summary. The summary is produced by an expert panel of forecasters that considers the model outputs together with other recent hydrological and meteorological information. Whilst the skill and uncertainty of the individual models have been explored and published, this study sets out to establish the performance of the high-level summary, and presents such an assessment of the river flow forecasts at the 1-month timescale. Both qualitative and quantitative assessments are presented and compared with two naïve forecasting methods. The UKHO summary is found to have a similar Gerrity skill score to a “same as last month” forecast, an outcome that generates suggestions for improvements in how the different model outputs should be considered and presented in the high-level summary.

A comparison of the rainfall forecasting skills of the WRF ensemble forecasting system using SPCPT and other cumulus parameterization error representation schemes

The scientific community mainly uses ensemble systems to represent various sources of uncertainties and to produce better forecasts. The inability to choose accurate initial conditions leads to failed forecasts due to the well-known butterfly effect; however, in modern weather models, researchers have paid more attention to inadequacies in the sense of physical parameterization schemes and other dynamic processes. In this study, the uncertainties caused by cumulus parameterization are represented by the stochastic perturbed parameterization tendency (SPPT) scheme, and the results are compared with those of classic schemes, including the multi cumulus parameterization scheme, the parameter-perturbed scheme, and the Gaussian-noise-perturbed tendency scheme. The impacts of these various schemes on the precipitation predictions of the Weather Research and Forecasting (WRF) model are compared for Southeast Asia. The results using the stochastic perturbed cumulus parameterization tendency (SPCPT) perturbation scheme are contrasted with those of the total parameterization tendency perturbation scheme. Compared to the other schemes, the multi cumulus parameterization scheme has better Brier skill scores (BSS) and greater spreads. However, the SPCPT scheme shows significant improvements in the root mean square error (RMSE) and the Gerrity skill score (GSS) for rainfall prediction. This result also implies that the noise pattern is critical to the ensemble system; thus, using a single error representation scheme may be insufficient to estimate the error in the cumulus parameterization process.

Attribution of Large‐Scale Climate Patterns to Seasonal Peak‐Flow and Prospects for Prediction Globally

AbstractFlood‐related fatalities and impacts on society surpass those from all other natural disasters globally. While the inclusion of large‐scale climate drivers in streamflow (or high‐flow) prediction has been widely studied, an explicit link to global‐scale long‐lead prediction is lacking, which can lead to an improved understanding of potential flood propensity. Here we attribute seasonal peak‐flow to large‐scale climate patterns, including the El Niño Southern Oscillation (ENSO), Pacific Decadal Oscillation (PDO), North Atlantic Oscillation (NAO), and Atlantic Multidecadal Oscillation (AMO), using streamflow station observations and simulations from PCR‐GLOBWB, a global‐scale hydrologic model. Statistically significantly correlated climate patterns and streamflow autocorrelation are subsequently applied as predictors to build a global‐scale season‐ahead prediction model, with prediction performance evaluated by the mean squared error skill score (MSESS) and the categorical Gerrity skill score (GSS). Globally, fair‐to‐good prediction skill (20% ≤ MSESS and 0.2 ≤ GSS) is evident for a number of locations (28% of stations and 29% of land area), most notably in data‐poor regions (e.g., West and Central Africa). The persistence of such relevant climate patterns can improve understanding of the propensity for floods at the seasonal scale. The prediction approach developed here lays the groundwork for further improving local‐scale seasonal peak‐flow prediction by identifying relevant global‐scale climate patterns. This is especially attractive for regions with limited observations and or little capacity to develop flood early warning systems.

Development of statistical prediction models for Changma precipitation: An ensemble approach

An ensemble statistical forecast scheme with a one-month lead is developed to predict year-to-year variations of Changma rainfall over the Korean peninsula. Spring sea surface temperature (SST) anomalies over the North Atlantic, the North Pacific and the tropical Pacific Ocean have been proposed as useful predictors in a previous study. Through a forward-stepwise regression method, four additional springtime predictors are selected: the northern Indian Ocean (NIO) SST, the North Atlantic SST change (NAC), the snow cover anomaly over the Eurasian continent (EUSC), and the western North Pacific outgoing longwave radiation anomaly (WNP (OLR)). Using these, three new prediction models are developed. A simple arithmetic ensemble mean produces much improved forecast skills compared to the original prediction model of Lee and Seo (2013). Skill scores measured by temporal correlation and MSSS (mean square error skill score) are improved by about 9% and 17%, respectively. The GMSS (Gerrity skill score) and hit rate based on a tercile prediction validation scheme are also enhanced by about 19% and 13%, respectively. The reversed NIO, reversed WNP (OLR), and reversed NAC are all related to the enhancement of a cyclonic circulation anomaly to the south or southwest of the Korean peninsula, which induces southeasterly moisture flux into the peninsula and increasing Changma precipitation. The EUSC predictor induces an enhancement of the Okhotsk Sea high downstream and thus strengthening of Changma front.

Assessing the Skill of Updated Precipitation-Type Diagnostics for the Rapid Refresh with mPING

Abstract Previous work has shown that the Rapid Refresh (RAP) model severely underrepresents ice pellets in its grid, with a skill near zero and a very low bias. An ice pellet diagnostic upgrade was devised at the Earth System Research Laboratory (ESRL) to resolve this issue. Parallel runs of the experimental ESRL-RAP with the fix and the operational NCEP-RAP without the fix provide an opportunity to assess whether this upgrade has improved the overall performance and the performance of the individual precipitation types of the ESRL-RAP. Verification was conducted using the mobile Phenomena Identification Near the Ground (mPING) project. The overall Gerrity skill score (GSS) for the ESRL-RAP was improved relative to the NCEP-RAP at a 3-h lead time but degraded with increasing lead time; the difference is significant at p < 0.05. Whether this difference is practically significant for users is unknown. Some improvement was found in the bias and skill scores of ice pellets and snow in the ESRL-RAP, although the model continues to underrepresent ice pellets, while rain and freezing rain were generally the same or slightly worse with the fix. The ESRL-RAP was also found to depict a more realistic spatial distribution of precipitation types in transition zones involving ice pellets and freezing rain.

Assessment of Arctic Cloud Cover Anomalies in Atmospheric Reanalysis Products Using Satellite Data

Abstract Cloud cover is one of the largest uncertainties in model predictions of the future Arctic climate. Previous studies have shown that cloud amounts in global climate models and atmospheric reanalyses vary widely and may have large biases. However, many climate studies are based on anomalies rather than absolute values, for which biases are less important. This study examines the performance of five atmospheric reanalysis products—ERA-Interim, MERRA, MERRA-2, NCEP R1, and NCEP R2—in depicting monthly mean Arctic cloud amount anomalies against Moderate Resolution Imaging Spectroradiometer (MODIS) satellite observations from 2000 to 2014 and against Cloud–Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) observations from 2006 to 2014. All five reanalysis products exhibit biases in the mean cloud amount, especially in winter. The Gerrity skill score (GSS) and correlation analysis are used to quantify their performance in terms of interannual variations. Results show that ERA-Interim, MERRA, MERRA-2, and NCEP R2 perform similarly, with annual mean GSSs of 0.36/0.22, 0.31/0.24, 0.32/0.23, and 0.32/0.23 and annual mean correlation coefficients of 0.50/0.51, 0.43/0.54, 0.44/0.53, and 0.50/0.52 against MODIS/CALIPSO, indicating that the reanalysis datasets do exhibit some capability for depicting the monthly mean cloud amount anomalies. There are no significant differences in the overall performance of reanalysis products. They all perform best in July, August, and September and worst in November, December, and January. All reanalysis datasets have better performance over land than over ocean. This study identifies the magnitudes of errors in Arctic mean cloud amounts and anomalies and provides a useful tool for evaluating future improvements in the cloud schemes of reanalysis products.

Analysis of Index Insurance Potential for Adaptation to Hydroclimatic Risks in the West African Sahel

Abstract Index insurance has been viewed as a financial adaptation to climate risks with the potential for widespread application, especially in a developing world context. The potential for index insurance is explored in the context of hypothetical drought and flood contracts at the national level for farmers in the West African Sahel nations of Niger, Burkina Faso, and Mali. The region’s climatology and dynamics are discussed and multiple datasets are considered as potential indices. Agricultural, precipitation, streamflow, remotely sensed vegetation, and Niño sea surface temperature indices were explored as potential bases for index insurance contract. Correlation analyses between the potential geophysical and agricultural indices are examined and two of the rainfall datasets are found to have robust positive correlations with millet production in all three nations, while a particular streamflow index is found to have a robust negative correlation with rice production in Niger. A methodological innovation of this research is the use of Gerrity skill score (GSS) analysis to analyze the indices of high correlation. The correlation and GSS analyses presented here indicate the potential for index insurance using two of the rainfall datasets for the millet crop (drought risk) of all three nations and the Niamey flood month streamflow dataset for the rice crop of Niger (flood risk).

The statistical extended-range (10–30-day) forecast of summer rainfall anomalies over the entire China

The extended-range (10–30-day) rainfall forecast over the entire China was carried out using spatial–temporal projection models (STPMs). Using a rotated empirical orthogonal function analysis of intraseasonal (10–80-day) rainfall anomalies, China is divided into ten sub-regions. Different predictability sources were selected for each of the ten regions. The forecast skills are ranked for each region. Based on temporal correlation coefficient (TCC) and Gerrity skill score, useful skills are found for most parts of China at a 20–25-day lead. The southern China and the mid-lower reaches of Yangtze River Valley show the highest predictive skills, whereas southwestern China and Huang-Huai region have the lowest predictive skills. By combining forecast results from ten regional STPMs, the TCC distribution of 8-year (2003–2010) independent forecast for the entire China is investigated. The combined forecast results from ten STPMs show significantly higher skills than the forecast with just one single STPM for the entire China. Independent forecast examples of summer rainfall anomalies around the period of Beijing Olympic Games in 2008 and Shanghai World Expo in 2010 are presented. The result shows that the current model is able to reproduce the gross pattern of the summer intraseasonal rainfall over China at a 20-day lead. The present study provides, for the first time, a guide on the statistical extended-range forecast of summer rainfall anomalies for the entire China. It is anticipated that the ideas and methods proposed here will facilitate the extended-range forecast in China.

Long‐range seasonal streamflow forecasting over the Iberian Peninsula using large‐scale atmospheric and oceanic information

AbstractIdentifying the relationship between large‐scale climate signals and seasonal streamflow may provide a valuable tool for long‐range seasonal forecasting in regions under water stress, such as the Iberian Peninsula (IP). The skill of the main teleconnection indices as predictors of seasonal streamflow in the IP was evaluated. The streamflow database used was composed of 382 stations, covering the period 1975–2008. Predictions were made using a leave‐one‐out cross‐validation approach based on multiple linear regression, combining Variance Inflation Factor and Stepwise Backward selection to avoid multicollinearity and select the best subset of predictors. Predictions were made for four forecasting scenarios, from one to four seasons in advance. The correlation coefficient (RHO), Root Mean Square Error Skill Score (RMSESS), and the Gerrity Skill Score (GSS) were used to evaluate the forecasting skill. For autumn streamflow, good forecasting skill (RHO>0.5, RMSESS>20%, GSS>0.4) was found for a third of the stations located in the Mediterranean Andalusian Basin, the North Atlantic Oscillation of the previous winter being the main predictor. Also, fair forecasting skill (RHO>0.44, RMSESS>10%, GSS>0.2) was found in stations in the northwestern IP (16 of these located in the Douro and Tagus Basins) with two seasons in advance. For winter streamflow, fair forecasting skill was found for one season in advance in 168 stations, with the Snow Advance Index as the main predictor. Finally, forecasting was poorer for spring streamflow than for autumn and winter, since only 16 stations showed fair forecasting skill in with one season in advance, particularly in north‐western of IP.

The Development of a Statistical Forecast Model for Changma

Abstract Forecasting year-to-year variations in East Asian summer monsoon (EASM) precipitation is one of the most challenging tasks in climate prediction because the predictors are not sufficiently well known and the forecast skill of the numerical models is poor. In this paper, a statistical forecast model for changma (the Korean portion of the EASM system) precipitation is proposed that was constructed with three physically based predictors. A forward-stepwise regression was used to select the predictors that included sea surface temperature (SST) anomalies over the North Pacific, the North Atlantic, and the tropical Pacific Ocean. Seasonal predictions with this model showed high forecasting capabilities that had a Gerrity skill score of ~0.82. The dynamical processes associated with the predictors were examined prior to their use in the prediction scheme. All predictors tended to induce an anticyclonic anomaly to the east or southeast of Japan, which was responsible for transporting a large amount of moisture to the southern Korean Peninsula. The predictor in the North Pacific formed an SST front to the east of Japan during the summertime, which maintained a lower-tropospheric baroclinicity. The North Atlantic SST anomaly induced downstream wave propagation in the upper troposphere, developing anticyclonic activity east of Japan. Forcing from the tropical Pacific SST anomaly triggered a cyclonic anomaly over the South China Sea, which was maintained by atmosphere–ocean interactions and induced an anticyclonic anomaly via northward Rossby wave propagation. Overall, the model used for forecasting changma precipitation performed well (R = 0.85) and correctly predicted information for 16 out of 19 yr of observational data.

Hydro-climatic forecasting using sea surface temperatures: methodology and application for the southeast US

Sea surface temperatures (SSTs) are often used for the development of hydro-climatic variable forecasts based on teleconnection methods. Such methods rely on projections or linear combinations of teleconnection indices [e.g. El Nino-Southern Oscillation (ENSO)] and other predictor fields. This study introduces a new hydro-climatic forecasting method identifying SST “dipole” predictors motivated by major teleconnection patterns. An SST dipole is defined as a function of average SST anomalies over two oceanic areas of specific sizes and geographic locations. An optimization algorithm is developed to search for the most significant SST dipole predictors of an external hydro-climatic series based on the Gerrity Skill Score. The significant dipoles are cross-validated and used to generate multiple forecast values. The new method is applied to the forecasting of seasonal precipitation over the southeast US. Hindcasting results show that significant dipoles related to ENSO as well as other prominent patterns at different lead times can indeed be identified. The dipole method also compares favorably with existing statistical forecasting schemes with respect to multiple skill measures. Furthermore, an operational forecasting framework able to produce ensemble forecast traces and uncertainty intervals that can support regional water resources planning and management is also developed.

Verification of marine forecasts using an objective area forecast verification system

ABSTRACTMany meteorological service providers issue forecasts that contain short summary statements. These forecasts attempt to capture all the expected conditions during a forecast period. The area forecast verification system has been developed to verify these forecasts when they are issued in pre‐determined geographical areas. This system verifies forecasts by comparing them with gridded nowcast model analysis fields. Results are presented for the wind speed component of the shipping forecast and the inshore waters forecast. Both of these products are issued by the UK Met Office on behalf of the Maritime and Coastguard Agency. Like most summary forecasts these products contain hand written text. This text is carefully interpreted into a series of invariant sub‐periods. These sub‐periods are then compared to the nowcast model analysis fields. During each forecast sub‐period all available hourly nowcast model analyses are accumulated to create an observation distribution within that sub‐period. Verification is performed using a contingency table approach which is specifically designed to reflect the distribution formed by the observations within each sub‐period of the forecast. Various performance measures are calculated including the Gerrity skill score – which is weighted according to the priorities of the customer. Shipping forecasts issued during the 12 month period ending on 31 August 2011 had a proportion correct to within ± 1 Beaufort Force of 0.886 and a Gerrity skill score of 0.823. Inshore waters forecasts issued during this period had a proportion correct to within ± 1 Beaufort Force of 0.940 and a Gerrity skill score of 0.848. Analysis indicates that both forecasts tend to exaggerate the wind speeds and often neglect to include the lower Beaufort Force categories. © 2013 British Crown copyright, the Met Office. Published by John Wiley & Sons Ltd.

Hindcast skill and predictability for precipitation and two-meter air temperature anomalies in global circulation models over the Southeast United States

This paper presents an assessment of the seasonal prediction skill of current global circulation models, with a focus on the two-meter air temperature and precipitation over the Southeast United States. The model seasonal hindcasts are analyzed using measures of potential predictability, anomaly correlation, Brier skill score, and Gerrity skill score. The systematic differences in prediction skill of coupled ocean–atmosphere models versus models using prescribed (either observed or predicted) sea surface temperatures (SSTs) are documented. It is found that the predictability and the hindcast skill of the models vary seasonally and spatially. The largest potential predictability (signal-to-noise ratio) of precipitation anywhere in the United States is found in the Southeast in the spring and winter seasons. The maxima in the potential predictability of two-meter air temperature, however, reside outside the Southeast in all seasons. The largest deterministic hindcast skill over the Southeast is found in wintertime precipitation. At the same time, the boreal winter two-meter air temperature hindcasts have the smallest skill. The large wintertime precipitation skill, the lack of corresponding two-meter air temperature hindcast skill, and a lack of precipitation skill in any other season are features common to all three types of models (atmospheric models forced with observed SSTs, atmospheric models forced with predicted SSTs, and coupled ocean–atmosphere models). Atmospheric models with observed SST forcing demonstrate a moderate skill in hindcasting spring-and summertime two-meter air temperature anomalies, whereas coupled models and atmospheric models forced with predicted SSTs lack similar skill. Probabilistic and categorical hindcasts mirror the deterministic findings, i.e., there is very high skill for winter precipitation and none for summer precipitation. When skillful, the models are conservative, such that low-probability hindcasts tend to be overestimates, whereas high-probability hindcasts tend to be underestimates.