Brier Skill Score Research Articles

Story Recall in Peer Conflict Resolution Discourse Task to Identify Older Adults Testing Within Range of Cognitive Impairment.

The current study used behavioral measures of discourse complexity and story recall accuracy in an expository discourse task to distinguish older adults testing within range of cognitive impairment according to a standardized cognitive screening tool in a sample of self-reported healthy older adults. Seventy-three older adults who self-identified as healthy completed an expository discourse task and neuropsychological screener. Discourse data were used to classify participants testing within range of cognitive impairment using multiple machine learning algorithms and stability analysis for identifying reliably predictive features in an effort to maximize prediction accuracy. We hypothesized that a higher rate of pronoun use and lower scores on story recall would best classify older adults testing within range of cognitive impairment. The highest classification accuracy exploited a single variable in a remarkably intuitive way: using 66% story recall as a cutoff for cognitive impairment. Forcing this decision tree model to use more features or increasing its complexity did not improve accuracy. Permutation testing confirmed that the 77% accuracy and 0.18 Brier skill score achieved by the model were statistically significant (p < .00001). These results suggest that expository discourse tasks that place demands on executive functions, such as working memory, can be used to identify aging adults who test within range of cognitive impairment. Accurate representation of story elements in working memory is critical for coherent discourse. Our simple yet highly accurate predictive model of expository discourse provides a promising assessment for easy identification of cognitive impairment in older adults. https://doi.org/10.23641/asha.26543824.

Improving sub-seasonal extreme precipitation forecasts over China through a hybrid statistical-dynamical framework

Skillful and reliable sub-seasonal extreme precipitation forecasts are crucial for disaster prevention and mitigation. In this study, we introduce a hybrid statistical-dynamical framework to predict monthly maximum one-day precipitation (Rx1D) and monthly maximum five-day precipitation (Rx5D) over China from May to October. In the hybrid statistical-dynamical framework, the ECMWF forecasts of precipitation and boreal summer intraseasonal oscillation (BSISO) indices are used as predictors to establish calibration model and bridging models, separately. The calibration model and bridging models are then merged to generate probabilistic forecasts of Rx1D and Rx5D. Our results suggest that the bridging models show better performance in predicting Rx1D and Rx5D than calibration model in May, June, and July when the BSISO indices are used as predictors. The forecast skill of calibration model is higher compared to bridging models in August, September, and October. The BMA merged forecasts take advantage of both calibration model and bridging models, and can provide skilful and reliable forecasts for both Rx1D and Rx5D prediction. To have a more comprehensive assessment, we also evaluate the prediction skill of the occurrence of extreme precipitation events with exceedance probabilities of 50%, 20%, and 5% for both Rx1D and Rx5D. The Brier skill score of merged forecasts indicates that the hybrid statistical-dynamical framework can also provide skilful forecasts for the occurrence of extreme precipitation events greater than one-in-5-year return value of Rx1D (5Rx1D) and one-in-5-year return value of Rx5D (5Rx5D) in comparison to long-term climatology. These findings demonstrate the great potential of combining dynamical models and statistical models in improving sub-seasonal extreme precipitation forecasts.

Modelling beach volume changes caused by moderate and weak hydrodynamic conditions

The goals of this study were the calibration of the XBeach model for moderate and weak hydrodynamic conditions in 2D mode and the determination of the volumetric changes induced by them. An evaluation of model performance was made based on the Brier Skill Score (BSS), the visual match of the profile shape (VMS), the absolute volumetric change error (m3/m) and the relative volumetric change error (%). An analysis of accuracy in determining volumetric changes in the dune coast, with XBeach applied, provided an average absolute error in determination of volumetric changes of 4.0 m3/m for a significant storm and 1.5 m3/m for moderate and weak hydrodynamic conditions in a 2D model calibration process. Simulations of morphological changes caused by moderate and weak hydrodynamic conditions for seven timeframes classified into three groups have been run in 2D mode. Within a validation process, the average absolute error in the determination of volumetric changes in the beach was ranging from 0.64 m3/m to 2.42 m3/m depending on the group of hydrodynamic conditions. A high value of the correlation coefficient (R) between the measured and modelled balance of volumetric changes (m3/m) for all timeframes – 0.79 and for the ‘strongest’ group no. 3 – 0.91 were revealed. For the remaining groups, i.e., group no. 1 and no. 2 of hydrodynamic conditions, no such correlation was observed. Furthermore, the temporal analysis of the sum of beach volumes proved that a rising trend was observed over 4 months. Thus, moderate and weak hydrodynamic conditions dominated the accumulation within the area of study. XBeach reflected this trend well, producing a difference between the measured and modelled sum of volumes at the end of the timeframe of 7.27 m3/m, which is close to the volume determination error by the model.

Transformation of significant wave height and set-up due to entrained air bubbles effect in breaking waves

The transformation of wave height is of paramount significance in coastal engineering and the design of coastal structures. Considering the influence of air bubbles, this study devised an optimal dissipation model for accurately calculating changes in significant wave height (Hm0) and wave set-up for irregular waves undergoing breaking. Existing regular wave breaking models, which consider the effects of air bubbles, were adapted for direct application to irregular waves by deriving novel formulations. The proposed models leverage the probability of the fraction of broken waves. Hm0 was computed using the energy balance equation, while the wave set-up was calculated based on the momentum balance equation. A wide range of test scenarios, incorporating diverse scales (small and large) and experimental field data, was considered for validation. One of the proposed models, namely model-I (M-I), particularly demonstrated superior performance, manifesting lower error indices (P20), root-mean-square relative error (RMSRE), and Brier skill score (BSS) values in computing both Hm0 and wave set-up. Therefore, utilising M-I is strongly recommended for the precise estimation of Hm0 and set-up transformation.

Significant advancement in subseasonal-to-seasonal summer precipitation ensemble forecast skills in China mainland through an innovative hybrid CSG-UNET method

Reliable Subseasonal-to-Seasonal (S2S) forecasts of precipitation are critical for disaster prevention and mitigation. In this study, an innovative hybrid method CSG-UNET combining the UNET with the censored and shifted gamma distribution based ensemble model output statistic (CSG-EMOS), is proposed to calibrate the ensemble precipitation forecasts from ECMWF over the China mainland during boreal summer. Additional atmospheric variable forecasts and the data augmentation are also included to deal with the potential issues of low signal-to-noise ratio and relatively small sample sizes in traditional S2S precipitation forecast correction. The hybrid CSG-UNET exhibits a notable advantage over both individual UNET and CSG-EMOS in improving ensemble precipitation forecasts, simultaneously improving the forecast skills for lead times of 1–2 weeks and further extending the effective forecast timeliness to ∼4 weeks. Specifically, the climatology-based Brier Skill Scores are improved by ∼0.4 for the extreme precipitation forecasts almost throughout the whole timescale compared with the ECMWF. Feature importance analyze towards CSG-EMOS model indicates that the atmospheric factors make great contributions to the prediction skill with the increasing lead times. The CSG-UNET method is promising in subseasonal precipitation forecasts and could be applied to the routine forecast of other atmospheric and ocean phenomena in the future.

Role of the forcing sources in morphodynamic modelling of an embayed beach

Abstract. The sensitivity of a 2DH coastal area (XBeach) and a reduced-complexity (Q2Dmorfo) morphodynamic model to using different forcing sources is studied. The models are tested by simulating the morphodynamic response of an embayed beach in the NW Mediterranean over a 6-month period. Wave and sea-level forcing from in situ data, propagated buoy measurements, and hindcasts, as well as combinations of these different data sources, are used, and the outputs are compared to in situ bathymetric measurements. Results show that when the two models are calibrated with in situ measurements, they accurately reproduce the morphodynamic evolution with a “good” Brier skill score (BSS). The calibration process reduces the errors by 65 %–85 % compared with the default setting. The wave data propagated from the buoy also produce reliable morphodynamic simulations but with a slight decrease in the BSS. Conversely, when the models are forced with hindcast wave data, the mismatch between the modelled and observed beach evolution increases. This is attributed to a large extent to biased mean directions in hindcast waves. Interestingly, in this small tide site, the accuracy of the simulations hardly depends on the sea-level data source, and using filtered or non-filtered tides also yields similar results. These results have implications for long-term morphodynamic studies, like those needed to validate models for climate change projections, emphasizing the need to use accurate forcing sources such as those obtained by propagating buoy data.

An Empirical Model of the Occurrence Rate of Low Latitude Post‐Sunset Plasma Irregularities Derived From CHAMP and Swarm Magnetic Observations

AbstractThe prediction of post‐sunset equatorial plasma depletions (EPDs), often called ionospheric plasma bubbles, has remained a challenge for decades. In this study, we introduce the Ionospheric Bubble Probability (IBP) model, an empirical model to predict the occurrence probability of EPDs derived from 9 years of CHAMP and 9 years of Swarm magnetic field measurements. The model predicts the occurrence probability of EPDs for a given longitude, day of year, local time and solar activity, for the altitude range of about 350–510 km, and low geographic latitudes of ±45°. IBP has been found to successfully reconstruct the distribution of EPDs as reported in previous studies from independent data. IBP has been further evaluated using 1‐year of untrained data of the Ionospheric Bubble Index (IBI). IBI is a Level 2 product of the Swarm satellite mission used for EPD identification. The relative operating characteristics (ROC) curve shows positive excursion above the no‐skill line with Hanssen and Kuiper's Discriminant (H&amp;KSS) score of 0.52, 0.51, and 0.55 at threshold model output of 0.16 for Swarm A, B, and C satellites. Additionally, the reliability plots show proximity to the diagonal line with a decent Brier Skill Score (BSS) of 0.249, 0.210, and 0.267 for Swarm A, B, and C respectively at 15% climatological occurrence rate. These tests indicate that the model performs significantly better than a no‐skill forecast. The IBP model offers compelling glimpses into the future of EPD forecasting, thus demonstrating its potential to reliably predict EPD occurrences. The IBP model is publicly available.

Skillful seasonal prediction of the 2022–23 mega soil drought over the Yangtze River basin by combining dynamical climate prediction and copula analysis

An unprecedented soil moisture drought broke out over the Yangtze River basin (YRB) in the summer of 2022 and lasted until the spring of 2023, caused great economic losses and serious environmental issues. With the rapid onset and long-lasting duration, the mega soil drought challenges the current seasonal prediction capacity. Whether the state-of-the-art climate models provide skillful predictions of the onset, persistence and recovery of the 2022–23 mega soil drought needs to be assessed. Identified by the drought area percentage, here we show that the mega soil drought over the YRB started from July, 2022, reached the peak in August, and diminished in April, 2023. Combined with real-time predictions of monthly precipitation released by three climate models participating in the North American multi-model ensemble (NMME) project, we predict the monthly evolution of the 2022–23 soil drought through a joint distribution between precipitation and soil moisture established by the copula method. The results indicate that the NMME/copula prediction well reproduced the spatiotemporal evolution of the mega soil drought at 1 month lead. Using the climatological prediction that relies on the information of initial soil moisture conditions as the reference forecast, the Brier skill score (BSS) values for NMME multi-model ensemble are 0.26, 0.23 and 0.2 for the forecast lead times increased from 1 to 3 months during the entire soil drought period. Specifically, the BSS is 0.14 at 2 months lead during drought onset stage, and 0.26 at 3 months lead during persistence stage, while it is close to zero at all leads during the recovery stage. Our study implies that climate models have great potential in probabilistic seasonal prediction of the onset and persistency of mega soil drought through combining with the copula method.

Early forecasting of corn yield and price variations using satellite vegetation products

Several approaches based on process-based, statistical or economic models have been developed and are currently used by market analysts and governmental agencies to forecast agricultural yields and commodity prices at the national and global scales. However, these models require input data that are often difficult to obtain before crop harvest and their predictions are not always reliable. Satellite data products provide easily accessible sources of information on vegetation characteristics, potentially relevant for predicting crop yield and commodity prices at a large scale, before harvest. This study presents and evaluates several methods for forecasting variations in US corn yield and global corn prices using regional vegetation satellite products. The spatial dimension of maps of vegetation indices derived from satellite products in the US Corn Belt is first reduced either by their spatial means or by principal components derived using the Empirical Orthogonal Function (EOF) method. Mean values and EOF principal components are then used as explanatory variables in generalized LASSO linear models to forecast corn price and yield changes at different time horizons, before harvest. A cross-validation method was implemented to assess the model performances using Area under the ROC curve (AUC), the Brier Skill Score (BSS) and the Matthew Correlation Coefficient (MCC). The results revealed that EOF principal components derived from satellite-based gross primary production (GPP) in July could be used to predict variations in US corn yields, with an AUC of 0.97 (95% CI 0.92-1), a BSS of 0.75 and an MCC of 0.83. The results also showed that the July spatial averages of satellite GPP data obtained in the US Corn Belt could be used to forecast the forthcoming increase or decrease of global corn prices as evidenced by an AUC reaching up to 0.91, a BSS up to 0.5, and an MCC up to 0.67. Finally, we compared our forecasts of yield and price variations to those derived from the WASDE forecasting method currently implemented in the US. This comparison showed that our approach was at least as accurate and often more accurate than WASDE. This study demonstrates how satellite data can be used to forecast variations in crop yields and corn prices several months before harvest.

Prospective validation of a seizure diary forecasting falls short.

Recently, a deep learning artificial intelligence (AI) model forecasted seizure risk using retrospective seizure diaries with higher accuracy than random forecasts. The present study sought to prospectively evaluate the same algorithm. We recruited a prospective cohort of 46 people with epilepsy; 25 completed sufficient data entry for analysis (median = 5 months). We used the same AI method as in our prior study. Group-level and individual-level Brier Skill Scores (BSSs) compared random forecasts and simple moving average forecasts to the AI. The AI had an area under the receiver operating characteristic curve of .82. At the group level, the AI outperformed random forecasting (BSS = .53). At the individual level, AI outperformed random in 28% of cases. At the group and individual level, the moving average outperformed the AI. If pre-enrollment (nonverified) diaries (with presumed underreporting) were included, the AI significantly outperformed both comparators. Surveys showed most did not mind poor-quality LOW-RISK or HIGH-RISK forecasts, yet 91% wanted access to these forecasts. The previously developed AI forecasting tool did not outperform a very simple moving average forecasting in this prospective cohort, suggesting that the AI model should be replaced.

Generative Ensemble Deep Learning Severe Weather Prediction from a Deterministic Convection-Allowing Model

Abstract An ensemble postprocessing method is developed for the probabilistic prediction of severe weather (tornadoes, hail, and wind gusts) over the conterminous United States (CONUS). The method combines conditional generative adversarial networks (CGANs), a type of deep generative model, with a convolutional neural network (CNN) to postprocess convection-allowing model (CAM) forecasts. The CGANs are designed to create synthetic ensemble members from deterministic CAM forecasts, and their outputs are processed by the CNN to estimate the probability of severe weather. The method is tested using High-Resolution Rapid Refresh (HRRR) 1–24-h forecasts as inputs and Storm Prediction Center (SPC) severe weather reports as targets. The method produced skillful predictions with up to 20% Brier skill score (BSS) increases compared to other neural-network-based reference methods using a testing dataset of HRRR forecasts in 2021. For the evaluation of uncertainty quantification, the method is overconfident but produces meaningful ensemble spreads that can distinguish good and bad forecasts. The quality of CGAN outputs is also evaluated. Results show that the CGAN outputs behave similarly to a numerical ensemble; they preserved the intervariable correlations and the contribution of influential predictors as in the original HRRR forecasts. This work provides a novel approach to postprocess CAM output using neural networks that can be applied to severe weather prediction. Significance Statement We use a new machine learning (ML) technique to generate probabilistic forecasts of convective weather hazards, such as tornadoes and hailstorms, with the output from high-resolution numerical weather model forecasts. The new ML system generates an ensemble of synthetic forecast fields from a single forecast, which are then used to train ML models for convective hazard prediction. Using this ML-generated ensemble for training leads to improvements of 10%–20% in severe weather forecast skills compared to using other ML algorithms that use only output from the single forecast. This work is unique in that it explores the use of ML methods for producing synthetic forecasts of convective storm events and using these to train ML systems for high-impact convective weather prediction.

A comparison of two statistical postprocessing methods for heavy‐precipitation forecasts over India during the summer monsoon

AbstractAccurate ensemble forecasts of heavy precipitation in India are vital for many applications and essential for early warning of damaging flood events, especially during the monsoon season. In this study we investigate to what extent Quantile Mapping (QM) and Ensemble Model Output Statistics (EMOS) statistical postprocessing reduce errors in precipitation ensemble forecasts over India, in particular for heavy precipitation. Both methods are applied to day‐1 forecasts at 12‐km resolution from the 23‐member National Centre for Medium Range Weather Forecasting (NCMRWF) global ensemble prediction system (NEPS‐G). By construction, QM leads to distributions close to the observed ones, while EMOS optimizes the ensemble spread, and it is not a priori clear which is better suited for practical applications. The methods are therefore compared with respect to several key aspects of the forecasts: local distributions, ensemble spread, and skill for forecasting precipitation amounts and the exceedance of heavy precipitation thresholds. The evaluation includes rank histograms, Continuous Ranked Probability Skill Scores (CRPSS), Brier Skill Scores (BSS), reliability diagrams, and receiver operating characteristic. EMOS performs best not only with respect to correcting under‐ or overdispersive ensembles, but also in terms of forecast skill for precipitation amounts and heavy precipitation events, with positive CRPSS and BSS in most regions (both up to about 0.4 in some areas), while QM in many regions performs worse than the raw forecast. QM performs best with respect to the overall local precipitation distributions. Which aspects of the forecasts are most relevant depends to some extent on how the forecasts are used. If the main criteria are the correction of under‐ or overdispersion, forecast reliability, match between the forecasted distribution for individual days and observations (CRPSS), and the skill in forecasting heavy‐precipitation events (BSS), then EMOS is the better choice for postprocessing NEPS‐G forecasts for short lead times.

Development and validation of a multivariable risk prediction model for hepatic steatosis in patients with HIV infection.

Early detection of hepatic steatosis in people with HIV (PWH) could prevent progression and inflammation. The aim was to develop and validate a multivariable risk prediction model for hepatic steatosis in German PWH. In this cohort study, 282 PWH were prospectively enrolled, and hepatic steatosis was defined via controlled attenuation parameter (CAP; ≥275 dB/m) using vibration-controlled transient elastography. Three multivariable logistic regression models were conducted. Missing values were imputed with multiple imputation. Cut-offs were derived based on Youden-Indices. Performance was assessed via discriminatory and calibrative ability and accuracy via Brier Skill Score. Sensitivity, specificity, and predictive values were calculated. Internal validation was performed via bootstrapping. The prevalence of hepatic steatosis was 35.3% (100/282). Univariate analyses revealed associations with age, waist circumference, BMI, hypertension, hyperlipidemia and gamma-gt. In multivariable analyses, male sex [odds ratio (OR) 2.07, 95% confidence interval (CI) 1.42-3.00, P = 0.001] and BMI (OR 1.27, 95% CI 1.18-1.36, P < 0.001) were identified as independent predictors of hepatic steatosis. The naive and optimism-corrected c -statistic of 79% showed a good discriminatory ability, the calibration was well with a slight tendency for overestimation for predicted probabilities above 70%. At the cutoff of 1.95, the specificity was 71% and the negative-predictive value 82.3%. Twenty-seven percent of the 282 patients would be misclassified, 17% as false positives and 10% as false negatives. The developed prediction model contributes to the lack of validated noninvasive tools to predict hepatic steatosis in people with HIV. Future studies should include more candidate predictors and externally validate the model.

Predictability of Rainfall over Equatorial East Africa in the ECMWF Ensemble Reforecasts on short to medium-range time scales

Abstract Despite the enormous potential of precipitation forecasts to save lives and property in Africa, low skill has limited their uptake. To assess the skill and improve the performance of the forecast, validation and postprocessing should continuously be carried out. Here, we evaluate the quality of reforecasts from the European Centre for Medium-Range Weather Forecasts over Equatorial East Africa (EEA) against satellite and rain gauge observations for the period 2001–2018. 24-hour rainfall accumulations are analysed from short to medium-range time scales. Additionally, 48- and 120-hour rainfall accumulations were also assessed. The skill was assessed using an extended probabilistic climatology (EPC) derived from the observations. Results show that the reforecasts overestimate rainfall, especially during the rain seasons and over high-altitude areas. However, there is potential of skill in the raw forecasts up to 14-day lead-time. There is an improvement of up to 30% in Brier score/continuous rank probability score relative to EPC in most areas, especially the higher-altitude regions, decreasing with lead-time. Aggregating the reforecasts enhances the skill further, likely due to a reduction in timing mismatches. However, for some regions of the study domain, the predictive performance is worse than EPC, mainly due to biases. Postprocessing the reforecasts using isotonic distributional regression considerably improves skill, increasing the number of grid-points with positive Brier skill score (continuous rank probability score) by an average of 81% (91%) for lead-times 1–14 days ahead. Overall, the study highlights the potential of the reforecasts, the spatio-temporal variation in skill and benefit of postprocessing in EEA.

Can Machine Learning Models Be a Suitable Tool for Predicting Central European Cold Winter Weather on Subseasonal to Seasonal Time Scales?

Abstract Skillful weather prediction on subseasonal to seasonal time scales is crucial for many socioeconomic ventures. But forecasting, especially extremes, on these time scales is very challenging because the information from initial conditions is gradually lost. Therefore, data-driven methods are discussed as an alternative to numerical weather prediction models. Here, quantile regression forests (QRFs) and random forest classifiers (RFCs) are used for probabilistic forecasting of central European mean wintertime 2-m temperatures and cold wave days at lead times of 14, 21, and 28 days. ERA5 reanalysis meteorological predictors are used as input data for the machine learning models. For the winters of 2000/01–2019/20, the predictions are compared with a climatological ensemble obtained from E-OBS observational data. The evaluation is performed as full distribution predictions for continuous values using the continuous ranked probability skill score and as binary categorical forecasts using the Brier skill score. We find skill at lead times up to 28 days in the 20-winter mean and for individual winters. Case studies show that all used machine learning models are able to learn patterns in the data beyond climatology. A more detailed analysis using Shapley additive explanations suggests that both random forest (RF)-based models are able to learn physically known relationships in the data. This underlines that RF-based data-driven models can be a suitable tool for forecasting central European mean wintertime 2-m temperatures and the occurrence of cold wave days. Significance Statement Because of the chaotic nature of weather, it is very complicated to make predictions with traditional numerical methods 2–4 weeks in advance. Therefore, we use alternative, interpretable methods that “learn” to find statistically relevant patterns in meteorological data that can be used for forecasting central European mean surface wintertime temperatures and cold wave days. These methods are part of the so-called machine learning methods that do not rely on the traditional numerical equations anymore. We test our methods for 20 winters between 2000/01 and 2019/20 against a static weather prediction consisting of the past 30 winters. For single winters and in a mean over the 20 predicted winters, we find improved predictions up to 4 weeks in advance.

Multiweek tropical cyclone prediction for the Southern Hemisphere in ACCESS‐S2: Maintaining operational skill and continuity of service

AbstractThe skill of subseasonal (multiweek) forecasts of tropical cyclone (TC) occurrence over the Southern Hemisphere is examined in the Australian Bureau of Meteorology's (BoM) multiweek to seasonal prediction system, ACCESS‐S2. Relative to its predecessor, ACCESS‐S1, ACCESS‐S2 shows improved biases in spatial TC frequency in the South Pacific and Southwest Indian Ocean. However, there is no improvement to the known negative bias in TC frequency off the coast of NW Australia. The ability of ACCESS‐S2 to provide probabilistic forecasts of TC occurrence for the Southern Hemisphere on multiweek timescales is examined using reliability measures and Brier skill scores. For the period November–February 1990–2012, both ACCESS‐S1 and ACCESS‐S2 show positive skill relative to climatology for calibrated forecasts out to week 5. However, the skill of ACCESS‐S2 is slightly reduced compared to ACCESS‐S1 at all lead times, which may be due to the fewer number of ensemble members available. For the full ACCESS‐S2 hindcast period, November–April 1981–2018, ACCESS‐S2 again shows positive skill of calibrated forecasts over climatology out to week 5. For weeks 1–2, skill is reduced compared to the shorter 1990–2012 period; whereas it is marginally improved for longer lead times (weeks 3–5). Use of lagged ensembles, an alternative linear regression calibration, as well as removing weaker model TCs were examined to potentially improve the skill of ACCESS‐S2 forecasts; however, none of these methods were able to significantly increase skill at all lead times. Continued use of the original calibration method is therefore recommended in order to retain skill and continuity of service of the BoM operational and public multiweek TC forecasts.

Evaluation of Probabilistic Forecasts of Extreme Cold Events in S2S Models

The probabilistic prediction skill of the weekly forecasts of extreme cold events (ECE) is illustrated and measured in the form of the Brier Skill Score (BSS) and the area under Relative Operating Characteristics (ROC) curves based on the subseasonal-to-seasonal (S2S) prediction project database. The ROC scores show that six S2S models have the good potential predictability skill required for use in ECE probabilistic forecasts, and they were more useful than climatologic probabilistic models in creating forecasts of about 3–4 weeks in length. However, the BSS results show that the actual prediction skill of six models used in ECE probabilistic forecasts are different. The ECMWF model has a good performance, and its actual probabilistic prediction skill of ECE for forecasts of about 3–4 weeks in length was higher than those of climatology, which operates close to its potential predictability. The actual probabilistic prediction skill of the NCEP model for ECE was only about 2 weeks over the extra-tropics, and no skill was recorded over the tropics given its bad reliability, especially over the tropics. BoM, JMA, and CNRM models only have a 1-week actual prediction skill over the Northern Hemisphere extra-tropics, and they have no skill over the rest of the world’s land area. The CNR-ISAC model has a 1-week actual prediction skill over the extra-tropics and about 4 weeks over the tropics. There is still much room for improvement in the prediction ability of models used for ECE. MJO in tropical regions has an important influence on the probabilistic prediction skill of ECE required at middle and high latitudes. When there is an MJO in the initial conditions, the potential predictability and actual prediction skill of ECE probabilistic forecasts over North America in the 3rd week and over Europe in the 3rd–4th weeks are higher than those without MJO.

Hybrid Dynamical–Statistical Forecasts of the Risk of Rainfall in Southeast Asia Dependent on Equatorial Waves

Abstract Equatorial waves are a major driver of widespread convection in Southeast Asia and the tropics more widely, a region in which accurate heavy rainfall forecasts are still a challenge. Conditioning rainfall over land on local equatorial wave phases finds that heavy rainfall can be between 2 and 4 times more likely to occur in Indonesia, Malaysia, Vietnam, and the Philippines. Equatorial waves are identified in a global numerical weather prediction ensemble forecast [Met Office Global and Regional Ensemble Prediction System (MOGREPS-G)]. Skill in the ensemble forecast of wave activity is highly dependent on region and time of year, although generally forecasts of equatorial Rossby waves and westward-moving mixed Rossby–gravity waves are substantially more skillful than for the eastward-moving Kelvin wave. The observed statistical relationship between wave phases and rainfall is combined with ensemble forecasts of dynamical wave fields to construct hybrid dynamical–statistical forecasts of rainfall probability using a Bayesian approach. The Brier skill score is used to assess the skill of forecasts of rainfall probability. Skill in the hybrid forecasts can exceed that of probabilistic rainfall forecasts taken directly from MOGREPS-G and can be linked to both the skill in forecasts of wave activity and the relationship between equatorial waves and heavy rainfall in the relevant region. The results show that there is potential for improvements of forecasts of high-impact weather using this method as forecasts of large-scale waves improve.

Addressing health disparities using multiply imputed injury surveillance data

BackgroundAssessing disparities in injury is crucial for injury prevention and for evaluating injury prevention strategies, but efforts have been hampered by missing data. This study aimed to show the utility and reliability of the injury surveillance system as a trustworthy resource for examining disparities by generating multiple imputed companion datasets.MethodsWe employed data from the National Electronic Injury Surveillance System-All Injury Program (NEISS-AIP) for the period 2014–2018. A comprehensive simulation study was conducted to identify the appropriate strategy for addressing missing data limitations in NEISS-AIP. To evaluate the imputation performance more quantitatively, a new method based on Brier Skill Score (BSS) was developed to assess the accuracy of predictions by different approaches. We selected the multiple imputations by fully conditional specification (FCS MI) to generate the imputed companion data to NEISS-AIP 2014–2018. We further assessed health disparities systematically in nonfatal assault injuries treated in U.S. hospital emergency departments (EDs) by race and ethnicity, location of injury and sex.ResultsWe found for the first time that significantly higher age-adjusted nonfatal assault injury rates for ED visits per 100,000 population occurred among non-Hispanic Black persons (1306.8, 95% Confidence Interval [CI]: 660.1 – 1953.5), in public settings (286.3, 95% CI: 183.2 – 389.4) and for males (603.5, 95% CI: 409.4 – 797.5). We also observed similar trends in age-adjusted rates (AARs) by different subgroups for non-Hispanic Black persons, injuries occurring in public settings, and for males: AARs of nonfatal assault injury increased significantly from 2014 through 2017, then declined significantly in 2018.ConclusionsNonfatal assault injury imposes significant health care costs and productivity losses for millions of people each year. This study is the first to specifically look at health disparities in nonfatal assault injuries using multiply imputed companion data. Understanding how disparities differ by various groups may lead to the development of more effective initiatives to prevent such injury.

Deep Learning Forecast Uncertainty for Precipitation over the Western United States

Abstract Reliably quantifying uncertainty in precipitation forecasts remains a critical challenge. This work examines the application of a deep learning (DL) architecture, Unet, for postprocessing deterministic numerical weather predictions of precipitation to improve their skills and for deriving forecast uncertainty. Daily accumulated 0–4-day precipitation forecasts are generated from a 34-yr reforecast based on the West Weather Research and Forecasting (West-WRF) mesoscale model, developed by the Center for Western Weather and Water Extremes. The Unet learns the distributional parameters associated with a censored, shifted gamma distribution. In addition, the DL framework is tested against state-of-the-art benchmark methods, including an analog ensemble, nonhomogeneous regression, and mixed-type meta-Gaussian distribution. These methods are evaluated over four years of data and the western United States. The Unet outperforms the benchmark methods at all lead times as measured by continuous ranked probability and Brier skill scores. The Unet also produces a reliable estimation of forecast uncertainty, as measured by binned spread–skill relationship diagrams. Additionally, the Unet has the best performance for extreme events (i.e., the 95th and 99th percentiles of the distribution) and for these cases, its performance improves as more training data are available. Significance Statement Accurate precipitation forecasts are critical for social and economic sectors. They also play an important role in our daily activity planning. The objective of this research is to investigate how to use a deep learning architecture to postprocess high-resolution (4 km) precipitation forecasts and generate accurate and reliable forecasts with quantified uncertainty. The proposed approach performs well with extreme cases and its performance improves as more data are available in training.