Prediction Limits Research Articles

Prediction error growth in a more realistic atmospheric toy model with three spatiotemporal scales

Abstract. This article studies the growth of the prediction error over lead time in a schematic model of atmospheric transport. Inspired by the Lorenz (2005) system, we mimic an atmospheric variable in one dimension, which can be decomposed into three spatiotemporal scales. We identify parameter values that provide spatiotemporal scaling and chaotic behavior. Instead of exponential growth of the forecast error over time, we observe a more complex behavior. We test a power law and the quadratic hypothesis for the scale-dependent error growth. The power law is valid for the first days of the growth, and with an included saturation effect, we extend its validity to the entire period of growth. The theory explaining the parameters of the power law is confirmed. Although the quadratic hypothesis cannot be completely rejected and could serve as a first guess, the hypothesis's parameters are not theoretically justifiable in the model. In addition, we study the initial error growth for the ECMWF forecast system (500 hPa geopotential height) over the 1986 to 2011 period. For these data, it is impossible to assess which of the error growth descriptions is more appropriate, but the extended power law, which is theoretically substantiated and valid for the Lorenz system, provides an excellent fit to the average initial error growth of the ECMWF forecast system. Fitting the parameters, we conclude that there is an intrinsic limit of predictability after 22 d.

Investigating decadal variations of the seasonal predictability limit of sea surface temperature in the tropical Pacific

El Niño and the Southern Oscillation (ENSO) have a worldwide impact on seasonal to yearly climate. However, there are decadal variations in the seasonal prediction skill of ENSO in dynamical and statistical models; in particular, ENSO prediction skill has declined since 2000. The shortcomings of models mean that it is very important to study ENSO seasonal predictability and its decadal variation using observational/reanalysis data. Here we quantitatively estimate the seasonal predictability limit (PL) of ENSO from 1900 to 2015 using Nonlinear local Lyapunov exponent (NLLE) theory with an observational/reanalysis dataset and explore its decadal variations. The mean PL of sea surface temperature (SST) is high in the central/eastern tropical Pacific and low in the western tropical Pacific, reaching 12–15 and 7–8 months, respectively. The PL in the tropical Pacific varies on a decadal timescale, with an interdecadal standard deviation of up to 2 months in the central tropical Pacific that has similar spatial structure to the mean PL. Taking the PL of SST in the Niño 3.4 region as representative of the PL in the central/eastern tropical Pacific, there are clearly higher values in the 1900s, mid-1930s, mid-1960s, and mid-1990s, and lower values in the 1920s, mid-1940s, and mid-2010s. Meanwhile, the PL of SST in the Niño 6 region—whose average value is 7 months—is in good agreement with the PL of most regions in the western tropical Pacific, with higher values in the 1910s, 1940s, and 1980s and lower values in the 1930s, 1950s, and mid-1990s. In the framework of NLLE theory, the PL is determined by the error growth rate (representing the dissipation rate of the predictable signal) and the saturation value of relative error (representing predictable signal intensity). We reveal that the spatial structure of the mean PL in the tropical Pacific is determined mainly by the error growth rate. The decadal variability of PL is affected more by the variation of the saturation value of relative error in the equatorial Pacific, whereas the error growth rate cannot be ignored in the PL of some regions. As an important source of predictability in ENSO dynamics, the relationship between warm water volume and SST in the Niño 3.4 region has a critical role in the decadal variability of PL in the tropical Pacific through the error growth rate and saturation value of relative error. This strong relationship reduces the error growth rate in the initial period and increases the saturated relative error, contributing to the high PL.

The Interday Repeatability of Parameters for the Assessment of Progressive Disease in Subjects With Less Advanced Keratoconus

To evaluate the interday repeatability in the measurement of parameters used for the detection of progression of keratoconus by prediction limits (PL) for single measurements, and the repeatability coefficient (RC) for the mean of replicate measurements. Prospective reliability analysis for cases and control eyes. Twenty-five eyes in 25 subjects with KC and 25 eyes in 25 healthy controls were included. Four consecutive measurements were made, 3days apart, with a Pentacam HR tomographic instrument (denoted the Pentacam) and a Nidek ARK 560-A auto-keratometer (denoted the keratometer). Main outcome measures were the intra- and interday RC of parameters used in the detection of progression of keratoconus. The most repeatable parameter obtained with the Pentacam was the curvature power of the central flat meridian (K1, 0.44 D [RC], -0.55 to 0.60 diopter [D] [PL]), followed by the central steep meridian (K2, 0.72 D [RC], -0.90 to 0.94 D [PL]). The interday repeatability of K1 and K2 was similar when using the keratometer (K1, 0.32 D [RC], -0.66 to 0.57 D [PL], K2, 0.93 D [RC], -1.36 to 1.08 D [PL]). The interday repeatability of the curvature power of the steepest point (Kmax, 0.84 D [RC], -0.90 to 1.11 D [PL]) would benefit from being stratified: RC= 0.44 D and PL= -0.49 to 0.67 D for Kmax < 49.0 D, and RC= 1.08 D and PL= -1.19 to 1.42 D for Kmax ≥ 49.0 D. The interday repeatability of measurements, single or replicate, in subjects with keratoconus should be considered when diagnosing progressive disease. K1 exhibited the best intraday repeatability. Kmax benefits from being stratified according to disease severity.

RainNet v1.0: a convolutional neural network for radar-based precipitation nowcasting

Abstract. In this study, we present RainNet, a deep convolutional neural network for radar-based precipitation nowcasting. Its design was inspired by the U-Net and SegNet families of deep learning models, which were originally designed for binary segmentation tasks. RainNet was trained to predict continuous precipitation intensities at a lead time of 5 min, using several years of quality-controlled weather radar composites provided by the German Weather Service (DWD). That data set covers Germany with a spatial domain of 900 km×900 km and has a resolution of 1 km in space and 5 min in time. Independent verification experiments were carried out on 11 summer precipitation events from 2016 to 2017. In order to achieve a lead time of 1 h, a recursive approach was implemented by using RainNet predictions at 5 min lead times as model inputs for longer lead times. In the verification experiments, trivial Eulerian persistence and a conventional model based on optical flow served as benchmarks. The latter is available in the rainymotion library and had previously been shown to outperform DWD's operational nowcasting model for the same set of verification events. RainNet significantly outperforms the benchmark models at all lead times up to 60 min for the routine verification metrics mean absolute error (MAE) and the critical success index (CSI) at intensity thresholds of 0.125, 1, and 5 mm h−1. However, rainymotion turned out to be superior in predicting the exceedance of higher intensity thresholds (here 10 and 15 mm h−1). The limited ability of RainNet to predict heavy rainfall intensities is an undesirable property which we attribute to a high level of spatial smoothing introduced by the model. At a lead time of 5 min, an analysis of power spectral density confirmed a significant loss of spectral power at length scales of 16 km and below. Obviously, RainNet had learned an optimal level of smoothing to produce a nowcast at 5 min lead time. In that sense, the loss of spectral power at small scales is informative, too, as it reflects the limits of predictability as a function of spatial scale. Beyond the lead time of 5 min, however, the increasing level of smoothing is a mere artifact – an analogue to numerical diffusion – that is not a property of RainNet itself but of its recursive application. In the context of early warning, the smoothing is particularly unfavorable since pronounced features of intense precipitation tend to get lost over longer lead times. Hence, we propose several options to address this issue in prospective research, including an adjustment of the loss function for model training, model training for longer lead times, and the prediction of threshold exceedance in terms of a binary segmentation task. Furthermore, we suggest additional input data that could help to better identify situations with imminent precipitation dynamics. The model code, pretrained weights, and training data are provided in open repositories as an input for such future studies.

A field-based characterization of conductivity in areas of minimal alteration: A case example in the Cascades of northwestern United States

The concentration of salts in streams is increasing world-wide making freshwater a declining resource. Developing thresholds for freshwater with low specific conductivity (SC), a measure of dissolved ions in water, may protect high quality resources that are refugia for aquatic life and that dilute downstream waters. In this case example, methods are illustrated for estimating protective levels for streams with low SC. The Cascades in the Pacific Northwest of the United States of America was selected for the case study because a geophysical model indicated that the SC of freshwater streams was likely to be very low. Also, there was an insufficient range in the SC data to accurately derive a criterion using the 2011, US Environmental Protection Agency field-based extirpation concentration distribution method. Instead, background and a regression model was used to estimate chronic and acute SC levels that could extirpate 5% of benthic invertebrate genera. Background SC was estimated at the 25th centile (33μS/cm) of the measured data and used as the independent variable in a least squares empirical background-to-criteria (B-C) model. Because no comparison could be made with effect levels estimated from a paired SC and biological data set from the Cascades, the lower 50% prediction limit (PL) was identified as an example chronic water quality criterion (97μS/cm). The maximum exposure threshold was estimated at the 90th centile SC of streams meeting the chronic SC level. The example acute SC level was 190μS/cm. Because paired aquatic life and SC data are often sparse, the B-C method is useful for developing SC criteria for other systems with limited data.

Six-Food Elimination Diet and Topical Steroids are Effective for Eosinophilic Esophagitis: A Meta-Regression.

Topical corticosteroids or six-food elimination diet is recommended as initial therapy for eosinophilic esophagitis (EoE). We aimed to summarize published manuscripts that report outcomes of these therapies for EoE. We performed a systematic review in MEDLINE, Web of Science, and Embase of published manuscripts describing topical fluticasone, topical budesonide, and six-food elimination diet as therapies for EoE. We conducted meta-analysis of symptom improvement and the change in peak mucosal eosinophil count, with heterogeneity between studies examined with meta-regression analysis. Systematic review yielded 51 articles that met inclusion criteria. Summary histologic response rates were 68.3% [95% prediction limits (PL) 16.2-96.0%] for fluticasone, 76.8% (95% PL 36.1-95.1%) for budesonide, and 69.0% (95% PL 31.9-91.4%) for six-food elimination diet. Corresponding decreases in eosinophil counts were 37.8 (95% PL 19.0-56.7), 62.5 (95% PL 125.6 to -0.67, and 44.6 (95% PL 26.5-62.7), respectively. Symptom response rates were 82.3% (95% PL 68.1-91.1%), 87.9% (95% PL 42.7-98.6%), and 87.3% (95% PL 64.5-96.3%), respectively. Meta-regression analyses decreased the initially large estimate of residual heterogeneity and suggested differences in histologic response rate associated with study populations' baseline eosinophil count and age. The literature describing topical corticosteroids and six-food elimination diet consists of small studies with diverse methods and population characteristics. Meta-analysis with meta-regression shows initial histologic and symptomatic response rates on the same order of magnitude for topical corticosteroids and six-food elimination diet, but heterogeneity of study designs prevents direct comparison of modalities.

Five-year forecasts of the Visual Field Index (VFI) with binocular and monocular visual fields

In clinical care, visual field (VF) damage is assessed using monocular VF testing, yet patients perceive the world binocularly. This study was conducted to compare 5-year forecasts for the Visual Field Index (VFI) generated from series of binocular and monocular VFs. Series of ten consecutive VFs (Humphrey 24-2 Full-threshold) spanning on average 3.7 (SD: ±0.8) years from 60 eyes of 30 glaucomatous patients were retrospectively examined. The VFs of both eyes were merged to produce the integrated VF and its VFI score (Binocular VFI) was estimated. Forecasts of binocular and monocular VFIs were calculated for each patient by projecting the fitted linear regression 5 years ahead from the last VF following the method on the Humphrey Guided Progression Analysis (GPA) print-out. The precisions of the forecasts were calculated as the width of the 95% prediction limit (PL). The mean 5 year forecast for binocular VFIs was 92% (SD: 11%), which was significantly higher than forecasts from right and left eyes (79% [SD: 19%] and 82% [SD: 16%] respectively; P < 0.05). The width of the 95% PL for 5-year predictions with monocular VFIs (mean right eye: 29% [SD: 19%] and mean left eye: 27% [SD: 16%]) were significantly larger than that of the binocular VFI (mean: 12% [SD: 7%]; P < 0.05). Five year forecasted VFI values using binocular measures return significantly better values, and can be made with greater confidence than those based on monocular measures. In turn, forecasts of a patient's binocular VFI might be more relevant to estimating the patient's future functional VF.

Empirical prediction limit estimation methods for feed-forward neural networks

Feed-forward neural networks (FFNs) have gained a lot of interest in the last decade as empirical models for their powerful representational capacity, non-parametric nature and multivariate characteristics. While these neural network models focus primarily on accurate prediction of output values, often outperforming their statistical counterparts in dealing with sparse date sets, they usually do not provide any information regarding the confidence with which they make these predictions. Since prediction limits (PLs) indicate the extent to which one can rely on predictions for making future decisions, it is of paramount importance to estimate these limits. Two empirical PL estimation methods for FFNs are presented. The two methods differ in one fundamental aspect: the method employed for modeling the properties of the neural network model residuals. While one method uses a local approximation scheme, the other utilizes a global approximation scheme. Simulation results reveal that both methods have their relative strengths and weaknesses.

Long-range weather forecasts through numerical and empirical methods

Abstract There has been much improvement in numerical weather prediction since L.F. Richardson (1922, Weather Prediction by Numerical Process , Cambridge University Press, Cambridge, p. 236) wrote his famous book. NWP has primarily been successful in improving day-by-day forecasts starting from an observed detailed Initial Condition (IC) out to about a week. The purpose of this paper is to discuss first the state of the art in long-range NWP by presenting results of a new large numerical experiment (named DERF90; from Dynamical Extended Range Forecasting in 1990 out to 90 days) conducted at the National Meteorological Center (NMC) during the summer and autumn of 1990 (Section 2). One hundred and twenty eight 90-day global forecasts were made from successive daily initial conditions (IC), thus giving us ample opportunity to assess skill of forecasts at lead times beyond 1 week. We then move on to define the notion of a limit of predictability (LOP), and following a procedure by Lorenz (1982), give a numerical estimate of the LOP using the DERF90 data set. We then produce a list of reasons, as to why this estimate (LOP = 18 days) should not be taken too literally. In particular, we argue that the LOP varies as a function of the flow itself, and it would be (much) larger if we had, as we will ultimately, a coupled ocean-atmosphere model for making long-lead forecasts. Last, but not least, we present results of empirical forecasts that point to modest but significant skill well beyond the traditional LOP (a few weeks). A specific recent example of empirical forecasting is discussed. Through Canonical Correlation Analysis (CCA), experimental forecasts are being made for the United States surface temperatures at lead times of several seasons. While modest, the skill is significant in that it defies the existence or a 3-week LOP, and so demonstrates the potential for model improvements.

LIMITS TO PREDICTION. By R. B. McKern and G. C. Lowenthal

LIMITS TO PREDICTION. By R. B. McKern and G. C. Lowenthal