Realistic Confidence Intervals Research Articles

Threshold models improve estimates of molt parameters in datasets with small sample sizes

Abstract The timing of events in birds’ annual cycles is important to understanding life history evolution and response to global climate change. Molt timing is often measured as an index of the sum of grown feather proportion or mass within the primary flight feathers. The distribution of these molt data over time has proven difficult to model with standard linear models. The parameters of interest are at change points in model fit over time, and so least-squares regression models that assume molt is linear violate the assumption of even variance. This has led to the introduction of other nonparametric models to estimate molt parameters. Hinge models directly estimate changes in model fit and have been used in many systems to find change points in data distributions. Here, we apply a hinge model to molt timing, through the introduction of a double-hinge (DH) threshold model. We then examine its performance in comparison to current models using simulated and empirical data. Our results suggest that the Underhill–Zucchini (UZ) and Pimm models perform well under many circumstances and appear to outperform the DH model in datasets with high variance. The DH model outperforms the UZ model at low sample sizes of birds in active molt and shorter molt durations and provides more realistic confidence intervals at smaller sample sizes. The DH model provides a novel addition to the toolkit for estimating molt phenology, expanding the conditions under which molt can accurately be estimated.

Long-term Trend Analysis of Extreme Coastal Sea Levels with Changepoint Detection

Abstract Sea level rise can bring disastrous outcomes to people living in coastal regions by increasing flood risk or inducing stronger storm surges. We study long-term linear trends in monthly maximum sea levels by applying extreme value methods. The monthly maximum sea levels are extracted from multiple tide gauges around the coastal regions of the world over a period of as long as 169 years. Due to instrument changes, location changes, earthquakes, land reclamation, dredging, etc., the sea level data could contain inhomogeneous shifts in their means, which can substantially impact trend estimates if ignored. To rigorously quantify the long-term linear trends and return levels for the monthly maximum sea level data, we use a genetic algorithm to estimate the number and times of changepoints in the data. As strong periodicity and temporal correlation are pertinent to the data, bootstrap techniques are used to obtain more realistic confidence intervals to the estimated trends and return levels. We find that the consideration of changepoints changed the estimated linear trends of 89 tide gauges (approximately 30% of tide gauges considered) by more than 20cmcentury-1. Our results are summarized in maps with estimated extreme sea level trends and 50-year return levels.

The discernible and hidden effects of clonality on the genotypic and genetic states of populations: Improving our estimation of clonal rates.

Partial clonality is widespread across the tree of life, but most population genetic models are designed for exclusively clonal or sexual organisms. This gap hampers our understanding of the influence of clonality on evolutionary trajectories and the interpretation of population genetic data. We performed forward simulations of diploid populations at increasing rates of clonality (c), analysed their relationships with genotypic (clonal richness, R, and distribution of clonal sizes, Pareto β) and genetic (FIS and linkage disequilibrium) indices, and tested predictions of c from population genetic data through supervised machine learning. Two complementary behaviours emerged from the probability distributions of genotypic and genetic indices with increasing c. While the impact of c on R and Pareto β was easily described by simple mathematical equations, its effects on genetic indices were noticeable only at the highest levels (c>0.95). Consequently, genotypic indices allowed reliable estimates of c, while genetic descriptors led to poorer performances when c<0.95. These results provide clear baseline expectations for genotypic and genetic diversity and dynamics under partial clonality. Worryingly, however, the use of realistic sample sizes to acquire empirical data systematically led to gross underestimates (often of one to two orders of magnitude) of c, suggesting that many interpretations hitherto proposed in the literature, mostly based on genotypic richness, should be reappraised. We propose future avenues to derive realistic confidence intervals for c and show that, although still approximate, a supervised learning method would greatly improve the estimation of c from population genetic data.

Chronostratigraphy and age modeling of Pleistocene drill cores from the Olduvai Basin, Tanzania (Olduvai Gorge Coring Project)

The Olduvai Gorge Coring Project drilled a total of 611.72 m of core (575.48 m recovered) of mostly fluvio-lacustrine and fan-delta volcaniclastic Pleistocene strata at three sites in the Olduvai Basin, Tanzania, in 2014. We have developed a chronostratigraphic framework for three of the cores based on 40Ar/39Ar dating of core and outcrop volcanic and volcaniclastic units, core paleomagnetic stratigraphy, and tephrochemical correlation between cores and from core to outcrop. This framework is then used to constrain Bayesian stratigraphic age models which permit age estimates for desired core levels with realistic confidence intervals. The age models reveal that the deepest core level reached at 245 mbs is ~2.24 Ma, ~210 kyr older than the oldest strata exposed at Olduvai Gorge. Strata net accretion rates in this early phase of basin history were relatively rapid (57–69 cm/kyr), but decreased within ~250 kyr to ~15 cm/kyr in Lower Bed I. Rates rebounded partially in Upper Bed I, but subsequently declined to <10 cm/kyr by Middle to Upper Pleistocene. The age models also provide new estimates for the basal contacts of upper Olduvai Gorge stratigraphic units that have been previously difficult to calibrate: Bed III at 1.14 ± 0.05 (95% confidence interval), Bed IV at 0.93 ± 0.08, Masek at 0.82 ± 0.06, and Ndutu at 0.50 ± 0.04 Ma. Finally, based on recently acquired seismic imaging identifying basement another 135 m beneath the bottom of the deepest core, extrapolation of net accretion rates suggests that sedimentation began at this site in the Olduvai Basin at ~2.5 Ma.

The discernible and hidden effects of clonality on the genotypic and genetic states of populations: improving our estimation of clonal rates

Partial clonality is widespread across the tree of life, but most population genetics models are designed for exclusively clonal or sexual organisms. This gap hampers our understanding of the influence of clonality on evolutionary trajectories and the interpretation of population genetics data. We performed forward simulations of diploid populations at increasing rates of clonality (c), analysed their relationships with genotypic (clonal richness, R, and distribution of clonal sizes, Pareto \beta) and genetic (FIS and linkage disequilibrium) indices, and tested predictions of c from population genetics data through supervised machine learning. Two complementary behaviours emerged from the probability distributions of genotypic and genetic indices with increasing c. While the impact of c on R and Pareto \beta was easily described by simple mathematical equations, its effects on genetic indices were noticeable only at the highest levels (c>0.95). Consequently, genotypic indices allowed reliable estimates of c, while genetic descriptors led to poorer performances when c<0.95. These results provide clear baseline expectations for genotypic and genetic diversity and dynamics under partial clonality. Worryingly, however, the use of realistic sample sizes to acquire empirical data systematically led to gross underestimates (often of one to two orders of magnitude) of c, suggesting that many interpretations hitherto proposed in the literature, mostly based on genotypic richness, should be reappraised. We propose future avenues to derive realistic confidence intervals for c and show that, although still approximate, a supervised learning method would greatly improve the estimation of c from population genetics data.

Risk bases can complement dose bases for implementing and optimising a radiological protection strategy in urgent and transition emergency phases

Current radiological emergency response recommendations have been provided by the International Commission on Radiological Protection and adopted by the International Atomic Energy Agency in comprehensive Safety Standards. These standards provide dose-based guidance for decision making (e.g., on sheltering or relocation) via generic criteria in terms of effective dose in the range from 20 mSv per year, during transition from emergency to existing exposure situation, to 100 mSv, acute or annual, in the urgent phase of a nuclear accident. The purpose of this paper was to examine how such dose reference levels directly translate into radiation-related risks of the main stochastic detrimental health effects (cancer). Methodologies, provided by the World Health Organization after the Fukushima accident, for calculating the lifetime and 20 year cancer risks and for attributing relevant organ doses from effective doses, have been applied here for this purpose with new software, designed to be available for use immediately after a nuclear accident. A new feature in this software is a comprehensive accounting for uncertainty via simulation technique, so that the risks may now be presented with realistic confidence intervals. The types of cancer risks considered here are time-integrated over lifetime and the first 20 years after exposure for all solid cancers and either the most radiation-sensitive types of cancer, i.e., leukaemia and female breast cancer, or the most radiation-relevant type of cancer occurring early in life, i.e., thyroid. It is demonstrated here how reference dose levels translate differently into specific cancer risk levels (with varying confidence interval sizes), depending on age at exposure, gender, time-frame at-risk and type of cancer considered. This demonstration applies German population data and considers external exposures. Further work is required to comprehensively extend this methodology to internal exposures that are likely to be important in the early stages of a nuclear accident. A discussion is provided here on the potential for such risk-based information to be used by decision makers, in the urgent and transition phases of nuclear emergencies, to identify protective measures (e.g., sheltering, evacuation) in a differential way (i.e., for particularly susceptible sub-groups of a population).

Calibration of fish-eye lens and error estimation on fireball trajectories: application to the FRIPON network

Context. Fireball networks are developing over the whole planet, with the aim of recovering meteorites and at the same time determining their orbits. The ultimate goal of such networks is to identify the parent bodies of meteorite families to achieve this, orbit accuracy is critical. Yet, the determination of an orbit relies on a long and complex reduction process including: (1) astrometry, with heavy distortion for fish-eye lenses, (2) estimation of the external bias on the observation, (3) fit of the trajectory, (4) deceleration model, and (5) actual orbit computation. Aims. Our goal is to compute accurate trajectories with an estimate of internal and external errors as realistic as possible, taking advantage of the dense observation network FRIPON (Fireball Recovery and InterPlanetary Observation Network), which comprises more than 100 cameras in France and Europe. In particular, we pay special attention to the distortion of images due to fish-eye lenses. In the present paper, we describe the analytical protocol that allows us to compute trajectories and their uncertainties. Methods. We developed a general distortion model to be used on the FRIPON fish-eye cameras. Such a model needs to be accurate even at low elevation, as most fireball observations are performed low on the horizon. The radial distortion is modelled by a nine-degree odd polynomial, hence by five parameters. In addition, we used three parameters to describe the geometry of the camera and two for non-symmetrical distortion. Lastly, we used a new statistical method taking systematic errors into account, which allows us to compute realistic confidence intervals. We tested our method on a fireball that fell on 2017-08-94 UT 00:06. Results. The accuracy of our astrometrical model for each camera is 2 arcmin (1σ), but the internal error on the fireball of 2017-08-94 UT 00:06 measurement is 0.7 arcmin (better than 1/10 pixel). We developed a method to estimate the external error considering that each station is independent and found it equal to 0.8 arcmin. Real residuals are coherent with our estimation of internal and external error for each camera, which confirms the internal consistency of our method. We discuss the advantages and disadvantages of this protocol.

Dynamic parameter estimation for hydrological model

Parameters of hydrological models are commonly assumed to be time independent even though some catchment properties are not stationary. The flexibility of hydrological models can be increased by introducing dynamic variation in the parameters of a hydrological model. The purpose of this paper is to develop a methodology to investigate the dynamic nature of model parameters. A robust dynamic parameter estimation (RDPE) algorithm is developed which can be used for model diagnosis as well as for improving model predictions. A moving window approach and simulated annealing are used to optimise the parameters of the HBV hydrological model for each window size. The resulting time series of parameters are used for defining sensitive and insensitive periods in parameter time series. To improve predictions a predictive parameter model is developed. Finally, it is shown that the new methodology leads to more realistic confidence intervals for model simulations and model structure identification.

Dynamic parameter estimation for hydrological model

Parameters of hydrological models are commonly assumed to be time independent even though some catchment properties are not stationary. The flexibility of hydrological models can be increased by introducing dynamic variation in the parameters of a hydrological model. The purpose of this paper is to develop a methodology to investigate the dynamic nature of model parameters. A robust dynamic parameter estimation (RDPE) algorithm is developed which can be used for model diagnosis as well as for improving model predictions. A moving window approach and simulated annealing are used to optimise the parameters of the HBV hydrological model for each window size. The resulting time series of parameters are used for defining sensitive and insensitive periods in parameter time series. To improve predictions a predictive parameter model is developed. Finally, it is shown that the new methodology leads to more realistic confidence intervals for model simulations and model structure identification.

Extreme Value Analysis of Solar Flare Events

AbstractSpace weather events such as solar flares can be harmful for life and infrastructure on Earth or in near‐Earth orbit. In this paper we employ extreme value theory to model extreme solar flare events; extreme value theory offers the appropriate tools for the study and estimation of probabilities for extrapolation to ranges outside of those that have already been observed. In this work the data points used are inherently independent and realistic confidence intervals are offered with respect to the estimates of future solar flare events. The data used in this study are peak X‐ray fluxes provided by National Oceanic and Atmospheric Administration that form a series of distinct flare events. The expected return levels for Carrington‐ or Halloween storm‐like events were calculated with the outcome that the existing data predict similar events happening in 110 and 38 years, respectively, which are consistent with the results and inferences provided by Elvidge and Angling (2018, https://doi.org/10.1002/2017SW001727). We also make a preliminary analysis of the implications of solar seasonality and found its effect on extreme flare events to be statistically insignificant.

Meta-analysis of Kinetic Parameter Uncertainty on Shelf Life Prediction in the Frozen Fruits and Vegetable Chain

Most studies on frozen foods’ deterioration focus on measurements of selected quality determining indices at the reference frozen storage conditions at limited time points (e.g., 6 and 12 months). This information is not sufficient to predict the frozen system behavior under a different storage temperature, or under the real, dynamic conditions of the actual cold chain. For this purpose, a systematic kinetic study is essential; additionally, the real uncertainty of model parameters needs to be taken into account, in order to proceed to realistic shelf life estimations. In this review work, published findings on kinetic data of deterioration of frozen food of plant origin were analyzed. Kinetic parameters (e.g., activation energy, shelf life, etc.) were extracted and some of them incorporated to a further investigation. The scope is to provide a critical assessment and a comprehensive meta-analysis of the literature information on quality loss modeling of frozen foods. Therefore, common quality indices for specific systems are reviewed, fundamental methodologies used to build kinetic models are assessed, and alternative approaches to improve practical applications of these models are proposed. Alternative methodologies are described in order to take into account the calculated uncertainty of models’ parameters when assessing the remaining shelf life of the product at any point within the cold chain. This was implemented in a FORTRAN code through a Monte Carlo scheme, on literature data of vitamin C loss in different frozen matrices, as well as for other quality indices (e.g., color). Results demonstrated the improved predictions obtained, with broader and more realistic confidence intervals.

The IDEA model: A single equation approach to the Ebola forecasting challenge

Mathematical modeling is increasingly accepted as a tool that can inform disease control policy in the face of emerging infectious diseases, such as the 2014–2015 West African Ebola epidemic, but little is known about the relative performance of alternate forecasting approaches. The RAPIDD Ebola Forecasting Challenge (REFC) tested the ability of eight mathematical models to generate useful forecasts in the face of simulated Ebola outbreaks. We used a simple, phenomenological single-equation model (the “IDEA” model), which relies only on case counts, in the REFC. Model fits were performed using a maximum likelihood approach. We found that the model performed reasonably well relative to other more complex approaches, with performance metrics ranked on average 4th or 5th among participating models. IDEA appeared better suited to long- than short-term forecasts, and could be fit using nothing but reported case counts. Several limitations were identified, including difficulty in identifying epidemic peak (even retrospectively), unrealistically precise confidence intervals, and difficulty interpolating daily case counts when using a model scaled to epidemic generation time. More realistic confidence intervals were generated when case counts were assumed to follow a negative binomial, rather than Poisson, distribution. Nonetheless, IDEA represents a simple phenomenological model, easily implemented in widely available software packages that could be used by frontline public health personnel to generate forecasts with accuracy that approximates that which is achieved using more complex methodologies.

Practical likelihood analysis for spatial generalized linear mixed models

We investigate an algorithm for maximum likelihood estimation of spatial generalized linear mixed models based on the Laplace approximation. We compare our algorithm with a set of alternative approaches for two datasets from the literature. TheRhizoctonia root rotand theRongelapare, respectively, examples of binomial and count datasets modeled by spatial generalized linear mixed models. Our results show that the Laplace approximation provides similar estimates to Markov Chain Monte Carlo likelihood, Monte Carlo expectation maximization, and modified Laplace approximation. Some advantages of Laplace approximation include the computation of the maximized log‐likelihood value, which can be used for model selection and tests, and the possibility to obtain realistic confidence intervals for model parameters based on profile likelihoods. The Laplace approximation also avoids the tuning of algorithms and convergence analysis, commonly required by simulation‐based methods. Copyright © 2015 John Wiley &amp; Sons, Ltd.

Using the BirdTree.org website to obtain robust phylogenies for avian comparative studies: A primer.

Comparative studies of trait evolution require accounting for the shared evolutionary history. This is done by including phylogenetic hypotheses into statistical analyses of species' traits, for which birds often serve as excellent models. The online publication of the most complete molecular phylogeny of extant bird species (www.birdtree.org, BirdTree hereafter) now allows evolutionary biologists to rapidly obtain sets of equally plausible phylogenetic trees for any set of species to be incorporated as a phylogenetic hypothesis in comparative analyses. We discuss methods to use BirdTree tree sets for comparative studies, either by building a consensus tree that can be incorporated into standard comparative analyses, or by using tree sets to account for the effect of phylogenetic uncertainty. Methods accounting for phylogenetic uncertainty should be preferred whenever possible because they should provide more reliable parameter estimates and realistic confidence intervals around them. Based on a real comparative dataset, we ran simulations to investigate the effect of variation in the size of the random tree sets downloaded from BirdTree on the variability of parameter estimates from a bivariate relationship between mass-specific productivity and body mass. Irrespective of the method of analysis, using at least 1,000 trees allows obtaining parameter estimates with very small (< 0.15%) coefficients of variation. We argue that BirdTree, due to the ease of use and the major advantages over previous 'traditional' methods to obtain phylogenetic hypotheses of bird species (e.g. supertrees or manual coding of published phylogenies), will become the standard reference in avian comparative studies for years to come.

On finite sample performance of confidence intervals methods for willingness to pay measures

This paper systematically compares finite sample performances of methods to build confidence intervals for willingness to pay measures in a choice modeling context. It contributes to the field by also considering methods developed in other research fields. Various scenarios are evaluated under an extensive Monte Carlo study. Results show that the commonly used Delta method, producing symmetric intervals around the point estimate, often fails to account for skewness in the estimated willingness to pay distribution. Both the Fieller method and the likelihood ratio test inversion method produce more realistic confidence intervals for small samples. Some bootstrap methods also perform reasonably well, in terms of effective coverage. Finally, empirical data are used to illustrate an application of the methods considered.

A comparison of bootstrap approaches for estimating uncertainty of parameters in linear mixed‐effects models

A version of the nonparametric bootstrap, which resamples the entire subjects from original data, called the case bootstrap, has been increasingly used for estimating uncertainty of parameters in mixed-effects models. It is usually applied to obtain more robust estimates of the parameters and more realistic confidence intervals (CIs). Alternative bootstrap methods, such as residual bootstrap and parametric bootstrap that resample both random effects and residuals, have been proposed to better take into account the hierarchical structure of multi-level and longitudinal data. However, few studies have been performed to compare these different approaches. In this study, we used simulation to evaluate bootstrap methods proposed for linear mixed-effect models. We also compared the results obtained by maximum likelihood (ML) and restricted maximum likelihood (REML). Our simulation studies evidenced the good performance of the case bootstrap as well as the bootstraps of both random effects and residuals. On the other hand, the bootstrap methods that resample only the residuals and the bootstraps combining case and residuals performed poorly. REML and ML provided similar bootstrap estimates of uncertainty, but there was slightly more bias and poorer coverage rate for variance parameters with ML in the sparse design. We applied the proposed methods to a real dataset from a study investigating the natural evolution of Parkinson's disease and were able to confirm that the methods provide plausible estimates of uncertainty. Given that most real-life datasets tend to exhibit heterogeneity in sampling schedules, the residual bootstraps would be expected to perform better than the case bootstrap.

Evaluation of realistic 95% confidence intervals for the activation energy calculated by the iterative linear integral isoconversional method

Cai and Chen have proposed a new iterative linear integral isoconversional method that is capable of providing more accurate values of the activation energy than the traditional linear isoconversional methods ( Cai, Junmeng, Chen, Siyu, 2009. Journal of Computational Chemistry 30, 1986–1991). However, an analysis of the estimation of confidence intervals for the activation energy determined by this iterative linear integral isoconversional method is still missing. Therefore, this short communication focuses on estimating realistic confidence intervals for the activation energy calculated by the isoconversional method. For this purpose, a comparison of confidence intervals estimated using the Vyazovkin–Sbirrazzuoli corrected Student's percentiles ( Vyazovkin, Sergey, Sbirrazzuoli, Nicolas, 1997. Analytica Chimica Acta 355, 175–180) and a nonparametric method has indicated that the use of the Vyazovkin–Sbirrazzuoli method tends to overestimate confidence intervals for three heating rate estimates. The new corrected Student's percentiles for realistic 95% confidence intervals ϑ n–2,0.975 of 2.9, 2.5 and 2.3 for three, four and five heating rate estimates, respectively, can be used to obtain realistic confidence intervals for the activation energy computed by the iterative linear integral isoconversional method.

Generic error model for calibration and uncertainty estimation of hydrological models

Because of the necessary simplification of the complex natural processes and the limited availability of observations, model simulations are always uncertain and this uncertainty should be quantified. In this contribution, the model error is quantified using a combined procedure. For the uncertainty of discharge due to meteorological input, a stochastic simulation method is used. To quantify the effect of process representation and parameterization, a sensitivity analysis is carried out. It is assumed that the model error due to process uncertainty is proportional to the sensitivity. The final model error variance can thus be calculated from the stochastic errors and the process sensitivities. The coefficients used for the quantification are estimated simultaneously with the model parameters. The methodology presented produces error series that are normally distributed and that represent the varying importance of different processes in time. This uncertainty time series can be used as a weighting factor to normalize the model residuals during calibration so that the assumptions of least squares optimization are fulfilled. Calibration and uncertainty estimation are demonstrated with an example application of a distributed Hydrological Bureau Waterbalance (HBV) model of three watersheds in the Neckar basin in southwest Germany. The model residual distributions are presented and compared to a standard calibration method. Further, it is shown that the new methodology leads to more realistic confidence intervals for model simulations. Although applied to the HBV model as an example, the method is general and can be applied to any model and also in conjunction with other uncertainty estimation techniques.

Uncertainty in spatially explicit population models

Spatially explicit population models (SEPMs) are often used in conservation planning. However, confidence intervals around predictions of spatially explicit population models can greatly underestimate model uncertainty. This is partly because some sources of uncertainty are not amenable to the classic methods of uncertainty analysis. Here, we present a method that can be used to include multiple sources of uncertainty into more realistic confidence intervals. To illustrate our approach, we use a case study of the wood thrush (Hylocichla mustelina) in the fragmented forest of the North Carolina Piedmont. We examine 6 important sources of uncertainty in our spatially explicit population model: (1) the habitat map, (2) the dispersal algorithm, (3) clutch size, (4) edge effects, (5) dispersal distance, and (6) the intrinsic variability in our model. We found that uncertainty in the habitat map had the largest effect on model output, but each of the six factors had a significant effect and most had significant interactions with the other factors as well. We also found that our method of incorporating multiple sources of uncertainty created much larger confidence intervals than the projections that incorporated only sources of uncertainty included in most spatially explicit population model predictions.

Application of smoothing methods for estimation of service life for polymers from tensile testing

We propose a procedure for extracting reliable information regarding service life of a polymer product from an experimental scatter plot obtained by mechanical tests. Consideration was given to the change in elongation at break as a result of natural weathering for stabilized polymers. We suggest using locally weighted regression scatter plot smoothing to estimate the service life (threshold time)—weathering time at which the elongation at break of a polymer reduces by half. For the first time the construction of realistic confidence intervals (accuracy of the estimations) for the service life obtained from tensile testing of aged polymers is discussed. The proposed approach is general and applicable to other degradation processes.