Confidence Intervals Of Model Predictions Research Articles

A Primer on Dose-Response Data Modeling in Radiation Therapy

An overview of common approaches used to assess a dose response for radiation therapy-associated endpoints is presented, using lung toxicity data sets analyzed as a part of the High Dose per Fraction, Hypofractionated Treatment Effects in the Clinic effort as an example. Each component presented (eg, data-driven analysis, dose-response analysis, and calculating uncertainties on model prediction) is addressed using established approaches. Specifically, the maximum likelihood method was used to calculate best parameter values of the commonly used logistic model, the profile-likelihood to calculate confidence intervals on model parameters, and the likelihood ratio to determine whether the observed data fit is statistically significant. The bootstrap method was used to calculate confidence intervals for model predictions. Correlated behavior of model parameters and implication for interpreting dose response are discussed.

Estimating HIV Incidence, Time to Diagnosis, and the Undiagnosed HIV Epidemic Using Routine Surveillance Data.

Estimates of the size of the undiagnosed HIV-infected population are important to understand the HIV epidemic and to plan interventions, including "test-and-treat" strategies. We developed a multi-state back-calculation model to estimate HIV incidence, time between infection and diagnosis, and the undiagnosed population by CD4 count strata, using surveillance data on new HIV and AIDS diagnoses. The HIV incidence curve was modelled using cubic splines. The model was tested on simulated data and applied to surveillance data on men who have sex with men in The Netherlands. The number of HIV infections could be estimated accurately using simulated data, with most values within the 95% confidence intervals of model predictions. When applying the model to Dutch surveillance data, 15,400 (95% confidence interval [CI] = 15,000, 16,000) men who have sex with men were estimated to have been infected between 1980 and 2011. HIV incidence showed a bimodal distribution, with peaks around 1985 and 2005 and a decline in recent years. Mean time to diagnosis was 6.1 (95% CI = 5.8, 6.4) years between 1984 and 1995 and decreased to 2.6 (2.3, 3.0) years in 2011. By the end of 2011, 11,500 (11,000, 12,000) men who have sex with men in The Netherlands were estimated to be living with HIV, of whom 1,750 (1,450, 2,200) were still undiagnosed. Of the undiagnosed men who have sex with men, 29% (22, 37) were infected for less than 1 year, and 16% (13, 20) for more than 5 years. This multi-state back-calculation model will be useful to estimate HIV incidence, time to diagnosis, and the undiagnosed HIV epidemic based on routine surveillance data.

Assessing the quality of landslide susceptibility maps – case study Lower Austria

Abstract. Landslide susceptibility maps are helpful tools to identify areas potentially prone to future landslide occurrence. As more and more national and provincial authorities demand for these maps to be computed and implemented in spatial planning strategies, several aspects of the quality of the landslide susceptibility model and the resulting classified map are of high interest. In this study of landslides in Lower Austria, we focus on the model form uncertainty to assess the quality of a flexible statistical modelling technique, the generalized additive model (GAM). The study area (15 850 km2) is divided into 16 modelling domains based on lithology classes. A model representing the entire study area is constructed by combining these models. The performances of the models are assessed using repeated k-fold cross-validation with spatial and random subsampling. This reflects the variability of performance estimates arising from sampling variation. Measures of spatial transferability and thematic consistency are applied to empirically assess model quality. We also analyse and visualize the implications of spatially varying prediction uncertainties regarding the susceptibility map classes by taking into account the confidence intervals of model predictions. The 95% confidence limits fall within the same susceptibility class in 85% of the study area. Overall, this study contributes to advancing open communication and assessment of model quality related to statistical landslide susceptibility models.

A model for the dynamics of malaria in Paria Peninsula, Sucre State, Venezuela

Malaria is a serious public health problem, with close to half of the world population at risk of infection. Mathematical models have been useful to guide malaria control. Most models have assumed that vector population density is constant over time. This assumption can introduce serious errors at time scales where mosquito density is variable. In the present work a model is constructed which takes into account the dynamics of the disease transmission between host and vector, and assumes that vector population density is a dynamic variable. The model assumes that vector population regulation occurs during the larval stage, and the density dependence is modeled with a hyperbolic function. Rainfall in the region changes dramatically and has annual seasonality, but temperature is almost constant during the year; so the only exogenous factor considered in the model is rainfall. The resulting model has four state variables. The original 4-dimensional system was reduced to a one-dimensional equation, with 4 delays, that tracks the dynamics of infected humans, the only state variable for which long time series are available. Parameters of the resulting equation were estimated by fitting the model to time series of human incidence from several localities of Paria Peninsula in Sucre State, Venezuela. About eighty per cent of the incidence records fall within the 95% confidence intervals of model predictions. There is also evidence that different localities have different dynamics. Finally, we compare our model with other modeling approaches in malaria studies, and its usefulness is discussed.

Likelihood based observability analysis and confidence intervals for predictions of dynamic models

BackgroundPredicting a system’s behavior based on a mathematical model is a primary task in Systems Biology. If the model parameters are estimated from experimental data, the parameter uncertainty has to be translated into confidence intervals for model predictions. For dynamic models of biochemical networks, the nonlinearity in combination with the large number of parameters hampers the calculation of prediction confidence intervals and renders classical approaches as hardly feasible.ResultsIn this article reliable confidence intervals are calculated based on the prediction profile likelihood. Such prediction confidence intervals of the dynamic states can be utilized for a data-based observability analysis. The method is also applicable if there are non-identifiable parameters yielding to some insufficiently specified model predictions that can be interpreted as non-observability. Moreover, a validation profile likelihood is introduced that should be applied when noisy validation experiments are to be interpreted.ConclusionsThe presented methodology allows the propagation of uncertainty from experimental to model predictions. Although presented in the context of ordinary differential equations, the concept is general and also applicable to other types of models. Matlab code which can be used as a template to implement the method is provided at http://www.fdmold.uni-freiburg.de/∼ckreutz/PPL.

Incorporating Uncertainty Into the Ranking of SPARROW Model Nutrient Yields From Mississippi/Atchafalaya River Basin Watersheds1

Excessive loads of nutrients transported by tributary rivers have been linked to hypoxia in the Gulf of Mexico. Management efforts to reduce the hypoxic zone in the Gulf of Mexico and improve the water quality of rivers and streams could benefit from targeting nutrient reductions toward watersheds with the highest nutrient yields delivered to sensitive downstream waters. One challenge is that most conventional watershed modeling approaches (e.g., mechanistic models) used in these management decisions do not consider uncertainties in the predictions of nutrient yields and their downstream delivery. The increasing use of parameter estimation procedures to statistically estimate model coefficients, however, allows uncertainties in these predictions to be reliably estimated. Here, we use a robust bootstrapping procedure applied to the results of a previous application of the hybrid statistical/mechanistic watershed model SPARROW (Spatially Referenced Regression On Watershed attributes) to develop a statistically reliable method for identifying “high priority” areas for management, based on a probabilistic ranking of delivered nutrient yields from watersheds throughout a basin. The method is designed to be used by managers to prioritize watersheds where additional stream monitoring and evaluations of nutrient-reduction strategies could be undertaken. Our ranking procedure incorporates information on the confidence intervals of model predictions and the corresponding watershed rankings of the delivered nutrient yields. From this quantified uncertainty, we estimate the probability that individual watersheds are among a collection of watersheds that have the highest delivered nutrient yields. We illustrate the application of the procedure to 818 eight-digit Hydrologic Unit Code watersheds in the Mississippi/Atchafalaya River basin by identifying 150 watersheds having the highest delivered nutrient yields to the Gulf of Mexico. Highest delivered yields were from watersheds in the Central Mississippi, Ohio, and Lower Mississippi River basins. With 90% confidence, only a few watersheds can be reliably placed into the highest 150 category; however, many more watersheds can be removed from consideration as not belonging to the highest 150 category. Results from this ranking procedure provide robust information on watershed nutrient yields that can benefit management efforts to reduce nutrient loadings to downstream coastal waters, such as the Gulf of Mexico, or to local receiving streams and reservoirs.

MODELING THE BIG BLACK RIVER: A COMPARISON OF WATER QUALITY MODELS1

ABSTRACT: The Mississippi Department of Environmental Quality uses the Steady Riverine Environmental Assessment Model (STREAM) to establish effluent limitations. While the U.S. Environmental Protection Agency has approved of its use, questions arise regarding the model's simplicity. The objective of this research was to compare STREAM with the more commonly utilized Enhanced Stream Water Quality Model (QUAL2E). The comparison involved a statistical evaluation procedure based on sensitivity analyses, input probability distribution functions, and Monte Carlo simulation with site‐specific data from a 46‐mile (74‐km) reach of the Big Black River in central Mississippi. Site specific probability distribution functions were derived from measured rates of reaeration, sediment oxygen demand, photosynthesis, and respiration. Both STREAM and QUAL2E reasonably predicted daily average dissolved oxygen (DO) based on a comparison of output probability distributions with observed DO. Observed DO was consistently within 90 percent confidence intervals of model predictions. The STREAM approach generally overpredicted while QUAL2E generally matched observed DO. Using the more commonly assumed lognormal distribution as opposed to a Weibull distribution for two of the sensitive input parameters resulted in minimal differences in the statistical evaluations. The QUAL2E approach had distinct advantages over STREAM in simulating the growth cycle of algae.

Coral injury and recovery: matrix models link process to pattern

In corals, extant damage to colonies depends on the balance between injury and recovery rates. Sclerochronology — the study of scleractinian coral growth rings — provides information about these historical processes, since corals preserve traces of previous injuries and annual bands representing regrowth. Cross-sections of two Caribbean coral species, Porites astreoides (Ellis and Solander) and Siderastrea siderea Le Sueur, were examined to determine the size-frequency and incidence of partial mortality and the rate at which these injuries recover. Each year's injuries were divided into four size classes, and subsequent changes in lesion size were developed into probabilities of switching size classes each year. Matrix models, composed of a matrix of transition probabilities and a vector added annually to account for new damage, predicted a dynamic equilibrium of extant damage to corals that matches field censuses well. A census of corals was carried out at two ( P. astreoides) or five ( S. siderea) sites. In 23 of 28 comparisons, values of extant damage were within the 95% confidence intervals of model predictions. Discrepancies arose because the model underestimated small lesions in P. astreoides. Sclerochronology may fail in this instance because small, rapidly-recovering injuries do not leave permanent scars in coral skeleton, although they are common in point observations of reef damage. Based on sclerochronology, P. astreoides and S. siderea are differentially susceptible to injury, and P. astreoides tends to recover more rapidly.