Interval Coverage Research Articles

Ensemble forecasting system based on decomposition-selection-optimization for point and interval carbon price prediction

In this study, a novel ensemble forecasting system based on data decomposition, feature selection, optimal sub-model determination and the improved multi-objective optimization is designed to conduct both point and interval carbon price predictions. To testify the performance of the developed system, three daily carbon price series are collected for experiments and verification. The results indicated that the ensemble system has achieved the optimal mean absolute percent errors of 0.9267, 1.6433 and 0.9303% for Sites 1–3 during point forecasting. In addition, the prediction interval coverage probabilities of three sites are 0.9800, 0.9600 and 0.9600 under the 95% confidence level for interval forecasting, which demonstrated that the developed system can provide scientific and sufficient reference for carbon trading markets.

Clustering-based interval prediction of electric load using multi-objective pathfinder algorithm and Elman neural network

Interval prediction of electric load has aroused widespread concern by the power industry because of variability and uncertainty. To quantify the potential uncertainty associated with prediction, this paper proposes a clustering-based approach to construct prediction intervals (PIs) for electric load data. The singular spectrum analysis (SSA) and k-means clustering are firstly performed to decompose the original data due to the high volatility and nonlinearity of load data. Then, we improve the multi-objective pathfinder algorithm (MOPATH) by using crowding degree of population in order to prevent premature, and further utilize the Elman neural network (ELMAN) optimized by IMOPATH to obtain the subseries PIs of electric load data. In addition, the interval width, coverage probability and deviation are used as three optimization objectives. Finally, the IMOPATH, as an ensemble approach, is applied to ensemble the three PIs together and achieves the final PIs. To verify the performance of the SSA-IMOPATH-ELMAN approach, the proposed approach is compared with 41 models. The forecasting outcomes indicate that PIs of the proposed approach have higher coverage probability, narrower width and lower deviation degree than other benchmark models. Moreover, the proposed approach has good performance on robustness and sensibility.

Accommodating a Latent XM Interaction in Statistical Mediation Analysis

Statistical mediation analysis is used in the social sciences and public health to uncover potential mechanisms, known as mediators, by which a treatment led to a change in an outcome. Recently, the estimation of the treatment-by-mediator interaction (i.e., the XM interaction) has been shown to play a pivotal role in understanding the equivalence between the traditional mediation effects in linear models and the causal mediation effects in the potential outcomes framework. However, there is limited guidance on how to estimate the XM interaction when the mediator is latent. In this article, we discuss eight methods to accommodate latent XM interactions in statistical mediation analysis, which fall in two categories: using structural models (e.g., latent moderated structural equations, Bayesian mediation, unconstrained product indicator method, multiple-group models) or scoring the mediator prior to estimating the XM interaction (e.g., summed scores and factor scores, with and without attenuation correction). Simulation results suggest that finite-sample bias is low, type 1 error rates and coverage of percentile bootstrap confidence intervals and Bayesian credible intervals are close to the nominal values, and statistical power is similar across approaches. The methods are demonstrated with an applied example, syntax is provided for their implementation, and general considerations are discussed.

Flexible propensity score estimation strategies for clustered data in observational studies.

Existing studies have suggested superior performance of nonparametric machine learning over logistic regression for propensity score estimation. However, it is unclear whether the advantages of nonparametric propensity score modeling are carried to settings where there is clustering of individuals, especially when there is unmeasured cluster-level confounding. In this work we examined the performance of logistic regression (all main effects), Bayesian additive regression trees and generalized boosted modeling for propensity score weighting in clustered settings, with the clustering being accounted for by including either cluster indicators or random intercepts. We simulated data for three hypothetical observational studies of varying sample and cluster sizes. Confounders were generated at both levels, including a cluster-level confounder that is unobserved in the analyses. A binary treatment and a continuous outcome were generated based on seven scenarios with varying relationships between the treatment and confounders (linear and additive, nonlinear/nonadditive, nonadditive with the unobserved cluster-level confounder). Results suggest that when the sample and cluster sizes are large, nonparametric propensity score estimation may provide better covariate balance, bias reduction, and 95% confidence interval coverage, regardless of the degree of nonlinearity or nonadditivity in the true propensity score model. When the sample or cluster sizes are small, however, nonparametric approaches may become more vulnerable to unmeasured cluster-level confounding and thus may not be a better alternative to multilevel logistic regression. We applied the methods to the National Longitudinal Study of Adolescent to Adult Health data, estimating the effect of team sports participation during adolescence on adulthood depressive symptoms.

A comparison of fixed and random effect models by the number of research in the meta-analysis studies with and without an outlier

The purpose of this research was to compare the performances of the Fixed Effect Model (FEM) and the Random Effects Model (REM) in the meta-analysis studies conducted through 5, 10, 20 and 40 studies with an outlier and 4, 9, 19 and 39 studies without an outlier in terms of estimated common effect size, confidence interval coverage rate and heterogeneity measures. In this descriptive study, real data set consisting of different studies examining teachers’ emotional burnout in terms of gender were used and a total of 72 meta-analyses were performed with R program. The results indicated that REM was more advantageous when compared to FEM for the meta-analysis of data sets with an outlier. On the other hand, without an outlier, it was determined that the common effect size was generally estimated to be similar for all methods. Moreover, the increase in the number of studies included in the meta-analysis reduced the effect of the outlier on the effect size estimation and decreased the heterogeneity. When the examination of the confidence interval coverage accuracy rates of the meta-analysis methods was examined, it was concluded that the confidence intervals included the estimated effect sizes in all data sets and all methods. The findings of the current study showed that the methods used in meta-analysis studies with 20 or more studies were less affected by the outlier runs in the estimated common effect size.

Confidence intervals for proportion of area estimated from a stratified random sample

Estimating proportion of area from a stratified random sample and reference class labels obtained by ground visit or interpretation of satellite imagery is a common strategy in land cover monitoring. Confidence intervals for the proportion of area are typically estimated using the Wald interval, a procedure that is known to yield less than nominal coverage (i.e., undercoverage) when the proportion of area is small (i.e., comprising 10% or less of the region of interest). The practical implication of undercoverage is that confidence intervals will include the true proportion of area for a smaller percent of samples than is implied by the stated confidence level (e.g., 95%). For simple random sampling, alternate confidence interval methods such as Wilson, Agresti-Coull, and Clopper-Pearson have been developed to remedy the undercoverage of the Wald interval, but these methods are not directly applicable to stratified sampling. In this study, we evaluated the Wald interval and two alternative general confidence interval approaches for the case of two strata. One alternative approach, previously proposed in the literature, is based on estimating effective sample size (neff method), whereas the other approach is a new method based on summing stratum-specific confidence bounds (sumstrat method) computed using one of the alternate confidence interval approaches (Wilson, Agresti-Coull, and Clopper-Pearson) for each stratum. The coverage and length of confidence intervals estimated from the different methods were compared using Monte Carlo simulation applied to a set of populations representing diverse conditions of prevalence of the target class and user's and producer's accuracies of the map used to construct the strata. The commonly applied Wald interval achieved coverage well below the nominal 95% confidence for many of the populations evaluated, and the neff method improved upon this undercoverage in only a few populations. The Wilson sumstrat intervals provided the best coverage as this method achieved close to the nominal 95% coverage or greater for most populations. Length of the Wilson sumstrat intervals was comparable to length of other methods. Further study is needed to determine which confidence interval methods are effective when the number of strata exceeds two.

Assessing the Impact of Features on Probabilistic Modeling of Photovoltaic Power Generation

Photovoltaic power generation has high variability and uncertainty because it is affected by uncertain factors such as weather conditions. Therefore, probabilistic forecasting is useful for optimal operation and risk hedging in power systems with large amounts of photovoltaic power generation. However, deterministic forecasting is the mainstay of photovoltaic generation forecasting; there are few studies on probabilistic forecasting and feature selection from weather or time-oriented features in such forecasting. In this study, prediction intervals were generated by the lower upper bound estimation (LUBE) using neural networks with two outputs to make probabilistic modeling for predictions. The objective was to improve prediction interval coverage probability (PICP), mean prediction interval width (MPIW), continuous ranked probability score (CRPS), and loss, which is the integration of PICP and MPIW, by removing unnecessary features through feature selection. When features with high gain were selected by random forest (RF), in the modeling of 14.7 kW PV systems, loss improved by 1.57 kW, CRPS by 0.03 kW, PICP by 0.057 kW, and MPIW by 0.12 kW on average over two weeks compared to the case where all features were used without feature selection. Therefore, the low gain features from RF act as noise and reduce the modeling accuracy.

Targeted maximum likelihood estimation of causal effects with interference: A simulation study.

Interference, the dependency of an individual's potential outcome on the exposure of other individuals, is a common occurrence in medicine and public health. Recently, targeted maximum likelihood estimation (TMLE) has been extended to settings of interference, including in the context of estimation of the mean of an outcome under a specified distribution of exposure, referred to as a policy. This paper summarizes how TMLE for independent data is extended to general interference (network-TMLE). An extensive simulation study is presented of network-TMLE, consisting of four data generating mechanisms (unit-treatment effect only, spillover effects only, unit-treatment and spillover effects, infection transmission) in networks of varying structures. Simulations show that network-TMLE performs well across scenarios with interference, but issues manifest when policies are not well-supported by the observed data, potentially leading to poor confidence interval coverage. Guidance for practical application, freely available software, and areas of future work are provided.

Solutions for surrogacy validation with longitudinal outcomes for a gene therapy.

Valid surrogate endpoints S can be used as a substitute for a true outcome of interest T to measure treatment efficacy in a clinical trial. We propose a causal inference approach to validate a surrogate by incorporating longitudinal measurements of the true outcomes using a mixed modeling approach, and we define models and quantities for validation that may vary across the study period using principal surrogacy criteria. We consider a surrogate-dependent treatment efficacy curve that allows us to validate the surrogate at different time points. We extend these methods to accommodate a delayed-start treatment design where all patients eventually receive the treatment. Not all parameters are identified in the general setting. We apply a Bayesian approach for estimation and inference, utilizing more informative prior distributions for selected parameters. We consider the sensitivity of these prior assumptions as well as assumptions of independence among certain counterfactual quantities conditional on pretreatment covariates to improve identifiability. We examine the frequentist properties (bias of point and variance estimates, credible interval coverage) of a Bayesian imputation method. Our work is motivated by a clinical trial of a gene therapy where the functional outcomes are measured repeatedly throughout thetrial.

Bounded error modeling using interval neural networks with parameter optimization

Aiming at the issue of interval modeling of uncertain systems, this article proposes a data-based interval neural network (INN) optimization modeling method that combines interval analysis with a neural network and particle swarm optimization (PSO) algorithm, adopts an INN model with interval weights and interval thresholds, and constructs an interval multiobjective PSO (IMOPSO) algorithm to evolve the network, thereby modeling an uncertain system under an unknown-but-bounded error (UBBE) condition. Considering the cases of known and unknown error bounds in UBBE, two optimization objectives, i.e., interval coverage and interval width, are constructed to optimize the parameters of INN, aiming to improve the accuracy and reliability of prediction. The proposed method effectively solves many constraints such as the model structure demanded and error bounds known in UBBE modeling, and provides a new method of data-based interval system modeling. The method was applied to model linear and nonlinear systems, and simulation results showed that the established INN models have good prediction effects and dynamic characteristics, which demonstrate the effectiveness of the method.

A comparison of design-based and model-based approaches for finite population spatial sampling and inference.

The design-based and model-based approaches to frequentist statistical inference rest on fundamentally different foundations. In the design-based approach, inference relies on random sampling. In the model-based approach, inference relies on distributional assumptions. We compare the approaches in a finite population spatial context.We provide relevant background for the design-based and model-based approaches and then study their performance using simulated data and real data. The real data is from the United States Environmental Protection Agency's 2012 National Lakes Assessment. A variety of sample sizes, location layouts, dependence structures, and response types are considered. The population mean is the parameter of interest, and performance is measured using statistics like bias, squared error, and interval coverage.When studying the simulated and real data, we found that regardless of the strength of spatial dependence in the data, the generalized random tessellation stratified (GRTS) algorithm, which explicitly incorporates spatial locations into sampling, tends to outperform the simple random sampling (SRS) algorithm, which does not explicitly incorporate spatial locations into sampling. We also found that model-based inference tends to outperform design-based inference, even for skewed data where the model-based distributional assumptions are violated. The performance gap between design-based inference and model-based inference is small when GRTS samples are used but large when SRS samples are used, suggesting that the sampling choice (whether to use GRTS or SRS) is most important when performing design-based inference.There are many benefits and drawbacks to the design-based and model-based approaches for finite population spatial sampling and inference that practitioners must consider when choosing between them. We provide relevant background contextualizing each approach and study their properties in a variety of scenarios, making recommendations for use based on the practitioner's goals.

Railway tie deterioration interval estimation with Bayesian deep learning and data-driven maintenance strategy

Wooden railway ties still occupy a significant proportion of the North American railway network. Many existing tie maintenance schemes are performed using expensive field tests. Using tie deterioration prediction to estimate the required tie inventory is a cost-efficient solution for tackling tie maintenance issues. This study proposes an improved multilayer neural network model using Bayesian deep learning (BDL-MLP). Balanced and efficient interval estimation of the tie deterioration rate was achieved by analyzing weather and historical detection datasets. A comprehensive evaluation metric, cover width deviation and error (CWDE), was also proposed to evaluate the result of prediction interval (PI). Using this metric, the CWDE value of the BDL-MLP model reached 0.9324, and the interval coverage rate was 97.50%. Compared with the combined model, the CWDE index increased by 1.48–5.40%, and the interval coverage rate increased by 4.21–22.15%. Finally, a deterioration data-driven maintenance strategy is established based on the quantile regression principle. This method estimates the tie-replacement inventory under different deterioration types, resulting in economical maintenance solutions.

Jenss–Bayley Latent Change Score Model With Individual Ratio of the Growth Acceleration in the Framework of Individual Measurement Occasions

Longitudinal data analysis has been widely employed to examine between-individual differences in within-individual changes. One challenge of such analyses is that the rate-of-change is only available indirectly when change patterns are nonlinear with respect to time. Latent change score models (LCSMs), which can be employed to investigate the change in rate-of-change at the individual level, have been developed to address this challenge. We extend an existing LCSM with the Jenss–Bayley growth curve and propose a novel expression for change scores that allows for (1) unequally spaced study waves and (2) individual measurement occasions around each wave. We also extend the existing model to estimate the individual ratio of the growth acceleration (that largely determines the trajectory shape and is viewed as the most important parameter in the Jenss–Bayley model). We present the proposed model by a simulation study and a real-world data analysis. Our simulation study demonstrates that the proposed model can estimate the parameters unbiasedly and precisely and exhibit target confidence interval coverage. The simulation study also shows that the proposed model with the novel expression for the change scores outperforms the existing model. An empirical example using longitudinal reading scores shows that the model can estimate the individual ratio of the growth acceleration and generate individual rate-of-change in practice. We also provide the corresponding code for the proposed model.

MISL: Multiple imputation by super learning.

Multiple imputation techniques are commonly used when data are missing, however, there are many options one can consider. Multivariate imputation by chained equations is a popular method for generating imputations but relies on specifying models when imputing missing values. In this work, we introduce multiple imputation by super learning, an update to the multivariate imputation by chained equations method to generate imputations with ensemble learning. Ensemble methodologies have recently gained attention for use in inference and prediction as they optimally combine a variety of user-specified parametric and non-parametric models and perform well when estimating complex functions, including those with interaction terms. Through two simulations we compare inferences made using the multiple imputation by super learning approach to those made with other commonly used multiple imputation methods and demonstrate multiple imputation by super learning as a superior option when considering characteristics such as bias, confidence interval coverage rate, and confidence interval width.

An interval prediction method for quantifying the uncertainties of cooling load based on time classification

The uncertainties of cooling load significantly impact the operations performance of building energy systems. A novel interval prediction method is proposed to quantify the uncertainties of cooling load. Due to the cooling load having a similar changing trend from 1:00 to 24:00, a cooling load classification method based on the time of the day is proposed to analyze the uncertainty. Firstly, the gated recurrent unit algorithm is used to construct the point prediction model. Secondly, the kernel density estimation method is employed for probability statistics of prediction error. Moreover, the percentile method constructs the upper and lower bounds of the cooling load prediction interval with given confidence levels. Finally, a commonly used interval prediction method is compared to the proposed method, and a real cooling load dataset is used to assess the prediction method. The results demonstrate that the point prediction model has a mean absolute percentage error of 10.19% in the interval prediction dataset. The interval prediction results illustrate that the prediction interval coverage probability is 91.75%, and the prediction interval normalized average width is 20.83% in the confidence level of 90%. Compared with the commonly used method, the proposed method has a narrower interval.

Optimal prediction intervals for macroeconomic time series using chaos and evolutionary multi-objective optimization algorithms

In a first-of-its-kind study, this paper formulates the problem of estimating the prediction intervals (PIs) in a macroeconomic time series as a bi-objective optimization problem and solves it with three evolutionary algorithms namely, Non-dominated Sorting Genetic Algorithm (NSGA-II), Non-dominated Sorting Particle Swarm Optimization (NSPSO) and Multi-Objective Evolutionary Algorithm based on Decomposition (MOEA-D). We also proposed modeling the chaos present in the time series as a preprocessor, which we called stage-1. Accordingly, we proposed 2-stage models, where stage-1 is followed by obtaining the optimal point prediction using NSGA-II/NSPSO/MOEA-D and using these point predictions to obtain PIs (stage-2). We then proposed a 3-stage hybrid, which is built on the 2-stage model, wherein the 3 rd stage also invokes NSGA-II/NSPSO/MOEA-D in order to estimate the PIs from the point predictions obtained in 2 nd stage by simultaneously and explicitly optimizing (i) prediction interval coverage probability (PICP) and (ii) prediction interval average width (PIAW). The proposed models yielded better results in terms of both PICP and PIAW compared to the state-of-the-art Lower Upper Bound Estimation Method (LUBE) with Gradient Descent (GD) and LUBE with long short-term memory (LSTM) network. The 3-stage models outperformed the 2-stage models with respect to PICP but showed similar performance in PIAW at the cost of running NSGA-II/NSPSO/MOEA-D second time. Overall, MOEA-D yielded best PIs in two datasets and NSGA-II outperformed the other two in the third dataset. But, in terms of hypervolume, in 2-stage MOEA-D produced most diverse solutions in two datasets, while NSGA-II was the winner in the third dataset.

Machine learning for recovery factor estimation of an oil reservoir: A tool for derisking at a hydrocarbon asset evaluation

Well-known oil recovery factor estimation techniques such as analogy, volumetric calculations, material balance, decline curve analysis, hydrodynamic simulations have certain limitations. Those techniques are time-consuming, and require specific data and expert knowledge. Besides, though uncertainty estimation is highly desirable for this problem, the methods above do not include this by default. In this work, we present a data-driven technique for oil recovery factor (limited to water flooding) estimation using reservoir parameters and representative statistics. We apply advanced machine learning methods to historical worldwide oilfields datasets (more than 2000 oil reservoirs). The data-driven model might be used as a general tool for rapid and completely objective estimation of the oil recovery factor. In addition, it includes the ability to work with partial input data and to estimate the prediction interval of the oil recovery factor. We perform the evaluation in terms of accuracy and prediction intervals coverage for several tree-based machine learning techniques in application to the following two cases: (1) using parameters only related to geometry, geology, transport, storage and fluid properties, (2) using an extended set of parameters including development and production data. For both cases, the model proved itself to be robust and reliable. We conclude that the proposed data-driven approach overcomes several limitations of the traditional methods and is suitable for rapid, reliable and objective estimation of oil recovery factor for hydrocarbon reservoir.

Reliability Analysis of Hybrid System Using Geometric Process in Multiple Level of Constant Stress Accelerated Life Test through Simulation Study for Type-II Progressive Censored Masked Data

Numerous studies have already been attempted to explore the reliability of systems considering mask data, though the mass of them has largely focused on basic series or parallel systems, where component failures are assumed to follow an exponential or Weibull distribution. However, most electrotonic products and systems are made up of numerous components integrated in parallel-series, series-parallel, and other bridge hybrid structures, and the number of studies in the area of accelerated life testing (ALT) employing masked data for hybrid systems is limited. In this paper, the constant-stress ALT (CSALT) is explored based on type-II progressive censoring scheme (TIIPCS) for a four-component hybrid system using geometric process (GmP). The failure times of the components of the system are assumed to follow the generalized Pareto (GP) distribution. The maximum likelihood estimate (MLE) technique is used to establish statistical inference for the model's unknown parameters under the premise that the failure reasons are unknown for the hybrid system. In addition, the asymptotic confidence intervals (ACIs) are also obtained by inverting the fisher information matrix. Finally, a simulation study is given to explain the proposed techniques and to evaluate the performance of the estimates. The performance of MLEs is assessed in terms of root mean square errors (RMSEs) and relative absolute biases (RABs), whereas the performance of ACIs is assessed in terms of their interval length (IL) and coverage probabilities (CPs). The findings show that the technique can deliver good estimation performance with small and intermediate sample sizes, and the estimates are more accurate when more failures are observed, showing the estimation method's efficiency.

Gecko: A time-series model for COVID-19 hospital admission forecasting

During the COVID-19 pandemic, concerns about hospital capacity in the United States led to a demand for models that forecast COVID-19 hospital admissions. These short-term forecasts were needed to support planning efforts by providing decision-makers with insight about future demands for health care capacity and resources. We present a SARIMA time-series model called Gecko developed for this purpose. We evaluate its historical performance using metrics such as mean absolute error, predictive interval coverage, and weighted interval scores, and compare to alternative hospital admission forecasting models. We find that Gecko outperformed baseline approaches and was among the most accurate models for forecasting hospital admissions at the state and national levels from January–May 2021. This work suggests that simple statistical methods can provide a viable alternative to traditional epidemic models for short-term forecasting.

Implementing planned missingness in stimulus sampling designs: Strategies for optimizing statistical power and precision while limiting participant burden

The stimulus sampling design (SSD) is a widely applied research paradigm in which participants evaluate a series of visual or auditory stimuli. Researchers are often interested in phenomena reflected by differences across stimuli, and recent simulation work suggests that more than 100 stimuli may be necessary to detect smaller effects. Unfortunately, administering a large number of stimuli to each participant may compromise the validity of responses due to individual differences in susceptibility to fatigue and distraction. The present work describes the application of planned missingness design strategies to maximize statistical power, while minimizing participant burden in SSD studies. A Monte Carlo simulation study was conducted to determine the number of stimuli per rater (SPR) and average raters per stimuli (RPS) needed to ensure unbiased estimates of model parameters, as well as the desired statistical power and interval coverage. Findings suggest that two commonly used statistical estimation techniques, restricted maximum likelihood and the Markov Chain Monte Carlo algorithm, provide reliable estimates of model parameters and maintain adequate statistical power under substantial levels of planned missingness. Incorporating these methods will allow researchers to design more efficient and powerful experiments by reducing redundancies and minimizing sources of methodological error. Recommendations for incorporating planned missingness strategies in SSDs are provided.