Prespecified Confidence Level Research Articles

Inference on optimal treatment assignments

We consider inference on optimal treatment assignments. Our methods allow inference on the treatment assignment rule that would be optimal given knowledge of the population treatment effect in a general setting. The procedure uses multiple hypothesis testing methods to determine a subset of the population for which assignment to treatment can be determined to be optimal after conditioning on all available information, with a prespecified level of confidence. A Monte Carlo study confirms that the inference procedure has good small sample behavior. We apply the method to study Project STAR and the optimal assignment of a small class intervention based on school and teacher characteristics.

A Robust Data-Driven Approach for the Newsvendor Problem with Nonparametric Information

Problem definition: For the standard newsvendor problem with an unknown demand distribution, we develop an approach that uses data input to construct a distribution ambiguity set with the nonparametric characteristics of the true distribution, and we use it to make robust decisions. Academic/practical relevance: Empirical approach relies on historical data to estimate the true distribution. Although the estimated distribution converges to the true distribution, its performance with limited data is not guaranteed. Our approach generates robust decisions from a distribution ambiguity set that is constructed by data-driven estimators for nonparametric characteristics and includes the true distribution with the desired probability. It fits situations where data size is small. Methodology: We apply a robust optimization approach with nonparametric information. Results: Under a fixed method to partition the support of the demand, we construct a distribution ambiguity set, build a protection curve as a proxy for the worst-case distribution in the set, and use it to obtain a robust stocking quantity in closed form. Implementation-wise, we develop an adaptive method to continuously feed data to update partitions with a prespecified confidence level in their unbiasedness and adjust the protection curve to obtain robust decisions. We theoretically and experimentally compare the proposed approach with existing approaches. Managerial implications: Our nonparametric approach under adaptive partitioning guarantees that the realized average profit exceeds the worst-case expected profit with a high probability. Using real data sets from Kaggle.com, it can outperform existing approaches in yielding profit rate and stabilizing the generated profits, and the advantages are more prominent as the service ratio decreases. Nonparametric information is more valuable than parametric information in profit generation provided that the service requirement is not too high. Moreover, our proposed approach provides a means of combining nonparametric and parametric information in a robust optimization framework.

Comparing adaptive interventions under a general sequential multiple assignment randomized trial design via multiple comparisons with the best

This paper considers screening of adaptive interventions or adaptive treatment strategies embedded in a sequential multiple assignment randomized trial (SMART). As a SMART typically consists of numerous adaptive interventions, inferential procedures based on pairwise comparisons of all interventions may suffer substantial loss in efficiency after accounting for multiplicity. We propose simultaneous confidence intervals that compare the values of interventions of interest to that of the unknown best intervention by generalizing the method in Edwards and Hsu (1983). The multiple comparison with the best (MCB) intervals are applied as screening tool: an intervention with MCB interval excluding zero will be declared as inferior to the true best at a pre-specified confidence level, and hence excluded from further exploration. Simulation studies show that the proposed method outperforms the multiple comparison procedures based on Bonferroni’s correction in terms of width of confidence intervals for estimation. The method is applied to analyze data from the CODIACS trial in patients with depression.

Confidence intervals of the difference between areas under two ROC curves in matched-pair experiments.

In a matched-pair study, when outcomes of two diagnostic tests are ordinal/continuous, the difference between two correlated areas under ROC curves (AUCs) is usually used to compare the overall discriminatory ability of two diagnostic tests. This article considers confidence interval (CI) construction problems of difference between two correlated AUCs in a matched-pair experiment, and proposes 13 hybrid CIs based on variance estimates recovery with the maximum likelihood estimation, Delong's statistic, Wilson score statistic (WS) and WS with continuity correction, the modified Wald statistic (MW) and MW with continuity correction and Agresti-Coull statistic, and three Bootstrap-resampling-based CIs. For comparison, we present traditional parametric and nonparametric CIs. Simulation studies are conducted to assess the performance of the proposed CIs in terms of empirical coverage probabilities, empirical interval widths, and ratios of the mesial noncoverage probabilities to the noncoverage probabilities. Two examples from clinical studies are illustrated by the proposed methodologies. Empirical results evidence that the hybrid Agresti-Coull CI with the empirical estimation (EAC) behaved most satisfactorily because its coverage probability was quite close to the prespecified confidence level with short interval width. Hence, we recommend the usage of the EAC CI in applications.

The Minimum Cost Flow Problem of Uncertain Random Network

The aim of this paper is to present a novel method for solving the minimum cost flow problem on networks with uncertain-random capacities and costs. The objective function of this problem is an uncertain random variable and the constraints of the problem do not make a deterministic feasible set. Under the framework of uncertain random programming, a corresponding [Formula: see text]-minimum cost flow model with a prespecified confidence level [Formula: see text], is formulated and its main properties are analyzed. It is proven that there exists an equivalence relationship between this model and the classical deterministic minimum cost flow model. Then an algorithm is proposed to find the maximum amount of [Formula: see text] such that for it, the feasible set of [Formula: see text]-minimum cost flow model is nonempty. Finally, a numerical example is presented to illustrate the efficiency of our proposed method.

Process capability in industry: Setting preliminary statistical specification limits

ABSTRACTIn industrial processes, capability indices like the Cpk are frequently used. This index requires that specification limits have been set. The current paper focuses on the issue to set preliminary specification limits based on the statistical performance of the process alone. This preliminary interval should correspond to a Cpk value that is considered to be sufficiently high, with a pre-specified confidence level. The paper presents a theoretical framework to set these limits.

Confidence interval construction for the difference between two correlated proportions with missing observations

Under the assumption of missing at random, eight confidence intervals (CIs) for the difference between two correlated proportions in the presence of incomplete paired binary data are constructed on the basis of the likelihood ratio statistic, the score statistic, the Wald-type statistic, the hybrid method incorporated with the Wilson score and Agresti–Coull (AC) intervals, and the Bootstrap-resampling method. Extensive simulation studies are conducted to evaluate the performance of the presented CIs in terms of coverage probability and expected interval width. Our empirical results evidence that the Wilson-score-based hybrid CI and the Wald-type CI together with the constrained maximum likelihood estimates perform well for small-to-moderate sample sizes in the sense that (i) their empirical coverage probabilities are quite close to the prespecified confidence level, (ii) their expected interval widths are shorter, and (iii) their ratios of the mesial non-coverage to non-coverage probabilities lie in interval [0.4, 0.6]. An example from a neurological study is used to illustrate the proposed methodologies.

Confidence, tolerance, and allowance in biological engineering: the nuts and bolts of living things.

The emphasis of systems and synthetic biology on quantitative understanding of biological objects and their eventual re-design has raised the question of whether description and construction standards that are commonplace in electric and mechanical engineering are applicable to live systems. The tuning of genetic devices to deliver a given activity is generally context-dependent, thereby undermining the re-usability of parts, and predictability of function, necessary for manufacturing new biological objects. Tolerance (acceptable limits within the unavoidable divergence of a nominal value) and allowance (deviation introduced on purpose for the sake of flexibility and hence modularity, i.e. fitting together with a variety of other components) are key aspects of standardization that need to be brought to biological design. These should endow functional building blocks with a pre-specified level of confidence for bespoke biosystems engineering. However, in the absence of more fundamental knowledge, fine-tuning necessarily relies on evolutionary/combinatorial gravitation toward a fixed objective. Also watch the Video Abstract.

Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study.

BackgroundExternal pilot or feasibility studies can be used to estimate key unknown parameters to inform the design of the definitive randomised controlled trial (RCT). However, there is little consensus on how large pilot studies need to be, and some suggest inflating estimates to adjust for the lack of precision when planning the definitive RCT.MethodsWe use a simulation approach to illustrate the sampling distribution of the standard deviation for continuous outcomes and the event rate for binary outcomes. We present the impact of increasing the pilot sample size on the precision and bias of these estimates, and predicted power under three realistic scenarios. We also illustrate the consequences of using a confidence interval argument to inflate estimates so the required power is achieved with a pre-specified level of confidence. We limit our attention to external pilot and feasibility studies prior to a two-parallel-balanced-group superiority RCT.ResultsFor normally distributed outcomes, the relative gain in precision of the pooled standard deviation (SDp) is less than 10% (for each five subjects added per group) once the total sample size is 70. For true proportions between 0.1 and 0.5, we find the gain in precision for each five subjects added to the pilot sample is less than 5% once the sample size is 60. Adjusting the required sample sizes for the imprecision in the pilot study estimates can result in excessively large definitive RCTs and also requires a pilot sample size of 60 to 90 for the true effect sizes considered here.ConclusionsWe recommend that an external pilot study has at least 70 measured subjects (35 per group) when estimating the SDp for a continuous outcome. If the event rate in an intervention group needs to be estimated by the pilot then a total of 60 to 100 subjects is required. Hence if the primary outcome is binary a total of at least 120 subjects (60 in each group) may be required in the pilot trial. It is very much more efficient to use a larger pilot study, than to guard against the lack of precision by using inflated estimates.

New Aspects in Time-Cost Tradeoff Analysis

AbstractThis paper presents a linear programming model for solution of the time-cost tradeoff problem. Although several analytical models have been developed for time-cost optimization (TCO), many of them mainly focused on projects where the contract duration is fixed. The optimization objective is therefore restricted to identify the minimum total cost only. Another group has primarily focused on project duration minimization. The model presented here considers scheduling characteristics that were ignored in prior research. In the new formulation, variability of funding and uncertainty of project duration are considered together. A chance-constrained programming is used to incorporate the variability of funding, which is quantified by the coefficient of variation. Financial feasibility is expressed as a stochastic constraint, which is transformed into a deterministic equivalent at a prespecified confidence level. Also, project duration uncertainty is incorporated into the model by applying the program ev...

Inference on Optimal Treatment Assignments

We consider inference on optimal treatment assignments. Our methods allow for inference on the treatment assignment rule that would be optimal given knowledge of the population treatment effect in a general setting. The procedure uses multiple hypothesis testing methods to determine a subset of the population for which assignment to treatment can be determined to be optimal after conditioning on all available information, with a prespecified level of confidence. A Monte Carlo study confirms that the inference procedure has good small sample behavior. We apply the method to study Project STAR and the optimal assignment of small class based on school and teacher characteristics.

Time–cost tradeoff analysis considering funding variability and time uncertainty

This paper presents a linear programming model for solution of the time–cost tradeoff problem. Although several analytical models have been developed for time–cost optimization (TCO), some of them mainly focused on projects where the contract duration is fixed. The optimization objective is therefore restricted to identify the minimum total cost only. Another group have primarily focused on project duration minimization. The model presented here considers scheduling characteristics that were ignored simultaneously in prior research. In the new formulation, variability of funding and uncertainty of project duration are considered simultaneously. A chance-constrained programming is used to incorporate the variability of funding, which is quantified by the coefficient of variation. The financial feasibility expressed as a stochastic constraint, which transformed into a deterministic equivalent at a pre-specified confidence level. Also, the project duration uncertainty incorporated into the model by applying PERT in scheduling and then the uncertainty is quantified by the coefficient of variation at a pre-specified confidence level. A system of objective function, which is minimizing direct cost and the group of constraints are solved by means of Lindo software. Two examples are conducted to demonstrate the model performance and its contributions. Four scenarios were adopted in solving the example problems to reflect the effect of each of funding variability and time uncertainty on project cost and duration. The results revealed that with 95% confidence level: 10% variability in funding versus neglecting it, would increase direct cost with 20% approximately for a pre-specified project deadline. Also, 10% variability in time versus neglecting it, would increase duration in range from 16.5% to 30% approximately, for a pre-specified direct cost. Also, considering 10% variability in funding and time would increase direct cost with more than 25% for a pre-specified project deadline. In parallel, an increase in project duration, more than 30% will occur for a pre-specified direct cost.

On Response-Constrained Optimal Experimental Conditions in a Polynomial Mixed Model Setting

This article examines four computational methods to determine optimal conditions under constraints to a specific value or region. The modeling of conditions to response uses a second-order polynomial mixed effects model. The approaches are based on the Delta method, Fieller’s theorem, and two bootstrap methods (Hall’s and Bootstrap-t). These allow us to determine confidence limits for optimal settings of one input factor. Prior information about other input factors is taken into account for prediction of a new observation under constraint of a prespecified threshold and confidence level. Implementation of the approaches is demonstrated for desirable buffer conditions in a monoclonal antibody purification experiment. A total of 115 input combinations were evaluated in the experiment. SE-HPLC aggregate peak percentage is the one output response that is primarily illustrated. Application to this purification experiment data shows apparent discrepancies between Fieller’s method and the others. The robustness of Hall’s method appears in cases of near-zero denominators in the ratio that estimates the optimal bound for pH. Finally, the article discusses features of the methods, including flexibility for a variety of constraint criteria such as controlling a new response above a threshold, and controlling the response average over a set of new observations.

A Simulated Annealing with Ranking and Selection for Stochastic Optimization

We consider the problem of stochastic optimization, where the objective function values are not available and need to be simulated to get their estimates. When the function values are available one can use the simulated annealing algorithm. In this paper, we modify an algorithm that uses the hill climbing feature of simulated annealing with fixed temperature to search the feasible solution set. The proposed algorithm uses indifference zone approach of ranking and selection method to compare the current optimal solution and the potential solution that guarantee the optimal solution with a pre specified level of confidence. The algorithm is tested on a (s, S) inventory problem and compared to other competing algorithm. The numerical results show that the proposed method outperforms the competing method and indeed locate the optimal solution quickly.

Sample size determination for disease prevalence studies with partially validated data

Disease prevalence is an important topic in medical research, and its study is based on data that are obtained by classifying subjects according to whether a disease has been contracted. Classification can be conducted with high-cost gold standard tests or low-cost screening tests, but the latter are subject to the misclassification of subjects. As a compromise between the two, many research studies use partially validated datasets in which all data points are classified by fallible tests, and some of the data points are validated in the sense that they are also classified by the completely accurate gold-standard test. In this article, we investigate the determination of sample sizes for disease prevalence studies with partially validated data. We use two approaches. The first is to find sample sizes that can achieve a pre-specified power of a statistical test at a chosen significance level, and the second is to find sample sizes that can control the width of a confidence interval with a pre-specified confidence level. Empirical studies have been conducted to demonstrate the performance of various testing procedures with the proposed sample sizes. The applicability of the proposed methods are illustrated by a real-data example.

Selecting the best simulated system with weighted control-variate estimators

Ranking and selection (R&S) procedures have been considered an effective tool to solve simulation optimization problems with a discrete and finite decision space. Control variate (CV) is a variance reduction technique that requires no intervention in the way the simulation experiment is performed, and the least-squares regression package needed to implement CV is readily available. In this paper we propose two provably valid selection procedures that employ weighted CV estimators in different ways. Both procedures are guaranteed to select the best system with a prespecified confidence level. Empirical results and simple analyses are performed to compare the efficiency of our new procedures with some existing procedures.

Exact and approximate unconditional confidence intervals for proportion difference in the presence of incomplete data

Confidence interval (CI) construction with respect to proportion/rate difference for paired binary data has become a standard procedure in many clinical trials and medical studies. When the sample size is small and incomplete data are present, asymptotic CIs may be dubious and exact CIs are not yet available. In this article, we propose exact and approximate unconditional test-based methods for constructing CI for proportion/rate difference in the presence of incomplete paired binary data. Approaches based on one- and two-sided Wald's tests will be considered. Unlike asymptotic CI estimators, exact unconditional CI estimators always guarantee their coverage probabilities at or above the pre-specified confidence level. Our empirical studies further show that (i) approximate unconditional CI estimators usually yield shorter expected confidence width (ECW) with their coverage probabilities being well controlled around the pre-specified confidence level; and (ii) the ECWs of the unconditional two-sided-test-based CI estimators are generally narrower than those of the unconditional one-sided-test-based CI estimators. Moreover, ECWs of asymptotic CIs may not necessarily be narrower than those of two-sided-based exact unconditional CIs. Two real examples will be used to illustrate our methodologies.

A Note on Sample Size Determination for the Estimation of the Mean Vector of a Multivariate Population

In this article, we present a straightforward Bonferroni approach for determining sample size for estimating the mean vector of a multivariate population under two scenarios: (1) a pre-specified overall confidence level is desired; and (2) a pre-specified confidence level needs to be guaranteed for each individual variable. It is demonstrated that correlation between variables helps reduce the sample size. The formula to calculate the reduced sample size is derived. A binormal example is presented to illustrate the effect of correlation on sample size reduction for various values of the correlation coefficient.

Ranking inequality: Applications of multivariate subset selection

Inequality measures are often presented in the form of a rank ordering to highlight their relative magnitudes. However, a rank ordering may produce misleading inference, because the inequality measures themselves are statistical estimators with different standard errors, and because a rank ordering necessarily implies multiple comparisons across all measures. Within this setting, if differences between several inequality measures are simultaneously and statistically insignificant, the interpretation of the ranking is changed. This study uses a multivariate subset selection procedure to make simultaneous distinctions across inequality measures at a pre-specified confidence level. Three applications of this procedure are explored using country-level data from the Luxembourg Income Study. The findings show that simultaneous precision plays an important role in relative inequality comparisons and should not be ignored.

Adaptive confidence balls

Adaptive confidence balls are constructed for individual resolution levels as well as the entire mean vector in a multiresolution framework. Finite sample lower bounds are given for the minimum expected squared radius for confidence balls with a prespecified confidence level. The confidence balls are centered on adaptive estimators based on special local block thresholding rules. The radius is derived from an analysis of the loss of this adaptive estimator. In addition adaptive honest confidence balls are constructed which have guaranteed coverage probability over all of $\mathbb{R}^N$ and expected squared radius adapting over a maximum range of Besov bodies.