Re-estimation Procedure Research Articles

Blinded and unblinded internal pilot study designs for clinical trials with count data

Internal pilot studies are a popular design feature to address uncertainties in the sample size calculations caused by vague information on nuisance parameters. Despite their popularity, only very recently blinded sample size reestimation procedures for trials with count data were proposed and their properties systematically investigated. Although blinded procedures are favored by regulatory authorities, practical application is somewhat limited by fears that blinded procedures are prone to bias if the treatment effect was misspecified in the planning. Here, we compare unblinded and blinded procedures with respect to bias, error rates, and sample size distribution. We find that both procedures maintain the desired power and that the unblinded procedure is slightly liberal whereas the actual significance level of the blinded procedure is close to the nominal level. Furthermore, we show that in situations where uncertainty about the assumed treatment effect exists, the blinded estimator of the control event rate is biased in contrast to the unblinded estimator, which results in differences in mean sample sizes in favor of the unblinded procedure. However, these differences are rather small compared to the deviations of the mean sample sizes from the sample size required to detect the true, but unknown effect. We demonstrate that the variation of the sample size resulting from the blinded procedure is in many practically relevant situations considerably smaller than the one of the unblinded procedures. The methods are extended to overdispersed counts using a quasi-likelihood approach and are illustrated by trials in relapsing multiple sclerosis.

Comparison of Operating Characteristics of Commonly Used Sample Size Re-Estimation Procedures in a Two-Stage Design

In group sequential clinical trials, there are several sample size re-estimation methods proposed in the literature that allow for change of sample size at the interim analysis. Most of these methods are based on either the conditional error function or the interim effect size. Our simulation studies compared the operating characteristics of three commonly used sample size re-estimation methods, Chen et al. (2004), Cui et al. (1999), and Muller and Schafer (2001). Gao et al. (2008) extended the CDL method and provided an analytical expression of lower and upper threshold of conditional power where the type I error is preserved. Recently, Mehta and Pocock (2010) extensively discussed that the real benefit of the adaptive approach is to invest the sample size resources in stages and increasing the sample size only if the interim results are in the so called “promising zone” which they define in their article. We incorporated this concept in our simulations while comparing the three methods. To test the robustness of these methods, we explored the impact of incorrect variance assumption on the operating characteristics. We found that the operating characteristics of the three methods are very comparable. In addition, the concept of promising zone, as suggested by MP, gives the desired power and smaller average sample size, and thus increases the efficiency of the trial design.

Blinded sample size re‐estimation in superiority and noninferiority trials: bias versus variance in variance estimation

The internal pilot study design allows for modifying the sample size during an ongoing study based on a blinded estimate of the variance thus maintaining the trial integrity. Various blinded sample size re-estimation procedures have been proposed in the literature. We compare the blinded sample size re-estimation procedures based on the one-sample variance of the pooled data with a blinded procedure using the randomization block information with respect to bias and variance of the variance estimators, and the distribution of the resulting sample sizes, power, and actual type I error rate. For reference, sample size re-estimation based on the unblinded variance is also included in the comparison. It is shown that using an unbiased variance estimator (such as the one using the randomization block information) for sample size re-estimation does not guarantee that the desired power is achieved. Moreover, in situations that are common in clinical trials, the variance estimator that employs the randomization block length shows a higher variability than the simple one-sample estimator and in turn the sample size resulting from the related re-estimation procedure. This higher variability can lead to a lower power as was demonstrated in the setting of noninferiority trials. In summary, the one-sample estimator obtained from the pooled data is extremely simple to apply, shows good performance, and is therefore recommended for application.

Blinded Continuous Monitoring of Nuisance Parameters in Clinical Trials

SummaryDetermination of a clinical trial's size is an important task in the planning of any trial because of the direct implications of the sample size on feasibility, costs and timelines. However, sample size calculations are often subject to substantial uncertainty due to limited prior information on the size of nuisance parameters such as variances or event rates. Continuous monitoring of the nuisance parameter in clinical trials has been proposed as a tool to size trials appropriately. With this approach, the nuisance parameter is continuously monitored during the trial. The trial is stopped when the actual estimate for the nuisance parameter and sample size fulfil a stopping criterion. Continuous monitoring can therefore be viewed as a stochastic process with stopping time. We describe the bias that occurs with unblinded continuous monitoring of the variance in clinical trials by means of a simulation study. Then we propose a procedure for blinded continuous monitoring that does not require breaking the treatment code during the on-going study and show that the procedure does not suffer from the same biases as observed in unblinded monitoring. Results on the performance properties of such designs are given and the designs are compared with blinded re-estimation procedures with a single data look. By means of asymptotic theoretical arguments and finite sample size simulations we find that the variability in sample size is smaller with blinded continuous monitoring than with blinded sample size re-estimation whenever the power for both designs is close to the target value. Repeated sample size re-estimation is in between continuous monitoring and sample size re-estimation in this respect. Furthermore, we present a hypertension trial where blinded sample size re-estimation with a single data look was applied and we investigate the properties of blinded continuous monitoring in this setting. Finally we close with a brief discussion.

A practical comparison of blinded methods for sample size reviews in survival data clinical trials

This paper presents practical approaches to the problem of sample size re-estimation in the case of clinical trials with survival data when proportional hazards can be assumed. When data are readily available at the time of the review, on a full range of survival experiences across the recruited patients, it is shown that, as expected, performing a blinded re-estimation procedure is straightforward and can help to maintain the trial's pre-specified error rates. Two alternative methods for dealing with the situation where limited survival experiences are available at the time of the sample size review are then presented and compared. In this instance, extrapolation is required in order to undertake the sample size re-estimation. Worked examples, together with results from a simulation study are described. It is concluded that, as in the standard case, use of either extrapolation approach successfully protects the trial error rates.

Blinded sample size re-estimation in clinical trials comparing several treatments

An important question that arises in clinical trials is how many additional observations, if any, are required beyond those originally planned. This has satisfactorily been answered in the case of two-treatment double-blind clinical experiments. However, one may be interested in comparing a new treatment with its competitors, which may be more than one. This problem is addressed in this investigation involving responses from arbitrary distributions, in which the mean and the variance are not functionally related. First, a solution in determining the initial sample size for specified level of significance and power at a specified alternative is obtained. Then it is shown that when the initial sample size is large, the nominal level of significance and the power at the pre-specified alternative are fairly robust for the proposed sample size re-estimation procedure. An application of the results is made to the blood coagulation functionality problem considered by Kropf et al. [Multiple comparisons of treatments with stable multivariate tests in a two-stage adaptive design, including a test for non-inferiority, Biom. J. 42(8) (2000), pp. 951–965].

Trends in annualized relapse rates in relapsing–remitting multiple sclerosis and consequences for clinical trial design

Background: Sample size calculation is a key aspect in the planning of any trial. Planning a randomized placebo-controlled trial in relapsing–remitting multiple sclerosis (RRMS) requires knowledge of the annualized relapse rate (ARR) in the placebo group. Objectives: This paper aims (i) to characterize the uncertainty in ARR by conducting a systematic review of placebo-controlled, randomized trials in RRMS and by modelling the ARR over time; and (ii) to assess the feasibility and utility of blinded sample size re-estimation (BSSR) procedures in RRMS. Methods: A systematic literature review was carried out by searching PubMed, Ovid Medline and the Cochrane Register of Controlled Trials. The placebo ARRs were modelled by negative binomial regression. Computer simulations were conducted to assess the utility of BSSR in RRMS. Results: Data from 26 placebo-controlled randomized trials were included in this analysis. The placebo ARR decreased by 6.2% per year (p < 0.0001; 95% CI (4.2%; 8.1%)) resulting in substantial uncertainty in the planning of future trials. BSSR was shown to be feasible and to maintain power at a prespecified level also if the ARR was misspecified in the planning phase. Conclusions: Our investigations confirmed previously reported trends in ARR. In this context adaptive strategies such as BSSR designs are recommended for consideration in the planning of future trials in RRMS.

Bayesian shrinkage analysis of QTLs under shape-adaptive shrinkage priors, and accurate re-estimation of genetic effects.

The successful implementation of Bayesian shrinkage analysis of high-dimensional regression models, as often encountered in quantitative trait locus (QTL) mapping, is contingent upon the choice of suitable sparsity-inducing priors. In practice, the shape (that is, the rate of tail decay) of such priors is typically preset, with no regard for the range of plausible alternatives and the fact that the most appropriate shape may depend on the data at hand. This study is presumably the first attempt to tackle this oversight through the shape-adaptive shrinkage prior (SASP) approach, with a focus on the mapping of QTLs in experimental crosses. Simulation results showed that the separation between genuine QTL effects and spurious ones can be made clearer using the SASP-based approach as compared with existing competitors. This feature makes our new method a promising approach to QTL mapping, where good separation is the ultimate goal. We also discuss a re-estimation procedure intended to improve the accuracy of the estimated genetic effects of detected QTLs with regard to shrinkage-induced bias, which may be particularly important in large-scale models with collinear predictors. The re-estimation procedure is relevant to any shrinkage method, and is potentially valuable for many scientific disciplines such as bioinformatics and quantitative genetics, where oversaturated models are booming.

Blinded Sample Size Reestimation with Negative Binomial Counts in Superiority and Non-inferiority Trials

In the planning of clinical trials with count outcomes such as the number of exacerbations in chronic obstructive pulmonary disease (COPD) often considerable uncertainty exists with regard to the overall event rate and the level of overdispersion which are both crucial for sample size calculations. To develop a sample size reestimation strategy that maintains the blinding of the trial, controls the type I error rate and is robust against misspecification of the nuisance parameters in the planning phase in that the actual power is close to the target. The operation characteristics of the developed sample size reestimation procedure are investigated in a Monte Carlo simulation study. Estimators of the overall event rate and the overdispersion parameter that do not require unblinding can be used to effectively adjust the sample size without inflating the type I error rate while providing power values close to the target. If only little information is available regarding the size of the overall event rate and the overdispersion parameter in the design phase of a trial, we recommend the use of a design with sample size reestimation as the one suggested here. Trials in COPD are expected to benefit from the proposed sample size reestimation strategy.

Operating characteristics of sample size re-estimation with futility stopping based on conditional power

Various methods have been described for re-estimating the final sample size in a clinical trial based on an interim assessment of the treatment effect. Many re-weight the observations after re-sizing so as to control the pursuant inflation in the type I error probability alpha. Lan and Trost (Estimation of parameters and sample size re-estimation. Proceedings of the American Statistical Association Biopharmaceutical Section 1997; 48-51) proposed a simple procedure based on conditional power calculated under the current trend in the data (CPT). The study is terminated for futility if CPT < or = CL, continued unchanged if CPT > or = CU, or re-sized by a factor m to yield CPT = CU if CL < CPT < CU, where CL and CU are pre-specified probability levels. The overall level alpha can be preserved since the reduction due to stopping for futility can balance the inflation due to sample size re-estimation, thus permitting any form of final analysis with no re-weighting. Herein the statistical properties of this approach are described including an evaluation of the probabilities of stopping for futility or re-sizing, the distribution of the re-sizing factor m, and the unconditional type I and II error probabilities alpha and beta. Since futility stopping does not allow a type I error but commits a type II error, then as the probability of stopping for futility increases, alpha decreases and beta increases. An iterative procedure is described for choice of the critical test value and the futility stopping boundary so as to ensure that specified alpha and beta are obtained. However, inflation in beta is controlled by reducing the probability of futility stopping, that in turn dramatically increases the possible re-sizing factor m. The procedure is also generalized to limit the maximum sample size inflation factor, such as at m max = 4. However, doing so then allows for a non-trivial fraction of studies to be re-sized at this level that still have low conditional power. These properties also apply to other methods for sample size re-estimation with a provision for stopping for futility. Sample size re-estimation procedures should be used with caution and the impact on the overall type II error probability should be assessed.

Applying Sparse KPCA for Feature Extraction in Speech Recognition

This paper presents an analysis of the applicability of Sparse Kernel Principal Component Analysis (SKPCA) for feature extraction in speech recognition, as well as, a proposed approach to make the SKPCA technique realizable for a large amount of training data, which is an usual context in speech recognition systems. Although the KPCA (Kernel Principal Component Analysis) has proved to be an efficient technique for being applied to speech recognition, it has the disadvantage of requiring training data reduction, when its amount is excessively large. This data reduction is important to avoid computational unfeasibility and/or an extremely high computational burden related to the feature representation step of the training and the test data evaluations. The standard approach to perform this data reduction is to randomly choose frames from the original data set, which does not necessarily provide a good statistical representation of the original data set. In order to solve this problem a likelihood related re-estimation procedure was applied to the KPCA framework, thus creating the SKPCA, which nevertheless is not realizable for large training databases. The proposed approach consists in clustering the training data and applying to these clusters a SKPCA like data reduction technique generating the reduced data clusters. These reduced data clusters are merged and reduced in a recursive procedure until just one cluster is obtained, making the SKPCA approach realizable for a large amount of training data. The experimental results show the efficiency of SKPCA technique with the proposed approach over the KPCA with the standard sparse solution using randomly chosen frames and the standard feature extraction techniques.

Regularized reestimation of stochastic duration models for phone-classification

Recent research has compared the performance of various distributions (uniform, boxcar, exponential, gamma, discrete) for modeling segment (state) durations in hidden semi-Markov models used for phone classification on the TIMIT database. These experiments have shown that a gamma distribution is more appropriate than exponential (which is implicit in first-order Markov models), and achieved a 3% relative reduction in phone-classification errors [Jackson, Proc. ICPhS, pp. 1349–1352 (2003)]. The parameters of these duration distributions were estimated once for each model from initial statistics of state occupation (offline), and remained unchanged during subsequent iterations of training. The present work investigates the effect of reestimating the duration models in training (online) with respect to the phone-classification scores. First, tests were conducted on duration models reestimated directly from statistics gathered in the previous iteration of training. It was found that the boxcar and gamma models were unstable, meanwhile the performance of the other models also tended to degrade. Secondary tests, using a scheme of annealed regularization, demonstrated that the losses could be recouped and a further 1% improvement was obtained. The results from this pilot study imply that similar gains in recognition accuracy deserve investigation, along with further optimization of the duration model reestimation procedure.

Basecalling using hidden Markov models

In this paper we propose hidden Markov models to model electropherograms from DNA sequencing equipment and perform basecalling. We model the state emission densities using artificial neural networks, and modify the Baum–Welch reestimation procedure to perform training. Moreover, we develop a method that exploits consensus sequences to label training data, thus minimizing the need for hand labeling. We propose the same method for locating an electropherogram in a longer DNA sequence. We also perform a careful study of the basecalling errors and propose alternative HMM topologies that might further improve performance. Our results demonstrate the potential of these models. Based on these results, we conclude by suggesting further research directions.

Robustness of sample size re-estimation procedure in clinical trials (arbitrary populations).

In clinical trials, one of the main questions that is being asked is how many additional observations, if any, are needed beyond those originally planned. In a two-treatment double-blind clinical experiment, one is interested in testing the null hypothesis of equality of the means against one-sided alternative when the common variance sigma2 is unknown. We wish to determine the required total sample size when the error probabilities alpha and beta are specified at a predetermined alternative. Shih provided a two-stage procedure which is an extension of Stein's one-sample procedure, assuming normal response. He estimates sigma2 by the method of maximum likelihood via the EM algorithm and carries out a simulation study in order to evaluate the effective level of significance and the power. The author proposed a closed-form estimator for sigma2 and showed analytically that the difference between the effective and nominal levels of significance is negligible and that the power exceeds 1-beta when the initial sample size is large. Here we consider responses from arbitrary distributions in which the mean and the variance are not functionally related and show that when the initial sample size is large, the conclusions drawn previously by the author still hold. The effective coverage probability of a fixed-width interval is also evaluated. Proofs of certain assertions are deferred to the Appendix.

Structured language modeling

This paper presents an attempt at using the syntactic structure in natural language for improved language models for speech recognition. The structured language model merges techniques in automatic parsing and language modeling using an original probabilistic parameterization of a shift-reduce parser. A maximum likelihood re-estimation procedure belonging to the class of expectation-maximization algorithms is employed for training the model. Experiments on the Wall Street Journal and Switchboard corpora show improvement in both perplexity and word error rate—word lattice rescoring—over the standard 3-gram language model.

Blinded sample size re-estimation with unequal variances and center effects in clinical trials

The two-stage sample size re-estimation procedure developed previously in the literature (e.g.,Shih, 1992; Gould and Shih, 1992) treats the blinded data from a randomized clinical trial as a sample from a mixture of two normal distributions of equal variances.This paper investigates the procedure when being applied to the situations where the variances are unequal and blocking effects by clinical centers are present.We show by simulations that in both situations the re-estimation procedure is still capable to update the sample size properly for attaining the required power and preserving the type-1 error at the nominal level.

The semi-relaxed algorithm for estimating parameters of hidden Markov models

For hidden Markov model (HMM) based speech recognition where the basic speech unit is smaller than the recognizer's output unit, the standard full Baum-Welch re-estimation procedure for the HMM training is very costly in computation. This is because it requires evaluation of the HMM output densities and of the forward/backward probabilities in the entire region of the state-frame trellis. In this paper, we present an algorithm which exploits the fact that the entries of the trellis are essentially zero except near the block diagonal and hence achieves significant computational saving. The algorithm is evaluated in experiments with a large vocabulary word recognizer based on mixture-density HMM representation of phonemes. The HMM parameters trained with the new algorithm are essentially identical to those trained with the full Baum-Welch algorithm in that the resulting HMMs have nearly the same likelihood values on the same set of training data. Identical word recognition accuracies are yielded using the HMMs trained with the two algorithms. However, the new algorithm is shown to be about an order of magnitude faster than the full Baum-Welch algorithm.

An extended role for thermoluminescent phosphors in personnel environmental and accident dosimetry using sensitisation, re-estimation and fast fading

This paper summarises some techniques for extending the usefulness of conventional phosphors in personnel, environmental and accident dosimetry. An optimised procedure for utilising radiation sensitisation and UV re-estimation in thermoluminescent LiF is presented. In particular it is shown that optimum performance is achieved by using a UV wavelength of 250 ± 10 nm for both the UV/thermal anneal following sensitisation, and for the UV re-estimation procedure. In the case of Harshaw LiF chips (3×3×0.9 mm 3) the sensitivity is increased by a factor of 4–5 to achieve a minimum detectable dose of ∼ 10 μGy (2 σ) and a minimum re-estimable dose of 50–100 mGy (2σ), dependent on batch. Sensitized LiF also exhibits improved tissue equivalence, extended linearity and improved precision at low doses. The information from fast-fading glow peaks, which is normally rejected, is shown to have a useful application to the evaluation of short-term increases in environmental dose rates such as may occur following accidental releases of radioactivity.