Statistical Significance Of Treatment Effects Research Articles

Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework

Randomization tests represent a class of significance tests to assess the statistical significance of treatment effects in randomized single-case experiments. Most applications of single-case randomization tests concern simple treatment effects: immediate, abrupt, and permanent changes in the level of the outcome variable. However, researchers are confronted with delayed, gradual, and temporary treatment effects; in general, with “response functions” that are markedly different from single-step functions. We here introduce a general framework that allows specifying a test statistic for a randomization test based on predicted response functions that is sensitive to a wide variety of data patterns beyond immediate and sustained changes in level: different latencies (degrees of delay) of effect, abrupt versus gradual effects, and different durations of the effect (permanent or temporary). There may be reasonable expectations regarding the kind of effect (abrupt or gradual), entailing a different focal data feature (e.g., level or slope). However, the exact amount of latency and the exact duration of a temporary effect may not be known a priori, justifying an exploratory approach studying the effect of specifying different latencies or delayed effects and different durations for temporary effects. We provide illustrations of the proposal with real data, and we present a user-friendly freely available web application implementing it.

Treatment Effect Validation via a Permutation Test in Stata

In this paper we describe the testing procedure for assessing the statistical significance of treatment effect under the experimental conditions of baseline imbalance across covariates and attrition from the survey, using the permutation tests proposed by Freedman and Lane (1983) and Romano and Wolf (2016). We discuss the testing procedure for these hypotheses based on a linear regression model and introduce the new Stata command [R] permtest for the implementation of the permutation test in Stata. Moreover, we investigate the finite-sample performance as well as the statistical validity of the test with a Monte Carlo simulation study in which we examine the empirical size and power properties under the conditions of baseline imbalance and attrition for a fixed number of permutation steps.

Subgroups from regression trees with adjustment for prognostic effects and postselection inference.

Identification of subgroups with differential treatment effects in randomized trials is attracting much attention. Many methods use regression tree algorithms. This article addresses 2 important questions arising from the subgroups: how to ensure that treatment effects in subgroups are not confounded with effects of prognostic variables and how to determine the statistical significance of treatment effects in the subgroups. We address the first question by selectively including linear prognostic effects in the subgroups in a regression tree model. The second question is more difficult because it falls within the subject of postselection inference. We use a bootstrap technique to calibrate normal-theory t intervals so that their expected coverage probability, averaged over all the subgroups in a fitted model, approximates the desired confidence level. It can also provide simultaneous confidence intervals for all subgroups. The first solution is implemented in the GUIDE algorithm and is applicable to data with missing covariate values, 2 or more treatment arms, and outcomes subject to right censoring. Bootstrap calibration is applicable to any subgroup identification method; it is not restricted to regression tree models. Two real examples are used for illustration: a diabetes trial where the outcomes are completely observed but some covariate values are missing and a breast cancer trial where the outcome is right censored.

Objective evaluation of refractive data and astigmatism: quantification and analysis

The aim of this study was to present methods to improve the analysis of refractive data. A comparison of methods is used to analyse refractive powers using individual powers and aggregate data. Equations are also developed for the representation of the average power of a lens or refractive data as a univariate measure, which includes spherical, coma, and/or other aberrations. The equations provide a precise representation of refractive power, which is useful for comparing individual and aggregate data. Average lens power in the principal meridian can be adequately computed as can the average lens power through orthogonal and oblique meridians, providing a good univariate representation of astigmatism and refractive power. Although these formulae are perhaps not as easy to use as, for example, the spherical equivalent, they are more precise and superior in principle involving fewer approximations and are not subject to systematic bias. These effects are of significance when dealing with high-powered lenses such as intraocular lenses or the cornea. They need to be taken into account particularly for calculations of intraocular lens power, toric intraocular lenses, and cornea refractive surgery, especially if outcomes are to be improved. Such issues are of particular importance when dealing with aggregate data and determining statistical significance of treatment effects.

The challenge of a 2-year follow-up after intervention for weight loss in primary care.

Background:Many weight loss programmes show short-term success, but long-term data in larger studies are scarce, especially in community settings. Attrition is common and complicates the interpretation of long-term outcomes.Objective:To investigate 2-year outcomes and explore issues of attrition and missing data.Subjects:A total of 772 overweight and obese adults recruited by primary care practices in Australia, Germany and the UK and randomised to a 12-month weight loss intervention delivered in a commercial programme (CP) or in standard care (SC).Measurement:Weight change from 0–24 and 12–24 months including measured weights only and measured and self-reported weights, using last observation carried forward (LOCF), baseline observation carried forward (BOCF), completers-only and missing-at-random (MAR) analyses.Results:A total of 203 participants completed the 24-month visit. Using measured weights only, there was a trend for greater 24-month weight loss in CP than in SC, but the difference was only statistically significant in the LOCF and BOCF analyses: LOCF: −4.14 vs −1.99 kg, difference adjusted for centre −2.08 kg, P<0.001; BOCF: −1.33 vs −0.74 kg, adjusted difference −0.60 kg, P=0.032; completers: −4.76 vs −2.99 kg, adjusted difference −1.53 kg, P=0.113; missing at random: −3.00 vs −1.94 kg, adjusted difference −1.04 kg, P=0.150. Both groups gained weight from 12–24 months and weight regain was significantly (P<0.001) greater for CP than for SC in all analysis approaches. Inclusion of self-reported weights from a further 138 participants did not change the interpretation of the findings.Conclusion:Initial weight loss was poorly maintained during the no-intervention follow-up, but both groups did have lower weight over the 24 months. Attrition was high in both groups, and assumptions about missing data had considerable impact on the magnitude and statistical significance of treatment effects. It is vital that trials on weight loss interventions consider the plausibility of these differences in an analytical approach when interpreting research findings and comparing data between studies.

Treatment of symptomatic uterine fibroids with green tea extract: a pilot randomized controlled clinical study

BackgroundUterine fibroids (UFs, also known as leiomyoma) affect 70% of reproductive-age women. Imposing a major burden on health-related quality-of-life (HRQL) of premenopausal women, UF is a public health concern. There are no effective medicinal treatment options currently available for women with symptomatic UF.ObjectivesTo evaluate the efficacy and safety of green tea extract (epigallocatechin gallate [EGCG]) on UF burden and quality of life in women with symptomatic UF, in a double-blinded, placebo-controlled randomized clinical trial.MethodsA total of 39 reproductive-age women (age 18–50 years, day 3 serum follicle-stimulating hormone <10 \\U/mL) with symptomatic UF were recruited for this study. All subjects had at least one fibroid lesion 2 cm3 or larger, as confirmed by transvaginal ultrasonography. The subjects were randomized to oral daily treatment with either 800 mg of green tea extract (45% EGCG) or placebo (800 mg of brown rice) for 4 months, and UF volumes were measured at the end, also by transvaginal ultrasonography. The fibroid-specific symptom severity and HRQL of these UF patients were scored at each monthly visit, using the symptom severity and quality-of-life questionnaires. Student’s t-test was used to evaluate statistical significance of treatment effect between the two groups.ResultsOf the final 39 women recruited for the study, 33 were compliant and completed all five visits of the study. In the placebo group (n = 11), fibroid volume increased (24.3%) over the study period; however, patients randomized to green tea extract (n = 22, 800 mg/day) treatment showed significant reduction (32.6%, P = 0.0001) in total UF volume. In addition, EGCG treatment significantly reduced fibroid-specific symptom severity (32.4%, P = 0.0001) and induced significant improvement in HRQL (18.53%, P = 0.01) compared to the placebo group. Anemia also significantly improved by 0.7 g/dL (P = 0.02) in the EGCG treatment group, while average blood loss significantly decreased from 71 mL/month to 45 mL/month (P = 0.001). No adverse effects, endometrial hyperplasia, or other endometrial pathology were observed in either group.ConclusionEGCG shows promise as a safe and effective therapeutic agent for women with symptomatic UFs. Such a simple, inexpensive, and orally administered therapy can improve women’s health globally.

Evaluating Fertilizer Application Strategy for Improving Crop Yields and Reducing Mineral Nitrogen Leaching in High Rainfall Areas of Taiwan

A 2-yr pot experiment conducted on an upland sandy clay loam soil in National Taiwan University, Taipei evaluated crop yields and mineral nitrogen (Nmin) leaching as affected by fertilizer application strategy, viz., inorganic (100% N from urea), organic (100% N from compost), combined (compost plus urea) and a zero control (Ctrl) treatments. Cabbage (Brassica oleracea L.) and corn (Zea mays L.) were grown in annual rotation under simulated high rainfall conditions. The Nmin leaching was monitored adding 83 and 62 mm of water fortnightly to cabbage and corn respectively for a total of 28 times. One-way ANOVA was used to determine the statistical significance of treatment effect and Tukey test for means comparisons (p = 0.05). Treatment differences were subtle for dry matter yields (DMY) but the treatments were strikingly different for 2-yr cumulative Nmin leaching (6.66% in organic, 30.6% in inorganic and 8.10% in combined treatments). This difference was largely related to the ability of soil to an increased retention of added N under organic and combined fertilization treatments. Unlike reported elsewhere, N recoveries were relatively high in compost applied soils, with or without urea fertilizer application. It is concluded that framers’ practices of fertilization that has inherent problem of excessive N leaching in high rainfall areas of Taiwan could be improved by proper compost and urea combinations within agronomically recommended rate of N application.DOI: http://dx.doi.org/10.3126/njst.v12i0.6472 Nepal Journal of Science and Technology 12 (2011) 7-16

Comparing paired vs non‐paired statistical methods of analyses when making inferences about absolute risk reductions in propensity‐score matched samples

Propensity-score matching allows one to reduce the effects of treatment-selection bias or confounding when estimating the effects of treatments when using observational data. Some authors have suggested that methods of inference appropriate for independent samples can be used for assessing the statistical significance of treatment effects when using propensity-score matching. Indeed, many authors in the applied medical literature use methods for independent samples when making inferences about treatment effects using propensity-score matched samples. Dichotomous outcomes are common in healthcare research. In this study, we used Monte Carlo simulations to examine the effect on inferences about risk differences (or absolute risk reductions) when statistical methods for independent samples are used compared with when statistical methods for paired samples are used in propensity-score matched samples. We found that compared with using methods for independent samples, the use of methods for paired samples resulted in: (i) empirical type I error rates that were closer to the advertised rate; (ii) empirical coverage rates of 95 per cent confidence intervals that were closer to the advertised rate; (iii) narrower 95 per cent confidence intervals; and (iv) estimated standard errors that more closely reflected the sampling variability of the estimated risk difference. Differences between the empirical and advertised performance of methods for independent samples were greater when the treatment-selection process was stronger compared with when treatment-selection process was weaker. We recommend using statistical methods for paired samples when using propensity-score matched samples for making inferences on the effect of treatment on the reduction in the probability of an event occurring. Copyright © 2011 John Wiley & Sons, Ltd.

Problems in the reporting of acne clinical trials: a spot check from the 2009 Annual Evidence Update on Acne Vulgaris.

In the course of producing the 2009 NHS Evidence - skin disorders Annual Evidence Update on Acne Vulgaris, 25 randomised controlled trials were examined. From these, at least 12 potentially serious problems of trial reporting were identified. Several trials concluded no effect of a treatment yet they were insufficiently powered to exclude potentially useful benefits. There were examples of duplicate publication and "salami publication", as well as two trials being combined and reported as one. In some cases, an incorrect "within-groups" statistical comparison was made and one trial report omitted original efficacy data and included only P values. Both of the non-inferiority studies examined failed to pre-specify a non-inferiority margin. Trials reported as "double-blind" compared treatments that were dissimilar in appearance or had differing adverse effect profiles. In one case an intention-to-treat analysis was not performed and there was a failure to account for all of the randomized participants. Trial results were made to sound more impressive by selective outcome reporting, emphasizing the statistical significance of treatment effects that were clinically insignificant, and by the use of larger-sounding odds ratios rather than rate ratios for common events. Most of the reporting problems could have been avoided by use of the CONSORT guidelines and prospective trial registration on a public clinical trials database.

Adjusting a Significance Test for Clustering in Designs With Two Levels of Nesting

A common mistake in analysis of cluster randomized experiments is to ignore the effect of clustering and analyze the data as if each treatment group were a simple random sample. This typically leads to an overstatement of the precision of results and anticonservative conclusions about precision and statistical significance of treatment effects. This article gives a simple adjustment to the t statistic that would be computed if clustering were (incorrectly) ignored in an experiment with two levels of nesting (e.g., classrooms and schools) where treatment assignment is made at the highest (e.g., school) level. The adjustment is a multiplicative factor depending on the number of clusters and subclusters, the cluster and subcluster sample sizes, and the cluster and subcluster intraclass correlations ρ S and ρ C. The adjusted t statistic has Student's t distribution with reduced degrees of freedom. The adjusted statistic reduces to the t statistic computed by ignoring clustering when ρ S = ρ C = 0. It reduces to the t statistic computed using cluster means when ρ S = 1. If ρ S and ρ C are between 0 and 1, the adjusted t statistic lies between these two and the degrees of freedom are in between those corresponding to these two extremes.

Anemia-Related Hemogram, Uterine Artery Pulsatility Index, and Blood Pressure for the Effects of Four-Agents-Decoction (Si Wu Tang) in the Treatment of Primary Dysmenorrhea

We previously reported pain intensity as the primary outcome of Four-Agents-Decoction (Si Wu Tang) for treating primary dysmenorrhea. We further explored other effects of this formula in accordance with the theory of Traditional Chinese Medicine on anemia-related hemogram parameters for "nourishing blood" and uterine artery pulsatility index (PI) for "blood stagnation." Data were assessed for post-hoc analysis from 76 young women from the Taipei metropolitan area of Taiwan who participated in the randomized double-blind clinical trial. Analysis of variance and t test were used to examine the treatment effect between and within the Four-Agents-Decoction and placebo groups. In addition to pain intensity, anemia-related hemogram parameters, uterine artery PI, menstruation characteristics, and blood pressure were compared. Compared to placebo, biochemical indicators for anemic status did not change differently after the treatment with Four-Agents-Decoction (Si Wu Tang). The changes in uterine artery PI, menstrual pain intensity, and blood pressure were statistically greater in the active treatment group than in the placebo group. In the 25 women receiving ultrasound examination on the same menstruation day, the directions of the changes in both PI and in visual analog scale (VAS) pain intensity aligned well. The changes in the left PI, right PI, and VAS pain intensity in the Four-Agents-Decoction group were -0.391 +/- 0.861, -0.344 +/- 1.03, and -0.251 +/- 2.02 cm; those in the placebo group were 0.208 +/- 1.10, 0.372 +/- 1.06, and 0.438 +/- 2.98 cm, respectively. Inversely, the changes in blood pressure were 11.4 +/- 10.5 mm Hg for systolic blood pressure and 8.36 +/- 8.65 mm Hg for diastolic pressure for the Four-Agents-Decoction group and 6.36 +/- 8.66 and 2.91 +/- 11.2 mm Hg, respectively, for the placebo group. The directions of the changes did not shift, nor did the values change drastically, after adjusting for the level at screening cycle. The anemia status evaluated in conventional medicine does not readily explain the effect of "nourishing blood" by Four-Agents-Decoction (Si Wu Tang) from this exploratory study. Although we did not find strong statistical significance of treatment effect, the decreased PI and extra blood pressure elevation may indicate an improvement of "blood stagnation" in the Four-Agents-Decoction group. The added information warrants a further larger clinical trial with an increased power to enhance our understanding of the efficacy, the dose-response relationship, and the mode of action of this ancient formula for treating primary dysmenorrhea.

MAID : An effect size based model for microarray data integration across laboratories and platforms

BackgroundGene expression profiling has the potential to unravel molecular mechanisms behind gene regulation and identify gene targets for therapeutic interventions. As microarray technology matures, the number of microarray studies has increased, resulting in many different datasets available for any given disease. The increase in sensitivity and reliability of measurements of gene expression changes can be improved through a systematic integration of different microarray datasets that address the same or similar biological questions.ResultsTraditional effect size models can not be used to integrate array data that directly compare treatment to control samples expressed as log ratios of gene expressions. Here we extend the traditional effect size model to integrate as many array datasets as possible. The extended effect size model (MAID) can integrate any array datatype generated with either single or two channel arrays using either direct or indirect designs across different laboratories and platforms. The model uses two standardized indices, the standard effect size score for experiments with two groups of data, and a new standardized index that measures the difference in gene expression between treatment and control groups for one sample data with replicate arrays. The statistical significance of treatment effect across studies for each gene is determined by appropriate permutation methods depending on the type of data integrated. We apply our method to three different expression datasets from two different laboratories generated using three different array platforms and two different experimental designs. Our results indicate that the proposed integration model produces an increase in statistical power for identifying differentially expressed genes when integrating data across experiments and when compared to other integration models. We also show that genes found to be significant using our data integration method are of direct biological relevance to the three experiments integrated.ConclusionHigh-throughput genomics data provide a rich and complex source of information that could play a key role in deciphering intricate molecular networks behind disease. Here we propose an extension of the traditional effect size model to allow the integration of as many array experiments as possible with the aim of increasing the statistical power for identifying differentially expressed genes.

Correcting a Significance Test for Clustering

A common mistake in analysis of cluster randomized trials is to ignore the effect of clustering and analyze the data as if each treatment group were a simple random sample. This typically leads to an overstatement of the precision of results and anticonservative conclusions about precision and statistical significance of treatment effects. This article gives a simple correction to the t statistic that would be computed if clustering were (incorrectly) ignored. The correction is a multiplicative factor depending on the total sample size, the cluster size, and the intraclass correlation ρ. The corrected t statistic has Student’s t distribution with reduced degrees of freedom. The corrected statistic reduces to the t statistic computed by ignoring clustering when ρ = 0. It reduces to the t statistic computed using cluster means when ρ = 1. If 0 < ρ < 1, it lies between these two, and the degrees of freedom are in between those corresponding to these two extremes.

An experimental approach for quantifying the spatial interactions of plants under different treatment conditions

:Spatial statistics were used in conjunction with standard plot experimentation to form an approach geared towards testing hypotheses about plant spatial interactions under different treatment conditions. Specifically, correlograms were used to quantify the changing patterns in the spatial distribution of plant size (total vegetative length) among pre-planted individuals within a sand box. A repeated-measures ANOVA procedure was then utilized to test for significant differences between the treatment effects as they evolved over time, where the experimental unit was the correlogram value obtained from within each sand box at a specific lag distance. Separate ANOVA tests were also executed for each spatial lag distance and temporal sampling period. In an example demonstrating both procedures, it was shown that a water-stress treatment provoked positive spatial autocorrelation to emerge up to 24 cm from Panicum amarum plants, while a normal water treatment resulted in no change in spatial structure. The difference between the two treatments, and the spatial correlograms of their total vegetative length values, were significant (P = 0.0007). The temporal divergence of the treatments, as spatial patterns evolved, was also significant (P < 0.0001). The approach discussed in this experiment is well suited for evaluating the statistical significance of treatment effects upon the spatial structure of interacting individuals, in both greenhouse and natural settings.

Maximum Likelihood Assessment of Clinical Trials Based on an Ordered Categorical Response

A unified maximum likelihood (ML) methodology is developed for assessing simultaneously both the statistical significance of treatment effects and the model fit when the response variable contains ordered categories. In general, for any treatment by response table it is shown that the better the model fit the more significant [he [realmenl effect. The paper begins by examining the fit for different logit model extensions to data derived under the assumption of an underlying bivariate normal. It is shown that the “parallel log-odds’’ model (I) based on adjacent odds often provides a parsimonious description of the data which fits better than the ‘proportional odds” model (2) based on cumulative odds. Using additional data from a published clinical study, the fit and descriptive utility of more general models (3,4) using an extended ML algorithm (5) are also examined. The paper concludes with some extensions to multi-way tables, where one or more covariates are present.