Magnitude Of Treatment Effect Research Articles

A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis

Objectives: To compare meta-analysis of summary study level data with the equivalent individual patient data (IPD) analysis when interest lies in identification of binary patient characteristics related to treatment efficacy. Design: A simulation study comparing meta-regression with IPD analyses of randomized controlled trials. Methods: Twenty-seven different meta-analysis situations were simulated with 1000 repetitions in each case. The following parameters were varied: (1) the treatment effect magnitude for different patient risk groups; (2) sample sizes of individual studies; and (3) number of studies. The meta-regression and IPD results were then compared for each situation. Results: The statistical power of meta-regression was dramatically and consistently lower than that of IPD analysis, with little agreement between the parameter estimates obtained from the two methods. Only in meta-analyses of large numbers of large trials, did meta-regression detect differential treatment effects between risk groups with any consistency. Conclusions: Meta-analysis of summary data may be adequate when estimating a single pooled treatment effect or investigating study level characteristics. However, when interest lies in investigating whether patient characteristics are related to treatment, IPD analysis will generally be necessary to discover any such relationships. In these situations practitioners should try to obtain individual-level data.

Trial duration and follow-up

The design of most add-on registration trials for anti epileptic drugs (AEDs) usually results in a calculated compromise regarding size, scope, cost, and duration. Appropriate trial duration incorporates issues such as efficacy, safety, and regulatory constraints and is dependent on sample size, titration usage, time to onset, and magnitude of treatment effect. Two phases are usually evaluated in these trials: baseline and treatment. Typical duration for add-on epilepsy registration trials range from 4 to 12 weeks of baseline evaluation followed by 12 to 18 weeks of double-blind treatment. Trials with shorter baseline and/or shorter double-blind periods would likely be less costly, more timely, and result in improved enrollment and retention rates. Data from large dose-response trial of 3 AEDs were retrospectively analyzed previously (Sahlroot and Pledger, 1998) to examine the effect of trial duration on the primary efficacy parameter. 2. Methods

Efficacy and toxicity of old and new disease modifying antirheumatic drugs

The goal of disease modifying antirheumatic drug (DMARD) treatment is “to induce remission and maintain control of inflammatory joint disease”.1 Disease modification has been recommended by Edmonds2 to focus on...

Cocaine medications, cocaine consumption and societal costs.

To estimate the benefits of reduced cocaine consumption in terms of reduced societal costs resulting from the introduction of a medication for cocaine dependence with a small incremental treatment effect. Cost-benefit analysis is applied to study the implications of reduced cocaine consumption. A modelling approach extrapolates the magnitude of treatment effects. Epidemiological data on cocaine use and consumption as well as economic methods of cost-benefit analysis are utilised. Estimates of societal costs associated with heavy users of cocaine, who are most likely addicted and in need of immediate treatment, are developed using 1995 data. In the first analysis, a postulated 1% reduction in consumption of cocaine among heavy users is examined to approximate a small treatment effect, resulting in a minimal consumption benefit. It is estimated that such a reduction would be valued at $US259 million. The cost-benefit analysis indicated that a cocaine medication with a small treatment effect (10 percentage point increase in abstinence rates) would result in a benefit to cost ratio in the range of 1.58 to 5.79, depending on prescribing behaviour and type of patient. Such estimates of the benefits of these small treatment effects are conservative, and they may be biased downwards since the willingness to pay for such a cocaine medication could far exceed the benefit to cost estimation used in this paper. Nevertheless, the substantial benefits found in this paper indicate how important investment in cocaine medication is for public health policy; costs may be reduced with efficient prescribing behaviour. Market and governmental barriers to the utilisation of a cocaine medication could reduce the benefits and increase costs. Clinical trials, cost-effectiveness studies, and cost-benefit studies must be conducted to establish the actual pattern of benefits and costs that could be obtained for an efficacious and effective cocaine medication.

Number needed to treat: a useful new method of assessing the magnitude of treatment effect and its application to the management of diabetic retinopathy.

The magnitude of treatment effect from randomized controlled trials can be measured in various ways. The number needed to treat is a recently described measure that has been shown to offer several advantages in the clinical interpretation and application of reported treatment effects. It quantifies the number of patents that must be treated in order to prevent one patient from developing the specified outcome. The methods necessary to calculate traditional measures of treatment effect, as well as the number needed to treat, are outlined using the management of diabetic retinopathy as an example. The uses and limitations of the number needed to treat are also discussed.

Size is everything--large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects.

Variability in patients' response to interventions in pain and other clinical settings is large. Many explanations such as trial methods, environment or culture have been proposed, but this paper sets out to show that the main cause of the variability may be random chance, and that if trials are small their estimate of magnitude of effect may be incorrect, simply because of the random play of chance. This is highly relevant to the questions of 'How large do trials have to be for statistical accuracy?' and 'How large do trials have to be for their results to be clinically valid?' The true underlying control event rate (CER) and experimental event rate (EER) were determined from single-dose acute pain analgesic trials in over 5000 patients. Trial group size required to obtain statistically significant and clinically relevant (0.95 probability of number-needed-to-treat within -/+0.5 of its true value) results were computed using these values. Ten thousand trials using these CER and EER values were simulated using varying group sizes to investigate the variation due to random chance alone. Most common analgesics have EERs in the range 0.4-0.6 and CER of about 0.19. With such efficacy, to have a 90% chance of obtaining a statistically significant result in the correct direction requires group sizes in the range 30-60. For clinical relevance nearly 500 patients are required in each group. Only with an extremely effective drug (EER > 0.8) will we be reasonably sure of obtaining a clinically relevant NNT with commonly used group sizes of around 40 patients per treatment arm. The simulated trials showed substantial variation in CER and EER, with the probability of obtaining the correct values improving as group size increased. We contend that much of the variability in control and experimental event rates is due to random chance alone. Single small trials are unlikely to be correct. If we want to be sure of getting correct (clinically relevant) results in clinical trials we must study more patients. Credible estimates of clinical efficacy are only likely to come from large trials or from pooling multiple trials of conventional (small) size.

Evidence-based management of patients with osteoporosis.

Evidence-based medicine (EBM) offers an approach to solving clinical problems that places a high value on systematic clinical investigation. Evidence-based clinicians look to the highest rung on a hierarchy of evidence to guide their patient management. When considering therapeutic decisions, randomized control trials examining impact on outcomes that patients feel are important are at the top of the hierarchy of individual studies. Systematic reviews of such trials provide the best evidence for patient care decisions. Systematic reviews include explicit eligibility criteria for studies they include, a comprehensive search, an explicit rating of the methodological quality of the individual trials, and explicit strategies for pooling data. Inferences are weakened if study design is weak (trials are not blinded or we have only observational studies on which to rely), if results are inconsistent across studies, or if studies rely on substitute end points (bone density rather than long-bone fractures). Evidence-based clinicians consider not only the strength of evidence, but the patients' risk of adverse target outcomes and the magnitude of treatment effects in making their therapeutic decisions. EBM encourages quantitative approaches to trading off benefits and risks. For example, in deciding whether to recommend hormone replacement therapy to a 50-yr-old, an evidence-based clinician would consider that the woman has a 15% lifetime risk of fracturing her hip and the median age of the fracture is 79. Observational studies suggest that long-term estrogen therapy will reduce this risk by 25%, and we must therefore treat 25 women for 30 yr to prevent a single fracture. Evidence-based clinicians are also aware that evidence never provides an adequate guide for treatment decisions when considered on its own. Each therapeutic decision involves a trade-off between benefits and risks, and value judgments are invariably involved in making that trade-off.

Issues for covariance analysis of dichotomous and ordered categorical data from randomized clinical trials and non-parametric strategies for addressing them.

Analysis of covariance is an effective method for addressing two considerations for randomized clinical trials. One is reduction of variance for estimates of treatment effects and thereby the production of narrower confidence intervals and more powerful statistical tests. The other is the clarification of the magnitude of treatment effects through adjustment of corresponding estimates for any random imbalances between the treatment groups with respect to the covariables. The statistical basis of covariance analysis can be either non-parametric, with reliance only on the randomization in the study design, or parametric through a statistical model for a postulated sampling process. For non-parametric methods, there are no formal assumptions for how a response variable is related to the covariables, but strong correlation between response and covariables is necessary for variance reduction. Computations for these methods are straightforward through the application of weighted least squares to fit linear models to the differences between treatment groups for the means of the response variable and the covariables jointly with a specification that has null values for the differences that correspond to the covariables. Moreover, such analysis is similarly applicable to dichotomous indicators, ranks or integers for ordered categories, and continuous measurements. Since non-parametric covariance analysis can have many forms, the ones which are planned for a clinical trial need careful specification in its protocol. A limitation of non-parametric analysis is that it does not directly address the magnitude of treatment effects within subgroups based on the covariables or the homogeneity of such effects. For this purpose, a statistical model is needed. When the response criterion is dichotomous or has ordered categories, such a model may have a non-linear nature which determines how covariance adjustment modifies results for treatment effects. Insight concerning such modifications can be gained through their evaluation relative to non-parametric counterparts. Such evaluation usually indicates that alternative ways to compare treatments for a response criterion with adjustment for a set of covariables mutually support the same conclusion about the strength of treatment effects. This robustness is noteworthy since the alternative methods for covariance analysis have substantially different rationales and assumptions. Since findings can differ in important ways across alternative choices for covariables (as opposed to methods for covariance adjustment), the critical consideration for studies with covariance analyses planned as the primary method for comparing treatments is the specification of the covariables in the protocol (or in an amendment or formal plan prior to any unmasking of the study.

Issues for covariance analysis of dichotomous and ordered categorical data from randomized clinical trials and non‐parametric strategies for addressing them

Analysis of covariance is an effective method for addressing two considerations for randomized clinical trials. One is reduction of variance for estimates of treatment effects and thereby the production of narrower confidence intervals and more powerful statistical tests. The other is the clarification of the magnitude of treatment effects through adjustment of corresponding estimates for any random imbalances between the treatment groups with respect to the covariables. The statistical basis of covariance analysis can be either non-parametric, with reliance only on the randomization in the study design, or parametric through a statistical model for a postulated sampling process. For non-parametric methods, there are no formal assumptions for how a response variable is related to the covariables, but strong correlation between response and covariables is necessary for variance reduction. Computations for these methods are straightforward through the application of weighted least squares to fit linear models to the differences between treatment groups for the means of the response variable and the covariables jointly with a specification that has null values for the differences that correspond to the covariables. Moreover, such analysis is similarly applicable to dichotomous indicators, ranks or integers for ordered categories, and continuous measurements. Since non-parametric covariance analysis can have many forms, the ones which are planned for a clinical trial need careful specification in its protocol. A limitation of non-parametric analysis is that it does not directly address the magnitude of treatment effects within subgroups based on the covariables or the homogeneity of such effects. For this purpose, a statistical model is needed. When the response criterion is dichotomous or has ordered categories, such a model may have a non-linear nature which determines how covariance adjustment modifies results for treatment effects. Insight concerning such modifications can be gained through their evaluation relative to non-parametric counterparts. Such evaluation usually indicates that alternative ways to compare treatments for a response criterion with adjustment for a set of covariables mutually support the same conclusion about the strength of treatment effects. This robustness is noteworthy since the alternative methods for covariance analysis have substantially different rationales and assumptions. Since findings can differ in important ways across alternative choices for covariables (as opposed to methods for covariance adjustment), the critical consideration for studies with covariance analyses planned as the primary method for comparing treatments is the specification of the covariables in the protocol (or in an amendment or formal plan prior to any unmasking of the study). © 1998 John Wiley & Sons, Ltd.

Meta-analysis as a clinical tool in nephrology

Meta-analysis as a clinical tool in nephrology

Metaanalysis of the Efficacy and Safety of Ampicillin/Sulbactam in the Treatment of Patients with Bacterial Infections of the Lower Respiratory Tract

The objective of this study was to compare the efficacy and safety of ampicillin/sulbactam with a combined set of cephalosporin antibiotics by using data from seven published studies of similar design. Differences in rates of response for key efficacy variables and in adverse events were calculated from individual studies and weighted by study size to balance differences in magnitude of treatment effects. Of 353 patients treated with ampicillin/sulbactam and 288 patients treated with a comparator drug, totals of 255 ampicillin/sulbactam and 164 comparator drug patients were evaluable for efficacy. Rates of clinical cure favored ampicillin/sulbactam by 9.56% (P = .055), with 60.30% of patients cured, whereas rates of clinical cure or improvement significantly favored ampicillin/sulbactam by 6.21% (P = .019), with 93.30% of patients cured or improved. Other measures of efficacy were equivalent, and the type and incidence of adverse events, including mortality, were comparable in both groups.

Effect of severity of meningitis on fungicidal activity of flucytosine combined with fluconazole in a murine model of cryptococcal meningitis.

We studied the effect of the severity of meningitis on the response to therapy with fluconazole and flucytosine in a murine model of cryptococcal meningitis. Meningitis was established by intracerebral injection of Cryptococcus neoformans. The severity of meningitis was varied by delaying the onset of treatment from 3 to 7 days. Animals were sacrificed after 14 days of treatment, and the numbers of C. neoformans per gram of brain tissue were quantified. The range of effective dose combinations of fluconazole and flucytosine became progressively reduced as the severity of meningitis increased. The magnitude of treatment effect, as measured by the numbers of CFU/gram of brain tissue, was also reduced with increasing severity of meningitis. In this model, as the severity of meningitis increases, higher doses of fluconazole are required to achieve equivalent levels of activity. The combination of fluconazole and flucytosine appears to have the most-potent antifungal effects. This is most readily observed in animals with more-severe meningitis.

The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials

When little or no data directly comparing two treatments are available, investigators often rely on indirect comparisons from studies testing the treatments against a control or placebo. One approach to indirect comparison is to pool findings from the active treatment arms of the original controlled trials. This approach offers no advantage over a comparison of observational study data and is prone to bias. We present an alternative model that evaluates the differences between treatment and placebo in two sets of clinical trials, and preserves the randomization of the originally assigned patient groups. We apply the method to data on sulphamethoxazole-trimethoprim or dapsone/pyrimethamine as prophylaxis against Pneumocystis carinii, in HIV infected patients. The indirect comparison showed substantial increased benefit from the former (odds ratio 0.37, 95% CI 0.21 to 0.65), while direct comparisons from randomized trials suggests a much smaller difference (risk ratio 0.64, 95% CI 0.45 to 0.90; p-value for difference of effect = 0.11). Direct comparisons of treatments should be sought. When direct comparisons are unavailable, indirect comparison meta-analysis should evaluate the magnitude of treatment effects across studies, recognizing the limited strength of inference.

Experimental Field Studies withMya arenaria(Bivalvia) on the Induction and Effect of Hematopoietic Neoplasia

A field experiment was conducted from 1991 to 1992 to examine induction and impact of hematopoietic neoplasia on the marine bivalveMya arenariain southeastern Massachusetts. Clams were collected from Little Buttermilk Bay and separated into three size classes (20–29, 30–39, and 40–49 mm shell length) in the laboratory. These sizes span the range of adults found in the population. A random subsample of these clams was taken to estimate disease prevalence at the start of the experiment, and this was found to be ≤10% in all size classes. Remaining clams were assigned randomly to two groups: Control and Treated. “Controls” were injected with filtered seawater, while “Treated” clams were injected with hemocytes extracted from diseased individuals. Injection of diseased hemocytes was performed to increase disease prevalence in the Treated group. Clams were returned to New Bedford Harbor, a more contaminated field location, where hematopoietic neoplasia is more prevalent, in January 1991, and characteristics of both groups were monitored for 555 days. Among Controls, probability of survival was size-dependent, with higher survival rates in larger clams. Treated clams had a lower probability of survival than Controls, and the magnitude of treatment effect increased with size class. The impact on survival was evident after 89 days, but it was first shown to be statistically significant after 189 days. Among Controls, probability of disease was strongly season-dependent, increasing in the large size class from 0.19 in spring to 0.50 in summer. During summer, Treated clams had a higher probability of being diseased than Controls. Among survivors, no significant sublethal effects due to treatment were detected in the field experiment. Experimental manipulation of disease prevalence may be a useful tool in future studies. In addition to results pertaining to disease, this study obtained long-term growth information, by size class, on somatic and reproductive tissue and shell size.

Prospective meta-analysis of cholesterol-lowering studies: The prospective pravastatin pooling (PPP) project and the cholesterol treatment trialists (CTT) collaboration

Meta-analyses of randomized trials evaluating cholesterol-lowering therapy have demonstrated clear reductions in coronary events and coronary mortality. However, the treatment impact on total mortality has been less certain. With the variable selection of trials and treatment questions, results of meta-analyses have sometimes given conflicting conclusions regarding the magnitude of treatment effects and the populations to whom benefits might accrue. Prospective meta-analysis can avoid these problems by clearly specifying the research questions, eligible studies, analysis plans, and outcome definitions in advance of trial results publication. This approach has been adopted in 2 major prospective meta-analyses of cholesterol-lowering treatments: the Prospective Pravastatin Pooling (PPP) project and the Cholesterol Treatment Trialists (CTT) collaboration. The PPP project is a prospectively planned combined analysis of 3 large-scale pravastatin trials comparing pravastatin against placebo over a minimum 5-year period. The analysis will contain data for > 19,500 patients and should have the power to examine the effects of treatment on total mortality, coronary mortality, and incidence of cancers as well as the ability to look at total coronary events in important subgroups underrepresented in previous trials. The CTT collaboration is a planned prospective meta-analysis of 12 major ongoing or planned randomized trials evaluating therapy with 3-hydroxy-3-methylglutaryl coenzyme A reductase inhibitors, a fibrate, or dietary modification. The trials were prospectively registered and the CTT protocols became final in November 1994. By the year 2000, the CTT collaboration is projected to have information on about 65,000 patients.(ABSTRACT TRUNCATED AT 250 WORDS)

The effects of crop diversification on herbivorous insects: a meta‐analysis approach

Abstract. We used a meta‐analysis to evaluate the hypothesis that diversified crops are subject to lower density of herbivorous insects. This method consists of integrating findings of independent studies by calculating the magnitude of treatment effects (effect size). A literature review covering a period of over 10 years yielded twenty‐one studies that contained sufficient information for the meta‐analysis. The data were analysed according to three criteria: yearly replications of the same study were considered independent data so that research repeated over time would have greater weight; a second meta‐analysis was conducted with results of only 1 year because of possible lack of independence between results of the same study; the third approach was to remove studies involving Brassica spp. from the analysis because they are redundant from the standpoint of testing a general hypothesis of crop diversification. The three analyses resulted in effect sizes of 0.50, 0.35 and 0.27, respectively, and all were statistically significant. These results correspond to 69%, 63% and 60% differences between average insect densities for treatment (diversified crops) and control groups. We considered the last criterion for data analysis as the most appropriate because of lowest bias from non‐independence between samples. Therefore we concluded that crop diversity caused moderate reduction of herbivorous insect populations. The implications and limitations of meta‐analyses are discussed.

Meta-analysis: the fashion of summing-up evidence. Part II: Interpretations and uses.

In this commentary, we use evidence produced by the Early Breast Cancer Trialists' Collaborative Group (EBCTCG) ten-year update of a meta-analysis of trials of adjuvant therapies for early breast cancer which started prior to 1985 to illustrate aspects of interpretations and uses of meta-analysis results. The following issues are discussed: i) The meta-analysis provides an average summary for the effect of a treatment. Greater statistical power is obtained by increasing the number of events contributing to the analysis. However, summing up the results of various trials necessitates the loss of individual information concerning the magnitude of treatment effects which depend on tumor- and patient-related factors. Subgroup analyses within the meta-analysis process allow some recovery of such features; ii) The absolute benefit obtained from an effective treatment depends not only on the relative benefit of the treatment but also on the prognosis of the individual patients; iii) The results are more immediately applicable if less reliance is placed on the arithmetic construct inherent in the overview, using instead unconfounded information about the value of treatments actually administered. This avoids the need to extrapolate the effect for one component of the therapy by assuming a lack of interaction with its other components; iv) Although indirect comparisons between different meta-analyses are regularly made to pick the "winner" from among tested treatment modalities, it is unlikely that the optimal therapeutic regimen can be defined via such indirect comparisons, though such comparisons may raise interesting, testable hypotheses.

How much 'better' is good enough? The magnitude of treatment effect in clinical trials.

Among the various factors required to calculate sample size for clinical trials, the magnitude of treatment effect anticipated is an important component. The objective of this report is to present some of the complexities involved in selection of treatment effect size in clinical trials. As a framework for discussion, an analysis of published reports related to surfactant therapy was carried out. Twenty-one consecutive exogenous surfactant trials for neonatal respiratory distress syndrome were analyzed. The "Methods" sections were reviewed for evaluating various components of sample size calculation, including the anticipated treatment effect size. Sixteen (76%) of the 21 reports provided a description of sample size calculations, and 12 of these gave some reasons for the choice of the anticipated treatment effect size. Expressed as percent change, the median treatment effect from intervention anticipated by the investigators was 50% (range, 15% to 90%), with a positively skewed distribution. The actual median percent reduction in adverse events from treatment (as compared with baseline) was 36% (range, 75% reduction to 5% excess). When the treatment effect was expressed as difference in adverse event rate, in the 14 (of 16) trials that could be analyzed, the median observed reduction in adverse events (death, bronchopulmonary dysplasia, or occurrence of respiratory distress syndrome) was 14.5% (range, 52% reduction to 2% excess). All trials except one concluded, however, that the intervention was effective, mostly based on additional subgroup calculations. Researchers often select sample sizes capable of detecting only large treatment effects, thus risking type II error, although sometimes a much smaller effect could be clinically important. While pragmatic considerations must be considered during the design of randomized clinical trials, researchers ought to present a rationale for anticipating a given magnitude of treatment effect in their sample size calculations. It may be possible to consider innovative trial designs that help determine the most appropriate treatment choice with the least possible sample size.

Meta-analysis of the effectiveness of prophylactic drug therapy in preventing supraventricular arrhythmia early after coronary artery bypass grafting

There has been considerable interest in evaluating the usefulness of drug therapy in patients who undergo operative coronary revascularization procedures to prevent the development of supraventricular tachyarrhythmia after surgery. 1,2 Much concern stems from the fact that postoperative atrial fibrillation prolongs hospitalization, thus increasing the cost of this already expensive and frequently used procedure. 3 There is an attendant risk of severe complications of atrial fibrillation (most importantly stroke), which necessitates anticoagulation in a significant percentage of these patients. 4 Furthermore, the medications used to convert atrial fibrillation to sinus rhythm and to prevent recurrence are not innocuous, and arguably may increase the risk of death during follow-up. 5 For all these reasons, it is worthwhile to investigate whether medication, administered either before or immediately after the procedure, may prevent atrial fibrillation and other supraventricular arrhythmias. Currently, the most frequently used medications for preventing the development of supraventricular tachyarrhythmia after surgery are β-adrenergic blocking agents and digitalis. Several studies have been undertaken to study their effectiveness. 6–17 Unfortunately, no consensus opinion has been reached regarding their relative usefulness. In fact, there has been considerable inconsistency of results. Consequently, in most hospitals there is no uniform policy regarding their routine use. Much of the problem occurs because of the small number of patients in each study, and the relatively modest treatment effect in most of these. The technique of meta-analysis can often help to interpret this kind of diverse data. 18 By combining the information obtainable from a number of small but well-done studies, it is possible to draw some conclusions about the relative magnitude of treatment effect. We reviewed 12 full-length reports in which digitalis or β-blocker therapy, or both, was compared with control therapy, combined the information and herein report the results.

Social skills training with schizophrenics: A meta-analytic evaluation.

A meta-analytic review of 27 studies on social skills training with schizophrenics was conducted to address 3 critical issues in the literature: (a) the magnitude of treatment effects relative to different outcome measures; (b) the extent of the generalization and maintenance of treatment effects; and (c) the impact of 2 methodological issues: diagnostic clarity and training variations. Social skills training has a strong, positive impact on behavioral measures of social skill, self-rated assertiveness, and hospital discharge rate, and a moderate impact on relapse rate. In addition, when behavioral measures are used, the data support generalization and maintenance of skill gains. However, effects are only marginally significant for broader ratings of symptoms and functioning. Diagnostic homogeneity, the number of techniques used, and the amount of training do not appear to be significantly associated with outcome. Specific directions for future research are discussed.