Treatment-covariate Interactions Research Articles

Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique

Freedman [Adv. in Appl. Math. 40 (2008) 180–193; Ann. Appl. Stat. 2 (2008) 176–196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman’s model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fixed. OLS adjustment cannot hurt asymptotic precision when a full set of treatment–covariate interactions is included. Asymptotically valid confidence intervals can be constructed with the Huber–White sandwich standard error estimator. Checks on the asymptotic approximations are illustrated with data from Angrist, Lang, and Oreopoulos’s [Am. Econ. J.: Appl. Econ. 1:1 (2009) 136–163] evaluation of strategies to improve college students’ achievement. The strongest reasons to support Freedman’s preference for unadjusted estimates are transparency and the dangers of specification search.

Mixed treatment comparisons using aggregate and individual participant level data

Mixed treatment comparisons (MTC) extend the traditional pair-wise meta-analytic framework to synthesize information on more than two interventions. Although most MTCs use aggregate data (AD), a proportion of the evidence base might be available at the individual level (IPD). We develop a series of novel Bayesian statistical MTC models to allow for the simultaneous synthesis of IPD and AD, potentially incorporating study and individual level covariates. The effectiveness of different interventions to increase the provision of functioning smoke alarms in households with children was used as a motivating dataset. This included 20 studies (11 AD and 9 IPD), including 11 500 participants. Incorporating the IPD into the network allowed the inclusion of information on subject level covariates, which produced markedly more accurate treatment-covariate interaction estimates than an analysis solely on the AD from all studies. Including evidence at the IPD level in the MTC is desirable when exploring participant level covariates; even when IPD is available only for a fraction of the studies. Such modelling may not only reduce inconsistencies within networks of trials but also assist the estimation of intervention subgroup effects to guide more individualised treatment decisions.

Using aggregate data to estimate the standard error of a treatment–covariate interaction in an individual patient data meta‐analysis

Subgroup analyses are important to medical research because they shed light on the heterogeneity of treatment effectts. A treatment-covariate interaction in an individual patient data (IPD) meta-analysis is the most reliable means to estimate how a subgroup factor modifies a treatment's effectiveness. However, owing to the challenges in collecting participant data, an approach based on aggregate data might be the only option. In these circumstances, it would be useful to assess the relative efficiency and power loss of a subgroup analysis without patient-level data. We present methods that use aggregate data to estimate the standard error of an IPD meta-analysis' treatment-covariate interaction for regression models of a continuous or dichotomous patient outcome. Numerical studies indicate that the estimators have good accuracy. An application to a previously published meta-regression illustrates the practical utility of the methodology.

Individual patient data meta-analysis of survival data using Poisson regression models

BackgroundAn Individual Patient Data (IPD) meta-analysis is often considered the gold-standard for synthesising survival data from clinical trials. An IPD meta-analysis can be achieved by either a two-stage or a one-stage approach, depending on whether the trials are analysed separately or simultaneously. A range of one-stage hierarchical Cox models have been previously proposed, but these are known to be computationally intensive and are not currently available in all standard statistical software. We describe an alternative approach using Poisson based Generalised Linear Models (GLMs).MethodsWe illustrate, through application and simulation, the Poisson approach both classically and in a Bayesian framework, in two-stage and one-stage approaches. We outline the benefits of our one-stage approach through extension to modelling treatment-covariate interactions and non-proportional hazards. Ten trials of hypertension treatment, with all-cause death the outcome of interest, are used to apply and assess the approach.ResultsWe show that the Poisson approach obtains almost identical estimates to the Cox model, is additionally computationally efficient and directly estimates the baseline hazard. Some downward bias is observed in classical estimates of the heterogeneity in the treatment effect, with improved performance from the Bayesian approach.ConclusionOur approach provides a highly flexible and computationally efficient framework, available in all standard statistical software, to the investigation of not only heterogeneity, but the presence of non-proportional hazards and treatment effect modifiers.

Effects of adult inpatient group psychotherapy on attachment characteristics: An observational study comparing routine care to an untreated comparison group

This study investigated the changes in attachment characteristics of patients undergoing inpatient group psychotherapy in routine care. We collected data from 265 consecutively recruited patients and 260 non-clinical control persons using self-report measures of attachment, depression, and socio-demographic characteristics. The effects of treatment on patients were analyzed using propensity score techniques (propensity strata and logit-transformed propensity scores) in combination with a generalized analysis of covariance. A moderate increase of attachment security was found which could be attributed to a decrease both in attachment anxiety and avoidance. Pre-post improvements in attachment with regard to romantic partnerships were stable after a 1-year follow-up. Furthermore, we found significant treatment-covariate interactions indicating that subjects with particularly high treatment propensities (propensities were highly correlated with depression and attachment anxiety) improved the most in terms of attachment security. Our results are encouraging for psychotherapeutic practice in that they provide evidence that long-term attachment improvements can be reached via psychotherapy. Our results will also provide a sound basis for future studies in the field of clinical attachment research, e.g., studies examining whether improved attachment security is correlated to symptom improvements in different psychological disorders.

A Simple Method for Variable Selection in Regression with Respect to Treatment Selection

In this paper, we compare the method of Gunter et al. (2011) for variable selection in treatment comparison analysis (an approach to regression analysis where treatment-covariate interactions are deemed important) with a simple stepwise selection method that we introduce. The stepwise method has several advantages, most notably its generalization to regression models that are not necessarily linear, its simplicity and its intuitive nature. We show that the new simple method works surprisingly well compared to the more complex method when compared in the linear regression framework. We use four generative models (explicitly detailed in the paper) for the simulations and compare spuriously identified interactions and where applicable (generative models 3 and 4) correctly identified interactions. We also apply the new method to logistic regression and Poisson regression and illustrate its performance in Table 2 in the paper. The simple method can be applied to other types of regression models including various other generalized linear models, Cox proportional hazard models and nonlinear models.

A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners

Objective Treatments may be more effective in some patients than others, and individual participant data (IPD) meta-analysis of randomized trials provides perhaps the best method of investigating treatment-covariate interactions. Various methods are used; we provide a comprehensive critique and develop guidance on method selection. Study Design and Setting We searched MEDLINE to identify all frequentist methods and appraised them for simplicity, risk of bias, and power. IPD data sets were reanalyzed. Results Four methodological categories were identified: PWT: pooling of within-trial covariate interactions; OSM: “one-stage” model with a treatment-covariate interaction term; TDCS: testing for difference between covariate subgroups in their pooled treatment effects; and CWA: combining PWT with meta-regression. Distinguishing across- and within-trial information is important, as the former may be subject to ecological bias. A strategy is proposed for method selection in different circumstances; PWT or CWA are natural first steps. The OSM method allows for more complex analyses; TDCS should be avoided. Our reanalysis shows that different methods can lead to substantively different findings. Conclusion The choice of method for investigating interactions in IPD meta-analysis is driven mainly by whether across-trial information is considered for inclusion, a decision, which depends on balancing possible improvement in power with an increased risk of bias.

A small sample study of the STEPP approach to assessing treatment–covariate interactions in survival data

A new, intuitive method has recently been proposed to explore treatment-covariate interactions in survival data arising from two treatment arms of a clinical trial. The method is based on constructing overlapping subpopulations of patients with respect to one (or more) covariates of interest and in observing the pattern of the treatment effects estimated across the subpopulations. A plot of these treatment effects is called a subpopulation treatment effect pattern plot. Here, we explore the small sample characteristics of the asymptotic results associated with the method and develop an alternative permutation distribution-based approach to inference that should be preferred for smaller sample sizes. We then describe an extension of the method to the case in which the pattern of estimated quantiles of survivor functions is of interest.

Meta‐analysis of continuous outcomes combining individual patient data and aggregate data

Meta-analysis of individual patient data (IPD) is the gold-standard for synthesizing evidence across clinical studies. However, for some studies IPD may not be available and only aggregate data (AD), such as a treatment effect estimate and its standard error, may be obtained. In this situation, methods for combining IPD and AD are important to utilize all the available evidence. In this paper, we develop and assess a range of statistical methods for combining IPD and AD in meta-analysis of continuous outcomes from randomized controlled trials. The methods take either a one-step or a two-step approach. The latter is simple, with IPD reduced to AD so that standard AD meta-analysis techniques can be employed. The one-step approach is more complex but offers a flexible framework to include both patient-level and trial-level parameters. It uses a dummy variable to distinguish IPD trials from AD trials and to constrain which parameters the AD trials estimate. We show that this is important when assessing how patient-level covariates modify treatment effect, as aggregate-level relationships across trials are subject to ecological bias and confounding. We thus develop models to separate within-trial and across-trials treatment-covariate interactions; this ensures that only IPD trials estimate the former, whilst both IPD and AD trials estimate the latter in addition to the pooled treatment effect and any between-study heterogeneity. Extension to multiple correlated outcomes is also considered. Ten IPD trials in hypertension, with blood pressure the continuous outcome of interest, are used to assess the models and identify the benefits of utilizing AD alongside IPD.

Testing Treatment-Covariate Interaction in a Nonparametric Analysis of Covariance Model

We consider a recently introduced nonparametric model for Analysis of Covariance and derive an asymptotic test for interaction between covariate and treatment. Furthermore, we suggest data depth techniques to obtain joint confidence regions for the covariate effects in this model. The finite sample behavior of the asymptotic method is evaluated in simulations. Application of the procedures is illustrated using an epileptic seizures and chemotherapy data set.

Covariate heterogeneity in meta‐analysis: Criteria for deciding between meta‐regression and individual patient data

Meta-analyses of clinical trials are increasingly seeking to go beyond estimating the effect of a treatment and may also aim to investigate the effect of other covariates and how they alter treatment effectiveness. This requires the estimation of treatment-covariate interactions. Meta-regression can be used to estimate such interactions using published data, but it is known to lack statistical power, and is prone to bias.The use of individual patient data can improve estimation of such interactions, among other benefits, but it can be difficult and time-consuming to collect and analyse. This paper derives, under certain conditions, the power of meta-regression and IPD methods to detect treatment-covariate interactions. These power formulae are shown to depend on heterogeneity in the covariate distributions across studies. This allows the derivation of simple tests, based on heterogeneity statistics, for comparing the statistical power of the analysis methods.

Case‐Only Analysis of Treatment–Covariate Interactions in Clinical Trials

Differential effectiveness of treatments across subgroups defined by pretreatment variables are of increasing interest, particularly in the expanding research field of pharmacogenomics. When the pretreatment variable is difficult to obtain or expensive to measure, but can be assessed at the end of the study using stored samples, nested case-control and case-cohort methods can be used to reduce costs in large efficacy trials with rare outcomes. Case-only methods are even more efficient, and reliable under a range of circumstances.

Monitoring event times in early phase clinical trials: some practical issues

In many early phase clinical trials it is scientifically inappropriate or logistically infeasible to characterize patient outcome as a binary variable. In such settings, it often is more natural to construct early stopping rules based on time-to-event variables. This type of design may involve a variety of complications, however. The purpose of this paper is to illustrate by example how one may deal with various complications that may arise when monitoring time-to-event outcomes in an early phase clinical trial. We present a series of Bayesian designs for a phase II clinical trial in kidney cancer. Each design includes a procedure for monitoring the times to a severe adverse event, disease progression and death. The first design, which is the simplest, is based on the time to failure, defined as any of the three events, assuming exponentially distributed failure times with an inverse gamma prior on the mean. This design is compared by simulation to the CMAP design (Cheung and Thall, Biometrics, 2002; 58: 89-97). The model and monitoring procedure are then extended successively to accommodate several common practical complications, and we also study the method's robustness. Our simulations show that 1) one may apply the monitoring rule periodically, rather than continuously, without a substantive degradation of the design's reliability; 2) it is very important to account for interval censoring due to periodic evaluation of disease status; 3) it is important to account for the effect of disease progression on the subsequent death rate; 4) conducting a randomized trial presents little additional difficulty and provides unbiased comparisons; and 5) the exponential-inverse gamma model is surprisingly robust in most cases. The methods discussed here do not account for patient heterogeneity. This is an important but complex issue that may be dealt with by extending the models and methods given here to accommodate patient covariates and treatment-covariate interaction. Bayesian procedures for monitoring time-to-event outcomes offer a practical way to conduct a variety of early phase trials. Considerable care must be given, however, to modeling the important aspects of the trial at hand, and to calibrating the prior and the design parameters to ensure that the design will have good operating characteristics.

Uses and limitations of randomization-based efficacy estimators

In randomized trials with departures from allocated treatment, intention-to-treat analysis is important but not always sufficient. The most common supplement to intention-to-treat analysis is per-protocol analysis, whose assumption of comparability between different nonrandomized groups is often implausible. Randomization-based methods avoid making this assumption and are preferable. Situations where intention-to-treat analysis is insufficient and a randomization-based method is useful include provision of patient information, exploration of treatment-covariate and treatment-time interactions, meta-analysis, and equivalence trials.

A General Multi-Treatment Adaptive Design for Multivariate Responses

ABSTRACT A general adaptive allocation design is proposed for continuous multivariate responses where the covariance matrix of the response vectors is unknown. There are K > 2 competing treatments, possible prognostic factors are considered in the allocation procedure, potential delayed responses are allowed for, and treatment-covariate interactions are incorporated. The allocation rule for any incoming patient is dependent on all the allocation-and-response-and-prognostic factor history of the previously allocated patients as well as the prognostic factor vector of the current patient. The design is a generalization of the approach suggested by Bandyopadhyay and Biswas (2001), which was presented for a much simpler scenario. The performance characteristics of the proposed design and some follow-up inference procedures are studied analytically and also numerically illustrated. An extension of the present approach to the situation where some components of the response vectors are continuous and some binary is then considered. Some further extensions of the work are briefly indicated. Recommended by Nitis Mukhopadhyay

Responder identification in clinical trials with censored data

A new technique is presented for identification of positive and negative responders to a new treatment which was compared to a classical treatment (or placebo) in a randomized clinical trial with respect to survival time. This method was primarily developed for trials in which the two treatment arms do not differ in overall survival. It checks in a systematic manner if certain subgroups, described by so-called predictive factors, do show difference in survival due to the new treatment. The method relies on a good prognostic model built on one arm treated with placebo or the classical therapy. It employs the martingale residuals of the prognostic model in a stabilized bump hunting procedure, which finds groups of responders in the new treatment group with large positive or large negative residuals. The results of a simulation study are presented, comparing the performance of the new method to that of the Cox-PH model with treatment–covariate interactions. In a simulation study on average the proposed method recognizes in 90% the correct positive responder group and in 99% the correct negative responder group. The method is to be used in explorative analysis for hypothesis generation. The results are to be validated in future studies.

The regression trunk approach to discover treatment covariate interaction

The regression trunk approach (RTA) is an integration of regression trees and multiple linear regression analysis. In this paper RTA is used to discover treatment covariate interactions, in the regression of one continuous variable on a treatment variable withmultiple covariates. The performance of RTA is compared to the classical method of forward stepwise regression. The results of two simulation studies, in which the true interactions are modeled as threshold interactions, show that RTA detects the interactions in a higher number of cases (82.3% in the first simulation study, and 52.3% in the second) than stepwise regression (56.5% and 20.5%). In a real data example the final RTA model has a higher cross-validated variance-accounted-for (29.8%) than the stepwise regression model (12.5%). All of these results indicate that RTA is a promising alternative method for demonstrating differential effectiveness of treatments.

Phenytoin versus valproate monotherapy for partial onset seizures and generalized onset tonic-clonic seizures.

Phenytoin and valproate are commonly used antiepileptic drugs. It is generally believed that phenytoin is more effective for partial onset (simple partial, complex partial and secondary generalized tonic-clonic seizures) seizures whilst valproate is more effective in generalized onset seizures (generalized tonic-clonic seizures, absence, myoclonus) although there is no evidence from randomized controlled trials to support this belief. The use of individual patient data meta-analysis enabled us to examine time to event outcomes which are important in epilepsy monotherapy trials, and also to examine treatment-covariate interactions. To review the best evidence comparing phenytoin and valproate when used as monotherapy in subjects with partial onset seizures, or generalized onset tonic-clonic seizures with or without other generalized seizure types. Our search strategy included: (i) the Cochrane Epilepsy Group trial register, (ii) MEDLINE 1966-2000, (iii) hand-searching relevant journals, (iv) the pharmaceutical industry, and (v) researchers in the field. Randomized controlled trials in children or adults with partial onset seizures or generalized onset tonic-clonic seizures. Trials must have included a comparison of phenytoin monotherapy with valproate monotherapy. This was an individual patient data review. Outcomes were time to (i) withdrawal of allocated treatment, (ii) 12 month remission, (iii) six month remission, and (iv) first seizure post randomization. Data were analysed using stratified logrank analysis with results expressed as hazard ratios (HR) and 95% confidence intervals (95% CI), where a HR>1 indicates an event is more likely on phenytoin. A test for interaction between treatment and seizure type (partial onset versus generalized onset) was also undertaken for each outcome. Data were available for 669 subjects from five trials, representing 60% of the subjects recruited into the eleven trials that met our inclusion criteria. One important limitation of these data is that in four of the five trials, for patients classified as having generalized onset seizures, tonic-clonic seizures were the only seizure types recorded at follow up, despite the fact that some patients will have been experiencing other generalized seizure types such as absence or myoclonus. Their results for the generalized seizures therefore relate only to generalized onset tonic-clonic seizures. The main overall results were as follows (HR(95% CI), HR>1 indicates a clinical advantage for phenytoin for both remission outcomes and a clinical advantage for valproate for the outcomes time to withdrawal and time to first seizure) (i) time to withdrawal of allocated treatment 1.10(0.79-1.54), (ii) time to 12 month remission 1.04(0.78-1.38), (iii) time to six month remission 0.89(0.71-1.11), and (iv) time to first seizure 0.92(0.74-1.14). The results suggest no overall difference between drugs for these outcomes. The test for an interaction between treatment and seizure type (generalized versus partial onset) was non significant for all outcomes. We have not found evidence that a significant difference exists between phenytoin and valproate for the outcomes examined in this review. As generalized seizures such as absence and myoclonus were counted in only one trial, results do not address the treatment of these seizure types. We found no unequivocal evidence to overthrow or support the policy of using valproate in generalized onset tonic-clonic seizures and phenytoin in partial onset seizures.

Estimation of the odds-ratio in an observational study using bandwidth-matching

In many practical situations, such as incidence of a disease in two groups, a binary response variate Y depends on the effect of one of two treatments as well as a covariate X having different distributions under the two treatments. For this, consider the model: in the two treatment groups. If the odds-ratio is a constant, then it represents the treatment effect; otherwise, one can only think of a treatment main effect in the presence of a treatment-covariate interaction. A bandwidth-matched version of the Mantel Haenszel estimator of the odds-ratio is constructed, which is shown to be a consistent estimator of the treatment main effect and normally distributed in large samples.

Changepoint Statistics for Assessing a Treatment-Covariate Interaction

We formulate a stochastic process approach, related to that of Miller and Siegmund (1982, Biometrics 38, 1001-1016), for comparing two treatments in the setting where a covariate might affect the difference between the treatments. We illustrate the approach with data from a recent clinical trial examining whether pretreating subjects scheduled for major elective surgery with erythropoietin reduces their subsequent requirement for blood transfusion. The approach is valuable in identifying a changepoint in baseline hemoglobin level below which subjects appear to be more responsive to the infusion of erythropoietin.