Rt-PA and Stroke: Does IST-3 Make It All Clear or Muddy the Waters?Answers to the November 2012 Journal Club Questions

Abstract

1The Third International Stroke Trial (IST-3) is described as a randomized, controlled, open-treatment trial to address the efficacy of intravenous recombinant tissue plasminogen activator (rt-PA) outside the exclusionary criteria of the European license for treatment of acute stroke. Enrollment began in 2000 and concluded in 2011.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar AThe authors summarize the eligibility criteria in terms of the “uncertainty principle.”1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar Describe this principle as applied in the context of IST-3 and compare it to the principle of clinical equipoise in general practice treatment decisions. Is this an ethical enrollment criterion? Will enrollment based on this principle be consistent across the entire study time frame?BThe investigators defined an “experienced” stroke center as one that treated at least 3 patients with rt-PA in the preceding 12 months before joining the trial.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar What do you think about this definition? What percentage of patients were enrolled at experienced centers? Consider also the temporal distribution of patient enrollment in the study period. How might these factors affect internal and external validity?CAfter enrolling 276 patients, IST-3 switched from a double-blind trial to an open-treatment investigation. Control group patients in the open-treatment arm did not receive a corresponding bolus or infusion. Describe the placebo and nocebo effects and how they might impact study results. What outcomes are more likely to be exaggerated by the lack of a placebo?DDescribe potential sources of bias present in the IST-3. Do these sources of bias favor the treatment arm or the control arm?2The authors conclude “for the types of patients recruited in IST-3, despite the early hazards, thrombolysis within 6 h improved functional outcome.”1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar AIs this statement consistent with the results of the primary outcome measured in this study? On what evidence do the authors base this conclusion?BThe authors allude to “early hazards” after treatment with rt-PA. There was a 1.59 (95% confidence interval 1.23 to 2.07) unadjusted increased odds of death within the first 7 days after treatment with rt-PA, but this effect disappeared at 6 months. Hypothesize possible underlying mechanisms for the disappearance of this difference over time. What data acquired in IST-3 might allow limited retrospective exploration of these hypotheses?CThe authors include an “ordinal shift analysis” as a secondary outcome in IST-3. Describe the advantages of shift analysis over a dichotomous endpoint in terms of information theory. What limitations might shift analysis have compared with a dichotomous endpoint? Should shift analysis increase or decrease power to detect a difference between treatment arms?3The editorial accompanying the study publication in the Lancet concludes with the statement “… the default situation for the first health-care professional who identifies the stroke patient should be to treat, and the role of stroke and emergency physicians is now not to identify patients who will be given rt-PA, but to identify the few who will not.”2Leys D. Cordonnier C. rt-PA for ischaemic stroke: what will the next question be?.Lancet. 2012; 379: 2320-2321Abstract Full Text Full Text PDF PubMed Scopus (21) Google Scholar AThis statement is based on the results from IST-3 and the accompanying updated meta-analysis of rt-PA for acute stroke.3Wardlaw J.M. Murray V. Berge E. et al.Recombinant tissue plasminogen activator for acute ischaemic stroke: an updated systematic review and meta-analysis.Lancet. 2012; 379: 2364-2372Abstract Full Text Full Text PDF PubMed Scopus (740) Google Scholar Does IST-3 provide evidence that only a “few” patients will not benefit from rt-PA? Concisely summarize the findings of this study in the context of evidence from previous stroke trials.BExtensive conflict of interest disclosures are reported for study authors.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar What effect might these disclosures have on clinicians when they interpret study results? What biases have been demonstrated as occurring in studies whose authors have substantial conflicts of interest?CA patient presents to your emergency department with an acute stroke that was noted nearly 5 hours before arrival. A noncontrast computed tomography scan of the head shows no hemorrhage or signs of recent ischemic change, and the patient has no important vital sign or laboratory abnormalities. The family at bedside asks whether the patient is eligible for “clot-busting treatment.” Describe the informed consent you might offer the patient and family for rt-PA in the context of the results of IST-3. Q1. The Third International Stroke Trial (IST-3) is described as a randomized, controlled, open-treatment trial to address the efficacy of intravenous recombinant tissue plasminogen activator (rt-PA) outside the exclusionary criteria of the European license for treatment of acute stroke. Enrollment began in 2000 and concluded in 2011.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar Q1.a The authors summarize the eligibility criteria in terms of the “uncertainty principle.”1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar Describe this principle as applied in the context of IST-3 and compare it to the principle of clinical equipoise in general practice treatment decisions. Is this an ethical enrollment criterion? Will enrollment based on this principle be consistent across the entire study time frame? The authors state that if a patient had either a clear indication or contraindication for treatment with rt-PA, he or she was not eligible for enrollment in the trial. They go on to state, after informed consent, that only if both the clinician and patient (or relevant proxy) believed that the treatment was “promising but unproven” could the patient be enrolled as a study subject.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar The trial included the oversight of an independent data monitoring committee that, according to the predefined measures, would have recommended protocol changes including termination of the trial if a state of clinical equipoise no longer existed. The authors state that no such recommendations were made. Djulbegovic, in his 2007 article, reviewed the following work regarding the concept of equipoise and the uncertainty principle. In 1963, Bradford Hill wrote that randomization should be accepted “only in our state of ignorance, the treatment given being a matter of indifference.” In 1974, Fried introduced the term “equipoise,” stipulating that a physician may only enroll a patient in a trial if he or she is genuinely uncertain which treatment is preferred. As defined, equipoise is based on uncertainty at the level of the individual physician rather than at the level of the patient or community of physicians. In 1987, Friedman suggested that the state of equipoise be shifted from the individual physician to a community of expert practitioners. In 1998, Peto coined the term “uncertainty principle,” requiring that a patient be enrolled in a trial only if both the patient and the physician met an unquantified threshold of uncertainty with regard to the relative merits of the treatment options. In 2001, Lilford et al restated the unresolved question as, “How much uncertainty can one accept before we enroll a patient into a trial and by whom (patients, physicians, and community)?”4Djulbegovic B. Articulating and responding to uncertainties in clinical research.J Med Philos. 2012; 32: 79-98https://doi.org/10.1080/03605310701255719Crossref Scopus (55) Google Scholar Specific to the IST-3 trial, a state of clinical equipoise exists about rt-PA's benefit for patients who do not meet criteria for treatment per the European license. For example, the authors state that fewer than 100 patients older than 80 years have been enrolled in randomized trials, resulting in insufficient data to conclude whether these patients should be treated with rt-PA. The authors state that the trial was intended from its outset to be “pragmatic,” focused on improving the external validity. This implies that the authors are willing to compromise internal validity to gain external validity and execute a trial that supports not just efficacy but also effectiveness.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar An “efficacy” trial is one that attempts to reject the null hypothesis by finding a difference in the dependent variable by testing the treatment in question under ideal conditions, prioritizing rigorous internal validity over generalizability across diverse practice settings. If a treatment fails under ideal conditions, then it is not likely to be additionally tested for “effectiveness,” or generalizability, in diverse practice conditions. The design consideration is that a treatment that is efficacious when applied in experienced centers by experts under ideal conditions may be less so when application is limited in diverse practice settings by variable experience or expertise. Enrollment was not likely to be uniform, given the sources of variability within the trial, including subjectivity in the application of the uncertainty principle, institutional experience, provider experience, and geography. Additionally, the 11-year duration of the trial is likely to create enrollment variability because of changes in practice standards regarding thrombolytics in stroke during that period. All these sources of variation likely altered the enrollment procedure and the type of patient enrolled, contributing to the underenrollment that required a recalculation in sample size. The lack of uniformity in the sampling of the population of interest represents a potentially important confounder because it generates a heterogeneous sample, and that heterogeneity will be translated to the results, limiting interpretation. As with many other dilemmas in clinical research, the tension between ethical considerations and scientific rigor is ever present. In their letter to the editor, the authors state that according to the trial's pragmatic nature performed predominantly on patients traditionally regarded as not meeting criteria for rt-PA, they expected modest results. Q1.b The investigators defined an “experienced” stroke center as one that treated at least 3 patients with rt-PA in the preceding 12 months before joining the trial.1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar What do you think about this definition? What percentage of patients were enrolled at experienced centers? Consider also the temporal distribution of patient enrollment in the study period. How might these factors affect internal and external validity? Given the conventional perceptions of the term “experienced stroke center,” the criterion of having treated 3 patients in the previous year with a protocol for open-label treatment for rt-PA is likely to be surprising to most readers. Can a center with an organized system for stroke care be considered truly experienced if they treat only 3 patients per year with rt-PA? The reader must decide whether such limited experience in a given site and large variations in experience between sites might result in differential outcomes between institutions and regions. To inform this decision, it would be useful to know what percentage of sites met the minimum criteria. The article does not report the individual sites' experience classification. Inexperience can be a threat to internal validity by introducing systematic bias in several ways, including protocol violations before, during, and after the administration of rt-PA. In a trial this diverse and expansive, it would be difficult to identify all protocol violations that may have occurred at the various sites. If inexperienced sites contribute disproportionately to poor outcomes for the treatment group, then experience (or lack thereof) may have confounded the results toward the null hypothesis (favoring the control). In the answer to question 1a, we discussed the potential bias that may result from the inherent subjectivity in the application of the uncertainty principle. For sites with experience close to the 3-patient minimum criterion, it is likely that the majority of acute care physicians who enrolled patients had little to no consistent experience with the pretreatment patient interaction, including consent. Their inexperience will also increase the variability in the application of the uncertainty principle in that they are more likely to be uncertain in comparison to more experienced physicians. Inexperience in medicine usually translates to discomfort rather than enthusiasm. A notable exception is the physician experienced in stroke thrombolysis who encounters a fatal hemorrhage and loses confidence. However, among those who encounter fatal bleeding, those with experience are more likely to put the bleeding in perspective compared with those who have little experience. These imbalances are likely to manifest as sampling bias in the prerandomization eligibility phase, in which limited experience may be an important threat to recruitment. The trial is described as international and multicenter. At face value, the expansive general design lends itself well to the intention of improving external validity and increasing treatment eligibility. The implication of such a design is that the results, at the very least, can be applied to the involved centers and additionally to other similar centers internationally. According to the recruitment, are the various regions equally represented? In their 2011 update, the authors display the recruitment by country (save the designation of “United Kingdom”) in Table 1.5Sandercock P.P. Lindley R.R. Wardlaw J.J. et al.Update on the Third International Stroke Trial (IST-3) of thrombolysis for acute ischaemic stroke and baseline features of the 3035 patients recruited.Trials. 2011; 12 (252-252)https://doi.org/10.1186/1745-6215-12-252Crossref Scopus (35) Google Scholar Almost 50% of all subjects were enrolled in the United Kingdom at 75 centers. Approximately 90% of all subjects were enrolled in Europe and nearly 96% were enrolled in Europe and Australia. The regional list in Table 1 in IST-3 groups countries by region, including “Australasia,” though no Asian countries are listed in Table 1 of the 2011 update, in which each country is listed individually. For readers to interpret this international distribution accurately, they must consider the relative representation of different regions of the world. Regarding external validity and generalizability of the results, the reader should consider differences in stroke care and acute treatment protocols in the site of interest, with standards in Europe and Australia. The trial was conducted during an 11-year period, resulting in temporal variations. During that time, the practice standards of a novel therapy should be expected to evolve because additional data and experience alter such practice. Official government approval and licensing for rt-PA dramatically affected the trial when the design was changed from a double-blind, placebo-controlled trial to an open-label trial. The previously discussed subjectivity in the enrollment criteria is also subject to temporal variation. The physician and patient decisionmaking and experience at the outset of the trial are likely to change during the 11 years as individual and institutional experience and opinions about stroke thrombolysis evolve. For physicians, new data from positive-outcome trials such as the European Cooperative Acute Stroke Study (ECASS)-3, which extended the treatment window to 4.5 hours, and other observational studies would serve to bolster confidence during the decade and alter enrollment. Thus, the characteristics of the patients enrolled at the beginning of the trial are likely to vary in important ways from those enrolled toward trial completion, creating a heterogeneous sample. Q1.c After enrolling 276 patients, IST-3 switched from a double-blind trial to an open-treatment investigation. Control group patients in the open-treatment arm did not receive a corresponding bolus or infusion. Describe the placebo and nocebo effects and how they might impact study results. What outcomes are more likely to be exaggerated by the lack of a placebo? The nocebo effect is the causation of sickness or death by the expectation of sickness or death. The placebo effect is the causation of improvement or wellness by the expectation of improvement or wellness. The placebo and nocebo effect have 3 distinct components, the expectation of the patient, the expectation of the physician or healer, and the expectation that results from the relationship between both patient and healer.6Benson H. The nocebo effect: history and physiology.Prev Med. 1997; 26: 612-615Crossref PubMed Scopus (44) Google Scholar The expectation can be relative to the patient's beliefs about the disease in the absence of a treatment or when a treatment is administered. Speigel described how the placebo effect has been demoted in modern medicine to the role of a contaminant in research. For ethical reasons, less is known about the nocebo effect because its study would require generating negative expectations, which are expected to generate worse outcomes. Placebo and nocebo effects have been found to be greater when the outcomes are continuous and subjective, as in the case of pain treatment.7Spiegel H. Nocebo: the power of suggestibility.Prev Med. 1997; 26: 616-621Crossref PubMed Scopus (59) Google Scholar Several recent meta-analyses have demonstrated conflicting results, with the placebo effect ranging from 7% in a meta-analysis by Hróbjartsson and Gøtzsche8Hróbjartsson A. Gøtzsche P.C. Placebo interventions for all clinical conditions.Cochrane Database Syst Rev. 2004; (CD003974)PubMed Google Scholar compared with 20% in a review by Wampold et al.9Wampold B.E. Minami T. Tierney S.C. et al.The placebo is powerful: estimating placebo effects in medicine and psychotherapy from randomized clinical trials.J Clin Psychol. 2005; 61: 835-854Crossref PubMed Scopus (195) Google Scholar Variability in the effects described in these reviews likely results from the types of diseases and outcomes studied. Studies with patient-reported outcomes had the largest effect. The initial pilot phase of IST-3 was a double-blind trial, but the design was modified to an open label after Boehringer Ingelheim's wish to stop supplying drug and placebo.5Sandercock P.P. Lindley R.R. Wardlaw J.J. et al.Update on the Third International Stroke Trial (IST-3) of thrombolysis for acute ischaemic stroke and baseline features of the 3035 patients recruited.Trials. 2011; 12 (252-252)https://doi.org/10.1186/1745-6215-12-252Crossref Scopus (35) Google Scholar Recruitment for the expansion phase began in August 2003, the same year that rt-PA was approved for use in Europe. The use of a placebo control in a trial is to blind the subject and investigator, if double blinded, of treatment assignment and balance the systematic bias introduced when outcomes are affected by subject expectation. Without blinding and with full knowledge of the rt-PA administration (versus no treatment), the subject's positive and negative expectations with regard to rt-PA may result in a larger corresponding positive (placebo) or negative (nocebo) effect on the outcome, which is unrelated to the drug itself. With the loss of a true placebo arm, we are unable to estimate the true treatment effect. This is a substantial threat to the trial's internal validity. The subject's expectations are unprotected from the expectations of all the various providers involved in his or her care, as well as family members. Their expectations are subject to myriad cues given by all nonblinded parties whether they are subtle, obvious, subconscious, or conscious. If the subject, either because of an individual bias or a cultural bias, has a positive expectation of rt-PA, the treatment effect is likely to be exaggerated by placebo effect and outcomes expected from the natural history of stroke in the no-treatment arm will be worse because of the nocebo effect. Some may argue that subjects with negative opinions of rt-PA may balance these effects; however, eligibility for enrollment was based on the explicit patient impression that rt-PA represented a “promising, yet unproven” treatment and therefore those with negative opinions and expectations are unlikely to be recruited. This likely leaves the described placebo and nocebo effects unopposed. As stated earlier, the placebo effect is largest in self-reported subjective outcomes. In IST-3, the authors describe 3 potential means of measuring outcomes: mailed questionnaires requiring self-reporting, telephone interview by blinded data collectors, and clinical evaluation by physicians blinded to the treatment and also not involved in the patients' acute care. The mailed questionnaires are likely to introduce the greatest placebo/nocebo-related bias, as well as reporting bias based on treatment. Also, the selection of very enthusiastic patients may introduce a sampling bias before randomization, and when considered in the context of an open-label trial, this may result in underreporting of neurologic deficits by patients who received rt-PA and overreporting by those who received placebo. This will result in overestimation of effect by creating a positive bias in the treatment group and a nocebo effect in the no-treatment group. Q1.d Describe potential sources of bias present in the IST-3. Do these sources of bias favor the treatment arm or the control arm? 1Selection bias favors treatment: The uncertainty principle applied in IST-3 enrollment necessitates that treatment with t-PA not be clearly indicated or contraindicated according to the European license, as well as the subjective belief in potential benefit (“promising but unproven”) on the part of the physician and patient. Many patients who were eligible for rt-PA were excluded from the trial on this basis; however, the authors do not report the individual reasons that the patients were not enrolled. Had they done so, the reader would be able to determine how many were excluded because of an unfavorable view of the drug by the physician or patient. Such subjectivity is a major potential source of bias because physicians among institutions, countries, cultures, and continents are likely to have variable opinions of the efficacy of rt-PA. A strong bias (unconscious or not) may result in a selection bias, were the study to enroll only those patients who view the drug most favorably.2Nonblinded favors treatment: More than 90% of enrollment was in the nonblinded phase and lacked a placebo control. When a study omits a placebo control and is not blinded, the potential for overreporting is significant. The positive bias in favor of the treatment on the part of the physician and patient is likely to be transferred to the reporting of outcomes by patients receiving rt-PA. For those in the control group, the nocebo effect may result in patients overreporting poor outcomes, further threatening internal validity by increasing the difference between reported treatment and control outcomes.3Subjective outcomes favors treatment: Changing the outcome of interest from objective (mortality) to subjective (self-reported function) threatens internal validity in that outcome classification can vary with individual application and interpretation. This is further compounded by the bias introduced by the design feature that included mail-in follow-up, in which patients mailed in their self-assessments. Subjective outcomes combined with no placebo and nonblinding results in a significant potential for bias that favors the treatment group, given the selection bias described above. Follow-up using self-reported “mail-in” further magnifies the reporting bias for treatment and against control because of the lack of trained data collectors who may have otherwise been able to limit biased reporting.4Inexperience bias could favor either treatment or control: We discussed earlier the bias introduced by the potential for relative inexperience in the definition of “experienced centers.” This relative inexperience and lack of consistent opportunities for individual providers to enroll patients within the expectations of the uncertainty principle would increase the variability in both enrollment and treatment. If the inexperience translates to discomfort and insecurity with rt-PA, the physician will tend to enroll only the most enthusiastic patients, favoring the treatment group. If physician inexperience translates to favorable opinions about rt-PA because no bleeding events were observed, then the physician may enroll a more diverse, higher-risk sample, favoring the control. Institutional and provider inexperience may also lead to delays and other protocol violations that favor the control group. The important concept here is that such inexperience may lead to a lack of uniformity in enrollment and treatment between individual physicians and institutions.5Conflict of interest favors treatment: Most reputable journals have made important progress in disclosure of relationships that may result in a conflict of interest. The article footnote details the financial and other conflicts of interests for the authors. However, simply listing the conflicts does not eliminate the potential for bias. The reader must appreciate this potential. There is no simple adjustment that can correct for such bias. Even reviewers without financial conflict may be conflicted by other personal and professional bias that cannot be classified easily. The additive or, worse, synergistic effect of the subjective outcome bias, selection bias in favor of treatment inexperience bias, reporting bias, and nonblinding bias could represent a very strong cumulative bias toward rejecting the null hypothesis favoring the treatment. The results of IST-3 are neutral before taking into account the biases favoring treatment. The reader must assess the effect of these biases for overestimation of treatment effect. If the exaggeration of effect is great enough, the seemingly neutral results may actually be masking a negative trial result. 2. The authors conclude “for the types of patients recruited in IST-3, despite the early hazards, thrombolysis within 6 h improved functional outcome.”1IST-3 Collaborative GroupThe benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the Third International Stroke Trial [IST-3]): a randomised controlled trial.Lancet. 2012; 379: 2352-2363Abstract Full Text Full Text PDF PubMed Scopus (918) Google Scholar Q2.a Is this statement consistent with the results of the primary outcome measured in this study? On what evidence do the authors base this conclusion? The authors' primary conclusion ignores the primary outcome. The pr