PRECIS-2 Domains Research Articles

The Case for Conducting Pragmatic Dementia Care Trials in Medicaid Home and Community-Based Service Settings

Medicaid-funded home and community-based services (HCBSs) reach large numbers of individuals living with dementia who would otherwise reside in nursing homes with Medicaid funding. Medicaid HCBSs also often augment care provided by family and other informal caregivers to individuals living with dementia. Although Medicaid-funded HCBSs are offered in most states in lieu of nursing home care, they have been largely overlooked as health care system partners for implementation and testing of evidence-based dementia care interventions using embedded pragmatic clinical trial (ePCT) designs. In this article, we make the case for the importance of Medicaid-funded HCBSs as dementia care ePCT partners because of the volume of vulnerable clients with dementia served and the potential positive impacts that evidence-based dementia care programs can have on clients and their informal caregivers. This article first characterizes the Medicaid HCBS setting in terms of populations served and organizational arrangements across states. We then characterize strengths and potential limitations presented by Medicaid HCBSs as settings within which to implement dementia care ePCTs, using as a conceptual framework the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) tool and its domains. We draw on our experiences implementing the Care of Persons with Dementia in their Environments (COPE) program in a statewide Medicaid HCBS setting to highlight how these potential ePCT partners can help optimize pragmatic approaches to several PRECIS-2 domains. We found that partners are especially effective in implementing pragmatic ways to determine eligibility for evidence-based dementia care programs; assist with recruitment of eligible individuals; incorporate dementia care interventions into the range of existing HCBSs; and track outcomes relevant to persons living with dementia, caregivers, HCBS providers, and Medicaid insurance stakeholders. We conclude with recommendations for researchers, potential ePCT partners, and policymakers to help facilitate the growth of dementia care ePCTs in Medicaid HCBS settings across the United States.

How pragmatic are randomized trials of remdesivir and favipiravir for in-hospital treatment of COVID-19: a descriptive methodological review of trial design using the PRECIS-2 framework

ObjectivesTo review the pragmatism of published randomized trials of remdesivir and favipiravir based on the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) framework. Study Design and SettingTen eligible trials were identified from an existing comprehensive living review and were evaluated across the nine PRECIS-2 domains by two independent reviewers. ResultsAll 10 trials had mostly pragmatic design characteristics. Four of the domains (i.e., recruitment, setting, organization, and primary analysis) were found to be pragmatic with most trials scoring four or five across the two interventions. In comparison scores for four other design domains (i.e., eligibility, follow-up, flexibility of delivery, and primary outcome) varied across the trials with some design choices being more explanatory. ConclusionIn our descriptive review of randomized controlled trails for two drugs for patients infected with COVID-19 early in the pandemic, we found that most trials had more pragmatic than explanatory characteristics. Some design choices for some of the trials, however, were not consistent with the urgent goal of informing clinical decision making in an epidemic. PRECIS-2 should be used as a guide by trialists, to help them match their trial design choices to the intended purpose of their trial.

An application of PRECIS-2 to evaluate trial design in a pilot cluster randomised controlled trial of a community-based smoking cessation intervention for women living in disadvantaged areas of Ireland

Background“We Can Quit2” (WCQ2) was a pilot cluster randomised controlled trial with an embedded process evaluation assessing the feasibility and acceptability of ‘We Can Quit’ (WCQ, a peer-delivered community-based stop-smoking programme for women in disadvantaged communities. The control group comprised ‘enhanced usual care’ offered by the Irish Health Service Executive (HSE). The PRagmatic Explanatory Continuum Indicator Summary (PRECIS-2) is a tool to assess whether a trial design is more explanatory (working under ideal conditions) or pragmatic (working under ‘real-world’ conditions). The aim of this paper was to retrospectively evaluate the WCQ2 pilot trial using PRECIS-2 to inform the decision-making process on progression to a future definitive trial (DT).MethodsThe WCQ2 trial protocol and HSE standard stop-smoking service were described across the nine PRECIS-2 domains: eligibility, recruitment, setting, organisation, flexibility-delivery, flexibility-adherence, follow-up and primary outcome. Team members scored the domains as pragmatic or explanatory for each arm in a half-day workshop.ResultsSeven team members (practitioners and researchers) assessed the overall trial design as more explanatory than pragmatic. Important differences emerged between the two arms. WCQ targeted adult women from disadvantaged communities whereas HSE run a limited enhanced service for all quitters. Trial recruitment was challenging, intense efforts were needed as the trial proceeded. WCQ was delivered in a non-clinical community setting, HSE services in a clinical setting. WCQ organisation was co-designed with community partners and comprises peer-to-peer group support delivered by trained lay community facilitators, whereas HSE one-to-one support is delivered by Smoking Cessation Officers with a clinical background. Only WCQ allowed flexibility in delivery and adherence. Follow-up was more intensive in WCQ. Greater efforts to improve participant retention will be required in a future DT.ConclusionsPRECIS-2 allowed the reflection of practitioners and researchers on similarities and differences between intervention and control arms. Results will inform the decision on progression to an effectiveness DT, which will require more a pragmatic and less explanatory design. This novel use of PRECIS-2 to retrospectively evaluate a complex community-based pilot trial in advance of a full DT will also support learning for those undertaking hybrid trials of implementation and effectiveness.Trial registrationThis trial is registered with the ISRCTN registry (No. 74721694).

Robustness and pragmatism of the evidence supporting the European Society for Medical Oncology guidelines for the diagnosis, treatment, and follow-up of follicular lymphoma

ABSTRACT Background: Results of randomized clinical trials may not be entirely applicable to clinical practice. The present manuscript aims to explore the pragmatism and robustness of the evidence that supports the European Society for Medical Oncology (ESMO) follicular lymphoma (FL) guidelines. Methods & design: Analysis of all trials used to support positive, therapeutic, oncological recommendations in the 2020 ESMO FL guidelines. Predefined data points were extracted from each trial. Pragmatism was assessed by means of the PRECIS-2 tool, the difference in overall survival in the interventions compared and the source of funding. Robustness was assessed by means of the fragility index and the p value. Results: 28 trials were included. The full protocol or a protocol summary was provided for 12 (43%). Based on the PRECIS-2 domains, trials were considered pragmatic in organization, analysis and flexibility and explanatory in eligibility. Robustness was high, with 4/24 (17%) trials with p values between 0.05 and 0.005 and a median fragility index of 18. Conclusions: Results of trials to support ESMO recommendations in FL were robust. Pragmatism was high in some domains but modest to low in others and the pattern was similar across trials. Transparency in the publication of trial protocols was suboptimal.

Designing Trials with Purpose: Pragmatic Clinical Trials of Nonpharmacological Approaches for Pain Management.

Pain is one of the most significant causes of morbidity and disability worldwide. The efficacy of several nonpharmacological approaches for pain management has been established, but significant gaps exist between this evidence and their limited availability and use in routine clinical practice. Questions remain about their effectiveness and how best to integrate them in usual care to optimize patient-centered outcomes. Pragmatic clinical trials (PCTs) may help address this gap. Informed by the Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2), we sought to describe the key features of optimized PCTs of nonpharmacological approaches for the management of pain and common co-occurring conditions. To accomplish this objective, we searched the published literature on PCTs of nonpharmacological pain management approaches from 2010-2019 and applied the PRECIS-2 criteria. We discuss key PRECIS-2 domains of interest for designing and performing PCTs and cite specific examples from the published literature as potential models for future PCTs. We found 13 nonpharmacological PCTs. They were heterogeneous in size, recruitment, follow-up time, and location. The lessons learned from these studies led us to explicate key features of trials on the explanatory-pragmatic continuum across the PRECIS-2 domains that can be used by future investigators when designing their clinical trials of nonpharmacological approaches to pain management. We encourage the increased application of PCTs to produce timely and valuable results and products that will inform the development of safe and effective integrated pain care plans that optimize important patient-centered outcomes.

PRECIS-2 for retrospective assessment of RCTs in systematic reviews

Background and ObjectivesA randomized controlled trial (RCT) may be intended either to support real-world decisions on choice between alternative interventions or to help researchers understand mechanisms of action of an intervention. PRECIS-2 is widely used to help investigators match detailed design elements to their main intention for that RCT. PRECIS-2 is increasingly being used retrospectively for assessing RCTs within reviews. In this commentary, we counter arguments that RCTs with a placebo control group, masking/blinding of participants or providers, or conducted in a single center should be retrospectively assessed as completely explanatory, overriding a detailed PRECIS-2 assessment. We also counter arguments that a trial cannot be assessed using only the main peer-reviewed trial report. Study Design and SettingThis is a commentary on the use of PRECIS-2 for systematic reviews. ResultsAlthough placebos are seldom openly prescribed in real-world care, knowing that an intervention achieves its impact via the placebo effect might change some clinical and policy decisions, which means that this feature does not always preclude decision-making use and so should not override a full PRECIS-2 assessment. A domain describing the comparator should be added to PRECIS-2. Conduct of an RCT in only a single centre should also not override PRECIS-2 as the decision support value of a single-centre RCT could be high for decision makers in that centre and others like it. Many journals require that submitted RCT reports meet CONSORT reporting guidelines, which standardizes the available information for all RCTs in systematic reviews; whereas information from registration and protocol documents is unstandardized and undermines comparison between RCTs and across reviews. Published RCT reports are thus more suitable for retrospective PRECIS-2 assessments, but PRECIS-2 domains with missing information should be scored as blank. Wider use of the CONSORT extension specific to pragmatic trials may reduce domains with missing data. ConclusionsPRECIS-2 can be used for retrospective assessments of trials in systematic reviews. The PRECIS-2 instrument should be expanded by including a domain describing the control group(s). Published RCT reports are suitable for retrospective PRECIS-2 assessments.

Articles provided insufficient information to conduct an appropriate retrospective assessment of the pragmatic/explanatory features of medicine trials with the PRECIS-2 tool.

To assess whether, in the retrospective assessment of the pragmatic/explanatory features of pragmatic randomized controlled trials (pRCTs), the nine PRECIS-2 domain scores using the information provided in articles were modified after using the information reported in other publicly available sources. This is a cross-sectional study of participant-level pRCTs published in July 2018 to December 2019 in the four highest-impact general medicine journals. The articles described the main results of pRCTs assessing medicines in one or more arms that were not in the pre-licensing phases. The information reported in trial full protocols, published protocols, and other publications, registries, and trial websites were assessed and scored, and compared with that previously obtained after reviewing the information reported in the articles. Out of 76 articles on pRCTs, 13 (17%) were included in the analysis. All were two-arm trials, assessing medicines only (n = 7), medicine vs device (n = 2), medicine vs surgery (n = 1), or medicine vs placebo (n = 3). Seven were open-label trials, and six had any type of masking. All except one had the full protocol available and/or published protocol; seven had other types of publication available. The assessment of the nine PRECIS-2 domains with the information reported in the 13 articles was changed in all trials after using the information included in other additional available sources. Between one (n = 1 article) and six (n = 2) domains were modified in each pRCT. The domains that most commonly changed were "organization" (n = 12), "recruitment" (n = 11), and "follow-up" (n = 8). "Primary outcome" and "primary analysis" were not modified in any trial. Eight percent of all domains could not be assessed due to inadequate or lack of information in seven articles; those were "recruitment" (n = 3), "organization" (n = 3), "setting" (n = 2), and "flexibility:adherence" (n = 1). Articles describing the trial main results are usually insufficient for the appropriate retrospective assessment of the pragmatic/explanatory features of a pRCT by authors not involved in the conduct of the trial. To address this issue, editors should require the submission of the original full protocol and final full protocol with the history of amendments to be published as supplementary material to the article.

A dynamic application of PRECIS-2 to evaluate implementation in a pragmatic, cluster randomized clinical trial in two nursing home systems

BackgroundPRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary-2) can assess how clinical trial design decisions (along the explanatory-pragmatic continuum) influence the applicability of trial results to intended stakeholders. The tool has been used to assess features of trials during the trial design phase and also upon completion. The ongoing PRagmatic trial Of Video Education in Nursing homes (PROVEN), which is evaluating the effectiveness of a suite of videos to improve advance care planning, is one of the first large pragmatic, cluster randomized trials within nursing home health care systems. While certain features of pragmatic trials remain static once designed (e.g., recruitment, outcomes), successful implementation of a system-wide program requires on-going evaluation and adaptation. This report’s objectives were to apply PRECIS-2 in a novel manner during the actual conduct of the PROVEN trial to assess how dynamic adaptations shifted implementation to either a more explanatory or a more pragmatic approach.MethodsWe assessed PROVEN’s protocol as initially designed according to the three PRECIS-2 domains pertinent to implementation: (1) Organization, (2) Flexibility-Delivery, and (3) Flexibility-Adherence. We then applied this framework to conduct a formative evaluation of decisions made while the trial was ongoing to adapt the implementation approach along the pragmatic versus the explanatory continuum in response to emergent challenges.ResultsBased on the PRECIS-2 rubric, the initial design of the PROVEN implementation approach reflected a hybrid of pragmatic and explanatory features. Most notably, within the Flexibility-Delivery, the trial had a relatively pragmatic approach to protocol delivery by front-line nursing home providers, balanced with a more explanatory approach to protocol monitoring enabled by the analytic capabilities of the research team. This more intensive monitoring proved critical in revealing implementation problems once the study began. Dynamic adaptations made in response to these challenges generally reflected shifts to more explanatory approaches within the Flexibility-Delivery and Flexibility-Adherence domains including ever more intensive compliance monitoring, as well as detailed coaching of front-line providers delivering the intervention by the research team.ConclusionsPragmatic trials conducted in the nursing home setting may benefit from a more dynamic approach to implementation. Allowing fluidity between pragmatic and explanatory features may still preserve the trial's applicability to intended stakeholders’ needs. PRECIS-2 provides a useful formative evaluation tool to assess these adaptations in “real-time.”Trial registrationUS National Library of Medicine, ClinicalTrials.gov, ID: NCT02612688. Registered on 19 November 2015

Framing the conversation: use of PRECIS-2 ratings to advance understanding of pragmatic trial design domains

BackgroundThere continues to be debate about what constitutes a pragmatic trial and how it is distinguished from more traditional explanatory trials. The NIH Pragmatic Trials Collaborative Project, which includes five trials and a coordinating unit, has adopted the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) instrument. The purpose of the study was to collect PRECIS-2 ratings at two points in time to assess whether the tool was sensitive to change in trial design, and to explore with investigators the rationale for rating shifts.MethodsA mixed-methods design included sequential collection and analysis of quantitative data (PRECIS-2 ratings) and qualitative data. Ratings were collected at two annual, in-person project meetings, and subsequent interviews conducted with investigators were recorded, transcribed, and coded using NVivo 11 Pro for Windows. Rating shifts were coded as either (1) actual change (reflects a change in procedure or protocol), (2) primarily a rating shift reflecting rater variability, or (3) themes that reflect important concepts about the tool and/or pragmatic trial design.ResultsBased on PRECIS-2 ratings, each trial was highly pragmatic at the planning phase and remained so 1 year later in the early phases of trial implementation. Over half of the 45 paired ratings for the nine PRECIS-2 domains indicated a rating change from Time 1 to Time 2 (N = 24, 53%). Of the 24 rating changes, only three represented a true change in the design of the trial. Analysis of rationales for rating shifts identified critical themes associated with the tool or pragmatic trial design more generally. Each trial contributed one or more relevant comments, with Eligibility, Flexibility of Adherence, and Follow-up each accounting for more than one.ConclusionsPRECIS-2 has proved useful for “framing the conversation” about trial design among members of the Pragmatic Trials Collaborative Project. Our findings suggest that design elements assessed by the PRECIS-2 tool may represent mostly stable decisions. Overall, there has been a positive response to using PRECIS-2 to guide conversations around trial design, and the project’s focus on the use of the tool by this group of early adopters has provided valuable feedback to inform future trainings on the tool.

Understanding the applicability of results from primary care trials: lessons learned from applying PRECIS-2

ObjectiveTo compare two approaches for trial teams to apply PRECIS-2 to pragmatic trials: independent scoring and scoring following a group discussion. Study Design and SettingWe recruited multidisciplinary teams who were conducting or had conducted trials in primary care in collaboration with the Pragmatic Clinical Trials Unit, Queen Mary University of London. Each team carried out two rounds of scoring on the nine PRECIS-2 domains: first independently using an online version of PRECIS-2 and second following a discussion. ResultsSeven teams took part in the study. Before the discussion, within-team agreement in scores was generally poor and not all raters were able to score all domains; agreement improved after the discussion. The PRECIS-2 wheels suggested that the trials were pragmatic, although some domains were more pragmatic than others. ConclusionPRECIS-2 can facilitate information exchange within trial teams. To apply PRECIS-2 successfully, we recommend a discussion between those with detailed understanding of what usual care is for the intervention, the trial's design including operational and technical aspects, and the PRECIS-2 domains. For some cluster-randomized trials, greater insight may be gained by plotting two PRECIS-2 wheels, one at the individual participant level and another at the cluster level.

The PRECIS-2 tool has good interrater reliability and modest discriminant validity

ObjectivesPRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. Study Design and SettingOver 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. ResultsInterrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. ConclusionWe have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders.

Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory.

BackgroundThe National Institutes of Health (NIH) Health Care Systems Research Collaboratory (NIH Collaboratory) seeks to produce generalizable knowledge about the conduct of pragmatic research in health systems. This analysis applied the PRECIS-2 pragmatic trial criteria to five NIH Collaboratory pragmatic trials to better understand 1) the pragmatic aspects of the design and implementation of treatments delivered in real world settings and 2) the usability of the PRECIS-2 criteria for assessing pragmatic features across studies and across time.Methods/DesignUsing the PRECIS-2 criteria, five pragmatic trials were each rated by eight raters. For each trial, we reviewed the original grant application and a required progress report written at the end of a 1-year planning period that included changes to the protocol or implementation approach. We calculated median scores and interrater reliability for each PRECIS domain and for the overall trial at both time points, as well as the differences in scores between the two time points. We also reviewed the rater comments associated with the scores.ResultsAll five trials were rated to be more pragmatic than explanatory, with comments indicating that raters generally perceived them to closely mirror routine clinical care across multiple domains. The PRECIS-2 domains for which the trials were, on average, rated as most pragmatic on the 1 to 5 scale at the conclusion of the planning period included primary analysis (mean = 4.7 (range = 4.5 to 4.9)), recruitment (4.3 (3.6 to 4.8)), eligibility (4.1 (3.4 to 4.8)), setting (4.1 (4.0 to 4.4)), follow-up (4.1 (3.4 to 4.9)), and primary outcome (4.1 (3.5 to 4.9)). On average, the less pragmatic domains were organization (3.3 (2.6 to 4.4)), flexibility of intervention delivery (3.5 (2.1-4.5)), and flexibility of intervention adherence (3.8 (2.8-4.5)). Interrater agreement was modest but statistically significant for four trials (Gwet’s AC1 statistic range 0.23 to 0.40) and the intraclass correlation coefficient ranged from 0.05 to 0.31. Rating challenges included assigning a single score for domains that may relate to both patients and care settings (that is, eligibility or recruitment) and determining to what extent aspects of complex research interventions differ from usual care.ConclusionsThese five trials in diverse healthcare settings were rated as highly pragmatic using the PRECIS-2 criteria. Applying the tool generated insightful discussion about real-world design decisions but also highlighted challenges using the tool. PRECIS-2 raters would benefit from additional guidance about how to rate the interwoven patient and practice-level considerations that arise in pragmatic trials.Trial registrationsClinicaltrials.gov trial registrations: NCT02019225, NCT01742065, NCT02015455, NCT02113592, NCT02063867.