Health Technology Assessment Review Research Articles

A landscape review to identify what matters to patients with thrombotic cardiovascular diseases and patient-reported outcome instruments which can be used to capture the patient experience.

Thrombotic cardiovascular diseases profoundly impact patients' health-related quality of life (HRQoL). However, patient-reported outcome (PRO) instruments that are disease-specific or antithrombotic-treatment focused, developed according to US Food and Drug Administration (FDA) guidance on PROs, and can be used in clinical trials, are lacking. The aim of this study was to understand concepts important to patients diagnosed with coronary artery disease (CAD) or acute coronary syndrome (ACS), atrial fibrillation (AF), or stroke who require antithrombotic treatment for reducing risk of future thrombotic events (indications being evaluated for an investigational new drug), identify PROs that measure relevant symptoms and impacts, and determine acceptability of PROs from a health technology assessment (HTA) perspective. A landscape review, conducted between January 2009 and October 2020, included a search of qualitative literature (OVID), a review of PRO instruments using multiple sources (e.g., OVID and clinical trials databases), and a survey of HTA decisions for antithrombotic medications. The qualitative literature review identified 27 publications used to develop a high-level conceptual summary of symptoms and HRQoL impacts reported by patients. The instrument landscape review indicated that generic PROs have been utilized for thrombotic indications, but disease-specific, fit-for-purpose instruments are lacking, and the HTA review revealed that although HTA agencies discussed PRO instruments, evidence of specific recommendations was not found. To ensure patients' experiences, perspectives, and priorities are incorporated into drug development and evaluation, a core set of PROs for thrombotic indications that meet health authority guidance and are acceptable to HTA agencies is needed.

Treatment Effect Waning in Immuno-oncology Health Technology Assessments: A Review of Assumptions and Supporting Evidence with Proposals to Guide Modelling.

Treatment effect waning (TEW) refers to the attenuation of treatment effects over time. Assumptions of a sustained immuno-oncologic treatment effect have been a source of contention in health technology assessment (HTA). We review how TEW has been addressed in HTA and in the wider scientific literature. We analysed company submissions to English language HTA agencies and summarised methods and assumptions used. We subsequently reviewed TEW-related work in the ISPOR Scientific Presentations Database and conducted a targeted literature review (TLR) for evidence of the maintenance of immuno-oncology (IO) treatment effects post-treatment discontinuation. We found no standardised approach adopted by companies in submissions to HTA agencies, with immediate TEW most used in scenario analyses. Independently fitted survival models do however suggest TEW may often be implicitly modelled. Materials in the ISPOR scientific database suggest gradual TEW is more plausible than immediate TEW. The TLR uncovered evidence of durable survival in patients treated with IOs but no evidence that directly addresses the presence or absence of TEW. Our HTA review shows the need for a consistent and appropriate implementation of TEW in oncology appraisals. However, the TLR highlights the absence of direct evidence on TEW in literature, as TEW is defined in terms of relative treatment effects-not absolute survival. We propose a sequence of steps for analysts to use when assessing whether a TEW scenario is necessary and appropriate to present in appraisals of IOs.

Development of a Prioritization Framework to Aid Healthcare Funding Decision Making in Health Technology Assessment in Australia: Application of Multicriteria Decision Analysis

ObjectivesThis study develops a prioritization framework to aid healthcare funding decision making in health technology assessment (HTA) in Australia using a multiple criteria decision analysis (MCDA) approach. MethodsMCDA frameworks for HTAs were reviewed through literature survey to identify the initial criteria and levels within each criterion. Key stakeholders and experts were consulted to confirm these criteria and levels. A conjoint analysis using 1000Minds was undertaken with policy makers from the Department of Health to establish ranking criteria and weighting scores. Monte Carlo simulations were used to examine the sensitivity of findings to factors affecting the ranking and weighting scores. The MCDA was then applied to 6 examples of chronic care models or technologies projects to demonstrate the performance of this approach. ResultsFive criteria (clinical efficacy/effectiveness, safety and tolerability, severity of the condition, quality/uncertainty, and direct impact on healthcare costs) were consistently ranked highest by healthcare decision makers. Among the criteria, patient-level health outcomes were considered the most important, followed by social and ethical values. The analyses were robust to inform the uncertainty in the parameter. ConclusionsThis study has developed an MCDA tool that effectively integrates key priorities for HTA reviews, reflecting the values and preferences of healthcare stakeholders in Australia. Although this tool aims to align the assessment process more closely with health benefits, it also highlights the importance of considering other criteria.

Health technology assessment of medicinal products in Greece: a 5-year (2018-2023) review of timelines and productivity.

To assess the health technology assessment (HTA) process in Greece from its establishment in 2018 until 2023 in terms of timeliness and productivity. Data were collected from the HTA Committee's database and other publicly available sources. The overall study timeframe was divided into three periods: (i) July 2018-January 2020, (ii) January 2020-July 2021, and (iii) July 2021-February 2023. During the study period, a total of 1,157 applications for medicinal products (MPs) (including 219 new active substances (NAS) and orphans) were submitted to the HTA Committee. The number of HTA recommendations increased from 60 (first period) to 641 (third period), while the backlog of MPs pending HTA and price negotiations decreased from 89 and 106 (January 2020) to 8 and 44 (February 2023), respectively. The median time intervals for all application types decreased significantly over time. In February 2023, the median time for clinical data assessment of NAS (excluding orphans) almost halved from 207days in the first period to 114days; median times for NAS and orphans from regulatory approval to HTA application were 420 and 457days, and from HTA application to reimbursement 228 and 417days, respectively. The performance of the HTA process in Greece improved significantly over time, with increased MP appraisals, backlog reduction, and decreased timelines. Delays in reimbursement of NAS were mainly caused by the long gap between regulatory approval and HTA application. Overall, HTA review times in Greece are now on par with that of well-established European HTA systems.

OP116 Building Alignment Between Industry, Academia, And Patients On Health Technology Assessment Methods

IntroductionWith the announcement of the health technology assessment (HTA) review in Australia, a HTA Summit was organized by ISPOR Australia in November 2022. The aim of the Summit was to provide a forum for industry, academia, and patients to share ideas and find common ground with respect to HTA policy, processes, and methods. Topics were determined by a Steering Committee and included: managing uncertainty; patient engagement; second order effects; genetics, genomics and precision medicines; conditional listing; and real-world evidence. Presentations on each topic were conducted by industry and non-industry experts. Breakout sessions led by facilitators were also held for each topic with members of the audience.MethodsDiscussions were recorded during the event and a thematic analysis was performed.ResultsThe following themes were identified from the event.•There was a strong sentiment that participants enjoyed the opportunity to discuss ideas and work toward solutions.•There was a consistent theme that many of the issues arising in HTA were due to a lack of communication between sponsors, evaluators, patients, and decision makers.•It was noted that HTA encompasses several technical terms that have different meanings among various stakeholders.•There was a clear consensus that patients should be involved in HTA earlier and throughout the process.•HTA reform can help drive better access to real-world evidence.•To improve the efficiency of the process, uncertainty could be reframed as risk management, which incorporates the effect of uncertainty in the funding decision.•HTA includes policies, processes, and methods and is used as a tool by decision makers to make informed funding decisions. It was noted on several occasions during the Summit that funding decisions have a political element that should be separated from the HTA process.ConclusionsIt is possible to achieve better collaboration between industry, academic, and patient groups with respect to HTA reform. To promote more collaborative work a consistent conflict of interest definition would be helpful.

EE430 Use of Single-Arm Trials in Health Technology Assessment Reviews of Axicabtagene Ciloleucel

EE430 Use of Single-Arm Trials in Health Technology Assessment Reviews of Axicabtagene Ciloleucel

Assessing Response in Atopic Dermatitis: A Systematic Review of the Psychometric Performance of Measures Used in HTAs and Clinical Trials.

Assessing treatment response is key to determining treatment value in atopic dermatitis (AD). Currently, response is assessed using various clinician- or patient-reported measures and response criteria. This variation creates a mismatch of evidence across trials, hindering the ability of clinicians, regulators, and payers to compare the efficacy of treatments. This review identifies which measures and criteria are used to determine response in clinical trials and health technology assessments (HTAs). Moreover, it systematically reviews the psychometric performance of those measures and criteria to understand which perform best in capturing patient-relevant symptoms and treatment benefits. A scoping review of clinical trials and HTAs in AD identified the following measures for inclusion: the Eczema Area and Severity Index (EASI), the Investigator's Global Assessment (IGA), the Dermatology Life Quality Index (DLQI) and the Peak Pruritus Numerical Rating Scale (PP-NRS). A systematic search was performed in MEDLINE and Embase to identify studies testing the psychometric performance of these measures in adults or adolescents with AD. A lack of consistency in the assessment of response was observed across clinical trials and HTAs. Important gaps in psychometric evidence were identified. No content validations of the EASI and IGA in AD were found, while some quantitative studies suggested that these measures fail to capture itch, a core symptom. The PP-NRS and DLQI performed well. No studies compared the performance of different response criteria. Content validation of the PP-NRS confirmed the importance of itch as a core symptom and treatment priority in AD; however, itch is not well covered in the EASI or IGA. Including the PP-NRS in clinical trials and HTAs will better capture patient-relevant benefit and response. Although various response criteria were used, no studies compared the performance of different criteria to inform which were most appropriate to compare treatments in clinical trials and HTAs.

Access in all areas? A round up of developments in market access and health technology assessment: part 1.

In this new series reviewing recent developments in market access, we highlight publications investigating health technology assessment (HTA) guidance, review processes and outcomes across the world and discuss how forthcoming changes in the HTA and regulatory environment in the European Union may allow for more consistency in decision making.

The RoboCOS Study: Development of an international core outcome set for the comprehensive evaluation of patient, surgeon, organisational and population level impacts of robotic assisted surgery.

The introduction of robot-assisted surgery is costly and requires whole system transformation, which makes the assessment of benefits (or drawbacks) complex. To date, there has been little agreement on which outcomes should be used in this regard. The aim of the RoboCOS study was to develop a core outcome set for the evaluation of robot-assisted surgery that would account for its impact on the whole system. Identification of a long-list of potentially relevant outcomes through systematic review of trials and health technology assessments; interviews with individuals from a range of stakeholder groups (surgeons, service managers, policy makers and evaluators) and a focus group with patients and public; prioritisation of outcomes via a 2-round online international Delphi survey; consensus meeting. 721 outcomes were extracted from the systematic reviews, interviews and focus group which were conceptualised into 83 different outcome domains across four distinct levels (patient, surgeon, organisation and population) for inclusion in the international Delphi prioritisation survey (128 completed both rounds). The consensus meeting led to the agreement of a 10-item core outcome set including outcomes at: patient level (treatment effectiveness; overall quality of life; disease-specific quality of life; complications (including mortality); surgeon level (precision/accuracy; visualisation); organisation (equipment failure; standardisation of operative quality; cost-effectiveness); and population (equity of access). The RoboCOS core outcome set, which includes the outcomes of importance to all stakeholders, is recommended for use in all future evaluations of robot-assisted surgery to ensure relevant and comparable reporting of outcomes.

How are Companion Diagnostics Considered in Economic Evaluations of Oncology Treatments? A Review of Health Technology Assessments.

BackgroundCompanion diagnostic (CDx) testing is increasingly used to identify eligible patients for targeted treatments. Guidance on how CDx testing should be incorporated into cost-effectiveness models (CEM) is limited. This review evaluated how health technology assessment bodies and research organizations considered CDx in CEMs of targeted therapies in oncology and whether this ultimately impacted their decisions or time from regulatory approval to recommendations.MethodsAn exhaustive list of approved CDx tests in oncology was compiled. For corresponding indications and treatments, NICE appraisals published between 2016 and 2019 were identified. Then, assessments for the same treatments issued from 11 other agencies were reviewed. Data extracted included background and CDx information, CDx's role in the CEM, and recommendations.ResultsTwenty-seven NICE appraisals were identified; 15 considered CDx testing in the CEM, while 12 did not, mainly because testing had already been established for the comparators within the same class or in clinical practice from a prior treatment line. Both testing costs and mutation prevalence drove CDx testing costs per patient. The cross-comparison of assessments showed that CDx test characteristics were inconsistently reported. Time from regulatory approval to recommendations was not impacted by CDx cost inclusion in CEMs.ConclusionCDx testing was included in cost-effectiveness models whenever mutation testing was required solely for patients receiving targeted treatment; cost per patient was based on test costs and mutation prevalence. It is unclear if expanded reliance on CDx testing will impact future assessments of targeted therapies. A future update is warranted to track trends over time.Supplementary InformationThe online version contains supplementary material available at 10.1007/s41669-022-00350-6.

FDA Accelerated Approval for Malignant Hematology and Oncology Indications in the Canadian Environment.

Accelerated approval (AA) by the FDA enables earlier access to promising new therapies. Health Canada has a similar process. Canada implemented a national health technology assessment (HTA) for reimbursement decisions in 2011. This study evaluated regulatory and funding timelines and decisions for FDA AA cancer therapies in Canada. The FDA’s AA of malignant hematology and oncology from January 2000–December 2019 was reviewed. Dates from Health Canada, HTA decisions and provincial listings were collected. There were 94 FDA AAs, two of which were subsequently withdrawn. Of the 92 AAs, 70 received full (46)/conditional (24) Health Canada approval, and 22 were not filed. Since the introduction of HTA, 31 out of 45 of Health Canada’s approved indications underwent HTA review: 18 received a positive recommendation conditional on cost-effectiveness, 8 were not recommended and 5 were withdrawn/suspended. The median time from the AA to any Health Canada approval is 9.4 months, from any Health Canada approval to HTA decision is 5.8 months and from HTA decision to the first formulary listing is 12.0 months. The access and timeline for the first formulary listing differences were observed between the USA and Canada due to the decision of pharmaceutical companies to submit (or not) to regulatory/HTA bodies, national procedural delays with different healthcare delivery models and submission timelines. This study demonstrates that there is delayed access to promising new therapies in Canada.

Patient-centered health technology assessment: a perspective on engagement in health technology assessment by three patient organizations and a health technology assessment body.

Patient engagement in health technology assessment (HTA) has become increasingly important over the past 20years. Academic and practitioner literature has produced numerous case studies and best practice accounts of patient involvement practices around the world. This text analyzes the experience of being involved in an Institute for Clinical and Economic Review (ICER) HTA review in the United States. The analysis comes from the joint perspective of three patient organizations: Lupus and Allied Diseases Association, Inc.; Lupus Foundation of America; and Black Women's Health Imperative, as well as ICER. We suggest that meaningful, patient-centered engagement, where patient communities are systematically integrated throughout the review, can be a way of returning to the discipline's roots focusing on technologies' societal and ethical impact. It is a process that requires robust commitment from all involved but produces assessments relevant to those directly affected by them.

OP135 Impact Of Updated Trial Data On The Cost-effectiveness Of Health Technologies: A Case Study On Percutaneous Mitral Repair

IntroductionExtrapolation methods are commonly used to model the cost-effectiveness of health technologies beyond observed data. Reassessing cost-effectiveness estimates using updated clinical trial data has the potential to reduce uncertainty and optimize decision-making. We present a case study based on percutaneous repair (PR) with the Mitraclip system, a technology to treat severe secondary mitral regurgitation (MR). For the study purpose, we considered the COAPT trial that evaluated the effectiveness of adding PR to medical treatment versus medical treatment alone.MethodsWe developed a time-varying Markov model to assess the cost-effectiveness of PR. Clinical inputs were based on reconstructed individual patient data from the COAPT trial results reported at 2 years, and at 3 years.We developed parametric modeling for overall survival (OS) and heart failure hospitalizations (HFH) to obtain clinically plausible extrapolations beyond observed data. We adopted the French perspective and used a 30-year time horizon. We expressed incremental cost-effectiveness ratios (ICERs) as cost per quality-adjusted life year (QALY).ResultsBased on 2 year-data, preferred parametric models for OS and HFH were exponential and log-logistic respectively, yielding an ICER of EUR21,918/QALY and >0.5 probability of PR being cost-effective (EUR50,000/QALY threshold).Updated analyses at 3 years showed a change of OS trajectory for PR that justified the use of piecewise modelling, yielding an updated ICER that went up to EUR77,904/QALY (base-case), and to a minimum of EUR58,175/QALY (scenario analysis). Using data at 3 years, PR had <0.5 probability of being cost-effective.ConclusionsIn this case study, the availability of updated survival analyses of the main trial is likely to have some impact on decision-making and/or pricing discussion as part of health-technology assessment (HTA). We aim to provide further updated analyses as 4 years results of the COAPT study become available.More broadly, original technology appraisals are frequently undertaken when mid/long-term follow-up trial data may be lacking. Our example suggests the need for continuous HTA review as new clinical data are released.

A Systematic Review of Health Technology Assessments of Chimeric Antigen Receptor T-Cell Therapies in Young Compared With Older Patients

ObjectivesThe objective of this review was to identify sources of variability in cost-effectiveness analyses of chimeric antigen receptor T-cell (CAR-T) therapies, tisagenlecleucel and axicabtagene ciloleucel, evaluated by health technology assessment (HTA) agencies, focusing on young compared with older patients. MethodsHTA evaluations in pediatric acute lymphoblastic leukemia (ALL) and adult diffuse large B-cell lymphoma (DLBCL) were included from Australia, Canada, England, Norway, and the United States. Key clinical evidence, economic approach, and outcomes (costs, quality-adjusted life-years [QALYs] and incremental cost-effectiveness ratios) were summarized. ResultsFourteen HTA evaluations were identified (5 ALL, 9 DLBCL [4 tisagenlecleucel, 5 axicabtagene]). Analyses were naive comparisons of prospective single-arm studies for the CAR-Ts with retrospective cohort studies for the comparators. Key clinical evidence and economic model approaches were generally consistent by CAR-T and indication, although outcomes varied. Notably, incremental QALYs varied substantially in ALL (3.67-10.6 QALYs gained), whereas variation in DLBCL was less (1.21-1.97 [tisagenlecleucel], 1.97-3.40 [axicabtagene]). Discounting of costs and outcomes varied, with the highest QALYs generated for tisagenlecleucel in ALL (10.95) associated with the lowest discount rate (1.5%) and vice versa (4.97 QALYs; 5% discount rate). The approach to extrapolation of overall survival data varied, even where the same empirical data were used. ConclusionModeled, long-term treatment benefit in young patients may be associated with greater uncertainty compared with adults because of potential life-long benefits with cell and gene therapies. This reflects the methodological challenges identified by HTA agencies associated with single-arm, short-term studies.

Use of External Comparators for Health Technology Assessment Submissions Based on Single-Arm Trials

BackgroundSingle-arm trial (SAT) data is increasingly reviewed for drug approvals by regulators and Health Technology Assessment (HTA) bodies. Supplementary data in the form of external comparators (ECs) can be used to provide clinical context to support these drug evaluations. In this study we characterized HTAs for SAT-based submissions, the use of supplementary EC data and outcomes from HTA review. MethodsHTA Accelerator database was used to describe SAT-based HTA submissions with decisions (2011-2019). ResultsA total of 433 SAT-based HTA submissions were identified between 2011 and 2019 with a 13-fold increase during this period. Around 65%(283/433) were in oncology or hem-oncology. Around 52%(226/433) of submissions contained some type of EC data, including prior clinical trials (24%, 104) and real-world data (RWD) (20%, 87), but 40%(175) contained no EC data. The overall acceptance rate for SAT-based submissions was 48% and with RWD EC data acceptance was 59%. In the latest 5-year period (2015-2019), use of RWD ECs increased 22% as a proportion of submissions per year, whereas, prior trial ECs decreased (–14%) and use of no EC remained stable (–2%). Between 2015 to 2017 and 2018 to 2019, acceptance rate for RWD ECs increased by 20% (41% in 2015-2017 to 61% in 2018-2019) whereas prior trial EC use decreased by 10% and no EC submissions decreased 16%. Of 226 submissions using ECs, only 29%(66) used an adjusted indirect treatment comparison method. ConclusionsSAT-based submissions to HTA bodies are rapidly evolving in terms of composition and acceptance. Types of EC and methodological approach used are important determinants of positive outcomes.

PDG23 Comparing Average Time to Listing and Drug Coverage Rates between Provinces in Canada

Following Health Technology Assessment (HTA) reviews by the Canadian Agency for Drugs and Technologies in Health (CADTH), the decision to publicly fund drugs remains to respective public drug plans (federal, provincial, and territorial). The objective of this analysis was to evaluate the varying times between CADTH Common Drug Review (CDR) recommendations and listings among Canadian provinces (except Quebec), along with the differences between proportions of drugs covered per province.

PCN142 Patient and Caregiver Preferences in Treatment Decision Making in Multiple Myeloma — Results of a Targeted Literature Review

Multiple myeloma (MM) burden on patients has been exhaustively addressed in research; however, the burden of MM on patients’ family-engaged caregivers is often overlooked. Decision-aids (DAs), such as treatment guidance documents, may facilitate shared decision-making for patients and caregivers who are burdened by health-related, financial, and social factors. However, patient and caregiver preferences may not be integrated in DAs. This analysis aimed to evaluate how patient and caregiver preferences were considered in DAs. A targeted literature review was conducted on peer-reviewed publications using PubMed, ProQuest, CINAHL Complete, and Cochrane databases, complemented with a gray literature search. All findings were limited to English language articles published in the past 5 years. Data on whether and how patient and caregiver preferences were mentioned in the identified DAs were extracted and synthesized qualitatively. Forty-one DAs were included. Twenty-two highlighted the importance of considering patient preferences; however, only 11 DAs mentioned any specific factors for consideration. Medical factors included treatment goals, tolerability of aggressive treatment options, frequency and length of hospital visits, mode of drug administration, and limiting future treatment options. Non-medical factors included ability to work and related unemployment, health insurance coverage, direct/indirect costs, distance to hospital, ability to drive, and need for social or transportation support services. Evidence that holistically maps patient and caregiver perspectives is lacking in DAs. There are no interactive MM treatment DAs incorporating patient and caregiver factors. An interactive DA built upon a patient value framework tailored for MM could support more personalized treatment decisions. Patients and caregivers could evaluate treatment options, harmonizing benefit-risk assessment received from healthcare professionals and provide an integrated voice for health technology assessment and regulatory reviews. Collection of real-world evidence to further understand patient and caregiver preferences is needed.

A Model for Determining the Number of Health Technologies in the Health Systems: An Approach Combining Multiple Attributes Decision Making and Goal Programming

In recent years, health technology policy-making science has gone beyond just a health technology assessment or systematic review or economic evaluation study and the science of operational research in decision making, i.e. multi-attributes and multi-objective decision-making has been included. Hence, currently, health technology policy-making follows a seven-step process. After a technology undergoes these steps and is proven that it is useful for the health system, it must be determined how many of it is needed in the health system. Determining the required number of health technologies is a challenge that remains to be considered. Therefore, this study was designed to overcome this problem. The authors intend to introduce a multi-objective decision-making methodology considering the limited budget, to determine the number of technologies required to complete for the health technology policymaking cycle.

Companies' Health Technology Assessment Strategies and Practices in Australia, Canada, England, France, Germany, Italy and Spain: An Industry Metrics Study.

Background: Health technology assessment (HTA) has increased in importance in supporting payer decision making by assessing the relative effectiveness and cost effectiveness of new medicines. Thus, pharmaceutical companies need to address the HTA requirements early during development to improve reimbursement outcomes. Currently, there is a lack of research to assess the impact of HTA on development and jurisdictional outcome from companies’ perspectives. This study aimed to assess companies’ HTA strategy and characterise HTA practice in seven jurisdictions. Methods: A multi-year, annual study collected information for individual products, focusing on development activities regarding inclusion of HTA requirements and selection of global comparators. The generation of local contextual information, submission strategies and predictability of HTA outcomes was examined jurisdictionally in Australia, Canada, England, France, Germany, Italy and Spain. The study questionnaire was built into a secure online data collection platform and data were provided annually by participating companies. Results: Data for 169 compounds were provided by nine international companies between 2014 and 2018. HTA requirements were implemented in evidence generation plan for 63% of products during development. Global comparators were accepted by HTA bodies for more than half of studied products; Spain showed the highest acceptance rate (85%). Companies took advantages of parallel process in Australia and Canada to shorten product rollout time. Australia demonstrated general consistency in HTA review time, and England had the longest variation (interquartile range, 216 days). Requirements for additional information after submission occurred at all HTA bodies. Germany and Italy showed the highest percentage of products being reimbursed as per regulatory label (80 and 68%, respectively). Canada was the most predictable jurisdiction, with the highest proportion of review outcome (90%) that met companies’ expectations. Conclusion: Companies are addressing HTA requirements during development for many products; however, they are challenged by varying requirements and practices and product success ultimately depends on how HTA organisations and payers assess added value in the context of the national healthcare systems. This ongoing study created a baseline to help capture fact-based changes for company HTA strategies and HTA body practices.

Assessing context suitability (generalizability, external validity, applicability or transferability) of findings in evidence syntheses in healthcare-An integrative review of methodological guidance.

Evidence syntheses provide the basis for evidence-based decision making in healthcare. To judge the certainty of findings for the specific decision context evidence syntheses should consider context suitability (ie, generalizability, external validity, applicability or transferability). Our objective was to determine the status quo and to provide a comprehensive overview of existing methodological recommendations of Health Technology Assessment (HTA) and Systematic Review (SR) producing organizations in assessing context suitability of evidence on effectiveness of health care interventions. Additionally, we analyzed similarities and differences between the recommendations. In this Integrative Review we performed a structured search for methods documents from evidence synthesis producing organizations that include recommendations on appraising context suitability in effectiveness assessments. Two reviewers independently selected documents according to predefined eligibility criteria. Data were extracted in standardized and piloted tables by one reviewer and verified by a second reviewer. We performed a thematic analysis to identify and summarize the main themes and categories regarding recommended context suitability assessments. We included 14 methods documents of 12 organizations in our synthesis. Assessment approaches are very heterogeneous both regarding the general concepts (eg, integration in the evidence synthesis preparation process) and the content of assessments (eg, assessment criteria). Some heterogeneity seems to be justified because of the need to tailor the assessment to different settings and medical areas. However, most differences were inexplicable. More harmonization is desirable and appears possible.