OP95 Are Patient-Reported Outcome Measures Meeting Today's Standards?

Excessive daytime sleepiness is common in Prader-Willi syndrome (PWS), with prevalence ranging from 52% to 100%. The goal of this study was to establish the content validity (ie, evidence that an instrument measures an intended concept of interest) of the parent/caregiver version of the Epworth Sleepiness Scale for Children and Adolescents (ESS-CHAD), a measure of daytime sleepiness, in PWS. Qualitative, dyadic semistructured video interviews were conducted with 18 caregivers and their children with PWS from April to June 2020. Concept elicitation and cognitive interview techniques were implemented. Thematic analyses allowed for examination of themes and data patterns. All caregivers (mean age 49 years) were mothers of individuals with PWS who experienced troublesome daytime sleepiness (mean age 14 years). The most prevalent observable signs/symptoms of daytime sleepiness were sleepy/sleepiness (n = 17; 94.4%), tired/tiredness (n = 16; 88.9%), exhaustion/exhausted (n = 5; 27.8%), anxious/stressed (n = 5; 27.8%), irritable/frustrated (n = 5; 27.8%), having tantrums/outbursts (n = 5; 27.8%), and lethargy (n = 4; 22.2%). Daytime sleepiness impacted various aspects of health including mental, emotional, physical, and social well-being. When caregivers were asked about the activities associated with daytime sleepiness, all salient concepts elicited mapped to the ESS-CHAD; saturation was met after the first 4 interviews. Only 2 concepts, after physical exertion and while inactive/bored, did not map. Caregiver statements indicated that these concepts, although related to daytime activities, were atypical of daily routines. The ESS-CHAD was well understood and relevant to caregivers. This study supports the content validity of the ESS-CHAD and its appropriateness for evaluating treatment efficacy of daytime sleepiness in PWS. Patel VP, Patroneva A, Glaze DG, Davis K, Merikle E, Revana A. Establishing the content validity of the Epworth Sleepiness Scale for Children and Adolescents in Prader-Willi syndrome. J Clin Sleep Med. 2022;18(2):485-496.

Development of a Patient Reported Outcome Daily Diary to Assess Symptom Burden in Sickle Cell Disease

PHP154 - Patient-Reported Outcome Measures in the FDA Pilot Compendium: Meeting Today’s Standards for Patient Engagement in Development?

The Increasing Use of Clinical Outcome Assessments in US Food and Drug Administration Labels and the German Gemeinsamer Bundesausschuss, National Institute for Health and Care Excellence, and the French Haute Autorité de Santé Assessments From 2012 to 2022.

The aim is to identify the extent to which EQ-5D is used as a clinical outcome assessment (COA) endpoint in a non-economic context in health technology assessment (HTA) decisions, regulatory labelling claims and published literature. Drug technology appraisals (TAs) published by HTA agencies in England, France, Germany and the USA between 2019 and 2021 were identified. Product labelling for drugs approved by the European Medicines Agency (EMA) and US Food and Drug Administration (FDA) between 2016 and 2021 were also identified. A systematic literature review (SLR) was also performed. Documents reporting EQ-5D in the context of economic evaluation only were excluded. EQ-5D data were reported for COA in 195 of 1072 (18%) published TAs, with the majority reported for Germany (n = 138). The EQ-5Dvisual analogue scale (EQ-VAS) was reported most frequently, in 68% of all TAs, and accounted for 100% of Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) and 94% of Gemeinsamer Bundesausschuss (G-BA) TAs. In total, 320 drugs were approved or reviewed by the EMA and 735 by the FDA. Of these, 15 reported EQ-5D data from the EMA and 35 from the FDA; however, all EQ-5D data submitted to the FDA were reported in supporting documentation. Reporting of both EQ-5D index and EQ-VAS was most frequent, occurring in 32% of all documents. For the SLR, 329 of 4248 (8%) retrieved records were included. Reporting of both EQ-5D index and EQ-VAS was most frequent, occurring in 36% of studies. Clinical evaluation of recent drug approvals, based on regulatory, HTA and systematic literature reviews, demonstrated limited use of EQ-5D outside the context of economic evaluations. This may be due to the likelihood that the EQ-5D may lack sensitivity to detect improvement in conditions with small expected therapeutic benefit, or because the EQ-5D is not considered an adequate COA tool for clinical evaluation of treatment benefit. EQ-5D, as a COA, was more likely to be used in clinical evaluation of cancer drugs than drugs for treatment in any other disease category. HTA bodies were more likely to use the EQ-5D for COA, especially in Germany.

Patient-Reported Outcome Measures in the Food and Drug Administration Pilot Compendium: Meeting Today’s Standards for Patient Engagement in Development?

IntroductionGlobally, 7,000 rare diseases affect 300 million people, which poses challenges for developing treatments in these discrete patient populations. Developing medicines for rare diseases requires innovation, but despite regulatory incentives challenges persist for health technology assessment (HTA) and payers. Understanding the current regulatory and HTA decision-making landscape for orphan products is essential for all stakeholders.MethodsData on new active substances (NAS) approvals (2013 to 2022) by the European Medicines Agency, the United States Food and Drug Administration (FDA), the Pharmaceuticals and Medical Devices Agency (PMDA), Swissmedic, and the Therapeutic Goods Administration were collected to analyze the timing, approval pathways, and global rollout trends for orphan and non-orphan products. Data were collected from HTA agencies in Australia, England, France, Germany, the Netherlands, Poland, Scotland, and Sweden to explore synchronization in decision timing and first HTA decision. Comparative analysis encompassed decision frameworks and funding mechanisms for orphan products among HTA agencies.ResultsOrphan drug approvals increased in the past decade, with the FDA having the highest designation rate (55% for 2018 to 2022). Flexible pathways, mostly used by the FDA (92%) and the PMDA (100%), expedited orphan drug reviews. However, international submissions for orphan drugs experienced longer gaps than non-orphan drugs across the jurisdictions. Divergence in rollout timing to HTA and recommendation resulted from varied submission strategies and review processes. Only the Scottish Medicines Consortium had a dedicated orphan pathway. In England, the National Institute for Health and Care Excellence patient access and managed access agreements (84% of orphan recommendations) and cancer drug fund (45% of orphan recommendations) facilitated patient access.ConclusionsThe study shows a decade long rise in global orphan drug approvals, underpinned by regulatory flexibility, particularly by the FDA and the PMDA. Identified divergences in decision frameworks among regulatory and HTA agencies, as well as HTA agencies themselves, call for increased stakeholder alignment. This necessitates synchronizing evidence generation during development and improving decision frameworks for streamlined review and reimbursement processes.

SummaryBackgroundPatient‐reported outcome (PRO) measures historically used in inflammatory bowel disease have been considered inadequate to support future drug labelling claims by regulatory agencies.AimsTo develop PRO tools for use in Crohn's disease (CD) and ulcerative colitis (UC) following guidance issued by the US FDA and the ISPOR (International Society for Pharmacoeconomics and Outcomes Research).MethodsConcept elicitation and cognitive interviews were conducted in adult patients (≥18 years) across the United States and Canada. Semi‐structured interview guides were used to collect data, and interview transcripts were coded and analysed. Concept elicitation results were considered alongside existing literature and clinical expert opinion to identify candidate PRO items. Cognitive interviews evaluated concept relevance, interpretability and structure, and facilitated instrument refinement. Concept elicitation participants, except those with an ostomy, underwent centrally read endoscopy to assess inflammatory status.ResultsIn all, 54 participants (mean age: 46.2 years; 66.7% female) were included in the CD concept elicitation interviews. In total, 80 symptom concepts and 61 impact concepts were identified. After three waves of cognitive interviews, the 31‐item Symptoms and Impacts Questionnaire for CD (SIQ‐CD) was developed. In the UC concept elicitation phase, 53 participants were interviewed (mean age: 41.4 years; 49.1% female). In total, 79 symptoms concepts and 49 impact concepts were identified. Following two waves of cognitive interviews, the 29‐item Symptoms and Impacts Questionnaire for UC (SIQ‐UC) was developed. Both instruments include four symptom and six impact domains.ConclusionsWe developed PROs to support CD and UC drug labelling claims. Psychometric validation studies to evaluate instrument reliability and responsiveness are ongoing.

PRM217 - Use of Surrogate Outcomes in Health Technology Assessments (HTAs)

IntroductionOmics technologies enable the measurements of genes (genomics), mRNA (transcriptomics), proteins (proteomics) and metabolites (metabolomics) and thus proved to be valuable tools for personalized decision-making in clinical practice. For their evaluation, a health technology assessment (HTA) framework is not standardized and accepted, yet. Therefore, we aim at designing an omics-technologies HTA evaluation framework to facilitate their assessment, through a mixed-method approach. This work is part of the ExACT project, which aims to produce a range of tools to facilitate the implementation of precision health in clinical practice.MethodsA systematic review was conducted to identify the existing HTA frameworks used for the evaluation of omics-technologies. Desk research on the HTA agencies’ websites was performed to identify the reports on omics-technologies HTA evaluation frameworks used by these agencies. A questionnaire evaluating HTA agencies’ experience on evaluation of omics-technologies was designed. The new framework will be elaborated based on the findings from the three methodological steps, and will be validated through a Delphi process.ResultsTwenty-three articles were included in the systematic review. The main identified HTA frameworks were ACCE and “Evaluation of Genomic Applications in Practice and Prevention” (EGAPP). The desk research showed that these frameworks were seldom used by HTA agencies, which for the evaluation of omics-technologies mostly refer to the HTA Core Model®, mainly assessing the following domains: clinical effectiveness and economic evaluation. Data collection process of the questionnaire HTA agencies’ experience is in progress.ConclusionsAlthough two main HTA frameworks for the evaluation of omics-technologies have been identified, these frameworks are sporadically used by HTA agencies in their practice. The particular interest of HTA agencies on clinical effectiveness and economic evaluation, might potentially reflect the uncertainty and difficulties when evaluating omics-technologies. This could indicate that these HTA frameworks are not feasible and practical to be used in routine HTA agency processes for omics technologies, emphasizing the need for a new framework. Our methodological approach might contribute to the development of a new HTA framework, feasible and practical to use not only for HTA agencies.

Plain English summaryA number of health technology assessment (HTA) organisations have developed processes to engage patients in the assessment of new health technologies such as pharmaceuticals, diagnostic tests, devices or medical procedures. Typically, this involves the HTA agency providing an opportunity for patient advocates and their patient organisations (support groups for patients with a specific disease or condition) to provide submissions detailing experiences with the disease and the health technology that is being assessed. While some literature exists about how HTA agencies view the engagement of patients in the HTA process, it is not yet clear how the patient advocates and patient organisations themselves view this engagement. To answer this question, we surveyed the views of patient advocates who were members of patient organisations known to be engaged in the process of HTA or evidence-based practice. Snowballing – that is, passing on the survey invitation from individuals invited to take part in the survey to other individuals – occurred in one of the countries. The responses in this country provided a very useful comparison between the views of people who were appointed as the ‘patient representatives’ on an HTA committee with those who contributed input as part of the general patient organisation engagement process. Our findings identify gaps in understanding of the purpose of patient involvement and whether patient organisations felt their input made a difference, the information and support provided, and if and how feedback is given to the patient organisations. Our work can help inform further research as well as continuing improvements in HTA patient engagement processes.BackgroundPatient involvement in health technology assessment (HTA) processes is becoming more frequent. However, it is not clear how patient advocates and their disease-based patient organisations that are involved in HTA view their involvement. We report on the results of an international survey of patient advocates and members of patient organisations about their experiences and perceptions of that involvement.MethodsA 16-question survey was sent out to patient advocates and members of patient groups known to be involved in HTA processes or evidence-based practice. The survey consisted of open-ended questions focusing on respondent characteristics, stage and nature of involvement, support from HTA agencies for involvement, purpose of involvement, feedback on involvement, and whether the respondents felt that their input made a difference.ResultsOf 16 individuals who received the survey, 15 responded. Three, from Italy, Israel and Japan, were not involved in HTA in their country. Respondents from the following countries reported involvement in HTA processes: Canada, England, Scotland, and Wales, The Netherlands, Australia, Taiwan. The respondents indicated that HTA agencies reach out to them either actively or passively, and that their involvement is often at the appraisal stage of HTA. Typically, they reported involvement as either participants in committees or providers of submissions to HTA agencies. A wide range of approaches to supporting patient involvement by the HTA agencies was identified by respondents – including personal and telephone support, online resources, training and provision of information – but the level and type of support reported was uneven across jurisdictions. Not all respondents were clear on the purpose of their involvement in HTA, although some were able to cite specific examples of how their input made a difference; members of an HTA decision-making committee appeared to have a better understanding and were able to give examples. Feedback from HTA agencies to the patient groups on their submissions is often not provided.ConclusionsAlthough considerable progress has been made in terms of engaging patients and patient groups in HTA, gaps remain in how involvement is supported, including facilitating involvement, clarity on roles, two-way flow of information, and methods for enhancing communication between patient organisations and HTA agencies.

Recommendations by countries' health technology assessment (HTA) agencies are used to decide which new therapies warrant the allocation of limited health-care resources to make them available through publicly funded health systems. This process is of public health importance for balancing the dual aims of optimising patient outcomes while ensuring financial sustainability. We evaluated which factors affect HTA outcomes and the time to positive HTA outcome, focussing on the role of clinical benefit evaluated with the European Society for Medical Oncology-Magnitude of Clinical Benefit Scale (ESMO-MCBS). In this retrospective analysis, data were extracted from publicly available HTA reports and related sources from six country settings and their respective HTA agencies (Australia, Canada, England, France, the Canadian province of Quebec, and Scotland). We evaluated new cancer medicines for treating solid tumours in a non-curative setting with published ESMO-MCBS scores and that had been assessed by at least three HTA agencies between Jan 1, 2011, and Dec 31, 2020. Using ESMO-MCBS score as an independent variable, we did descriptive and multivariable regression analyses to evaluate: (1) factors associated with the time between marketing authorisation and positive (unrestricted [List] and restricted [List with Constraints]) HTA outcome; and (2) factors associated with HTA outcomes. 67 medicine-indication pairs used in non-curative settings were identified, totalling 360 HTA submissions (medicine-indication-country triplets) reviewed by the six HTA agencies. Factors significantly associated with a reduced interval between marketing authorisation and a positive (unrestricted or restricted) HTA outcome included a high ESMO-MCBS score (ie, 4 or 5, vs a low or average score of 1-3; hazard ratio [HR] per 1 month increment 1·42 [95% CI 1·11-1·81], p=0·0055), parallel review (vs standard marketing authorisation process; HR 1·69 [1·13-2·54], p=0·011), having a risk-sharing agreement or special funding arrangements (vs no funding agreement, HR 4·62 [95% CI 2·51-8·51], p<0·0001, and HR 4·16 [2·03-8·50], p=0·0001, respectively), and assessment by particular HTA agencies (pan-Canadian Oncology Drug Review vs National Institute for Health and Care Excellence [NICE], HR 2·82 [1·68-4·75], p=0·0001; and Haute Autorité de Santé vs NICE, HR 5·70 [2·87-11·33], p<0·0001). Accelerated marketing authorisation was significantly associated with a longer time to positive HTA outcome (vs standard authorisation process; HR 0·70 [95% CI 0·51-0·95], p=0·024). Positive HTA outcomes (both unrestricted and restricted) were significantly associated with a high ESMO-MCBS score (vs low or average ESMO-MCBS score; relative risk ratio [RRR] 14·10 [95% CI 3·54-56·20], p=0·0002, and RRR 4·52 [1·90-10·75], p=0·0006, respectively) and acknowledgment of unmet medical need (vs unmet need not recorded, RRR 22·73 [5·51-93·73], p<0·0001, and RRR 1·87 [1·18-2·97], p=0·0075, respectively). By contrast, positive HTA outcomes (unrestricted and restricted) were inversely associated with uncertainties regarding inputs to economic models informing HTA submissions (vs uncertainties not recorded, RRR 0·28 [0·10-0·78], p=0·014, and RRR 0·45 [0·25-0·82], p=0·010, respectively). Regarding country-relevant effects, inverse associations with positive HTA outcomes (both unrestricted and restricted) were observed for assessment in Quebec (vs England; RRR 1·15×10-6 [1·44×10-7-9·09×10-6], p<0·0001, and RRR 0·33 (0·24-0·46), p<0·0001, respectively) and for assessment in Australia (vs England; RRR 1·78×10-6 [1·04×10-8-3·00×10-4], p<0·0001, and RRR 0·30 [0·15-0·61], p=0·0008, respectively). Several factors informed HTA outcomes for new cancer medicines. A high ESMO-MCBS score, defined as indicating substantial clinical benefit, increased the likelihood of a positive HTA outcome and shortened the interval between marketing authorisation and HTA outcome, and this association was not affected by other variables. Additional factors informing HTA outcomes include evidence uncertainties and unmet medical need. Country-relevant differences exist in the time-to-HTA outcome and the propensity of some countries to achieve positive (restricted or unrestricted) outcomes compared with others. None.

Drug expenditures in the US are higher than in any other country and are projected to continue increasing, so US health systems may benefit from evaluating international regulatory and reimbursement decision-making of new drugs. To evaluate regulatory decisions and health technology assessments (HTAs) in Australia, Canada, and the UK regarding new drugs approved by the US Food and Drug Administration (FDA) in 2017 through 2020, as well as to estimate the US cost per patient per year for drugs receiving negative recommendations. In this cross-sectional study, recommendations issued by agencies in Australia, Canada, and the UK were collected for new drugs approved by the FDA in 2017 through 2020. All data were current as of May 31, 2022. Authorizations and HTAs in selected countries. All FDA-approved drugs were matched by active ingredient to decision summary reports published by drug regulators and HTA agencies in Australia, Canada, and the UK. Regulatory approval concordance and reasons for negative recommendations were assessed using descriptive statistics. For drugs not recommended by an international agency, the annual US drug cost per patient was estimated from FDA labeling and wholesale acquisition costs. The FDA approved 206 new drugs in 2017 through 2020, of which 162 (78.6%) were granted marketing authorization by at least 1 other regulatory agency at a median (IQR) delay of 12.1 (17.7) months following US approval. Conversely, 5 FDA-approved drugs were refused marketing authorization by an international regulatory agency due to unfavorable benefit-to-risk assessments. An additional 42 FDA-approved drugs received negative reimbursement recommendations from HTA agencies in Australia, Canada, or the UK due to uncertainty of clinical benefits or unacceptably high prices. The median (IQR) US cost of the 47 drugs refused authorization or not recommended for reimbursement by an international agency was $115 281 ($166 690) per patient per year. Twenty drugs were for oncology indications, and 36 were approved by the FDA through expedited regulatory pathways or the Orphan Drug Act. This cross-sectional study assessed reasons for which drugs recently approved by the FDA were refused marketing authorization or not recommended for public reimbursement in other countries. Drugs with limited international market presence may require close examination by US health care professionals and health systems.

A central function of health technology assessment (HTA) agencies is the production of HTA reports to support evidence-informed policy and decision making. HTA agencies are interested in understanding the mechanisms of HTA impact, which can be understood as the influence or impact of HTA report findings on decision making at various levels of the health system. The members of the International Network of Agencies for HTA (INAHTA) meet at their annual Congress where impact story sharing is one important activity. This paper summarizes four stories of HTA impact that were finalists for the David Hailey Award for Best Impact Story.The methods to measure impact include: document review; claims analysis and review of reimbursement status; citation analysis; qualitative evaluation of stakeholders' views; and review of media response. HTA agency staff also observed changes in government activities and priorities based on the HTA. Impact assessment can provide information to improve the HTA process, for example, the value of patient and clinician engagement in the HTA process to better define the assessment question and literature reviews in a more holistic and balanced way.HTA reports produced by publicly funded HTA agencies are valued by health systems around the globe as they support decision making regarding the appropriate use, pricing, reimbursement, and disinvestment of health technologies. HTAs can also have a positive impact on information sharing between different levels of government and across stakeholder groups. These stories show how HTA can have a significant impact, irrespective of the health system and health technology being assessed.

ObjectivesTo identify and describe patient experience data (PED)-related guidance issued by regulatory and health technology assessment (HTA) agencies in North America, Europe, and Asia Pacific.MethodsNational agency websites were manually searched (October 2021, updated October–December 2023) to identify standalone PED guidance, and general submission/clinical trial or disease-specific guidance in four therapy areas, including PED-related recommendations. Identified documents were reviewed for requirements/expectations on types of PED; concepts of interest and instruments/tools; validation, analysis, and interpretation; study design, and endpoint definitions.ResultsA total of 34 regulatory and 21 HTA documents were reviewed across 7 and 11 agencies, respectively. Most regulatory agencies expected data collected via clinical outcome assessments; concepts of interest were similar; specific tools were not usually recommended, although the use of validated instruments was expected; and meaningful within-patient change was generally considered key to interpretation. HTA guidance on PED varied primarily based on country requirement for utility estimates to inform economic evaluation (little/no guidance beyond the concept of interest) versus interest in PED on signs, symptoms, or disease impact on functioning (generally more comprehensive details on PED expectations).ConclusionThrough PED, patient voice is an increasingly important factor in regulatory, access, and reimbursement decision-making. Regulatory and HTA agencies in North America, Europe, and Asia Pacific have varying expectations regarding PED. This misalignment can necessitate the collection of multiple types of PED to meet all requirements (resulting in complex clinical trial protocols and significant patient burden), and lead to differences in data interpretation or gaps in the expected data. Therefore, PED measurement standards should be harmonized globally, and the collection of required PED should be ensured early during drug development to ensure decisions can be made based on valid and reliable data.