Methodological changes made to NIHR-funded randomised controlled trials: review of 100 trials.

Utilisation d’outils de la démarche qualité pour structurer un projet visant à améliorer l’évaluation et la prise en charge de la douleur des patients atteints de cancer dans un hôpital local à partir d’une réflexion sur l’application du plan cancer

Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles

RCTs are a key block in the evidence pyramid, but their quality relies on detailed, consistent reporting, and one best-practice standard is prospective registration. Prospective trial registration was intended to reduce publication bias, and adherence to a prespecified protocol helps limit bias from selective reporting; any protocol or end point changes should be transparently documented and justified. However, the degree to which articles published in the leading journals on orthopaedic surgery comply with this best-practice standard has, to our knowledge, not been evaluated. (1) Do RCTs published in leading, general-interest orthopaedic surgery journals comply with best practices regarding prospective clinical trial registration? (2) Do major discrepancies exist between registered protocols and published orthopaedic RCTs? (3) Are there specific study types that are more likely to demonstrate discrepancies? A review was performed on RCTs published in the top five general-interest orthopaedic surgery journals, based on the 2022 scientific journal rankings (from SciMago): Journal of Bone and Joint Surgery, Bone and Joint Journal, Clinical Orthopaedics and Related Research®, Journal of the American Academy of Orthopaedic Surgeons (JAAOS), and Acta Orthopaedica. During the study period, all journals maintained editorial policies requiring prospective clinical trial registration as a condition of consideration for publication except for JAAOS. A systematic search on PubMed retrieved 705 potential publications, of which 324 RCTs fulfilled the inclusion criteria. For each trial, nine essential elements from the 24-item WHO minimum data set were extracted and compared between the published article and its trial registry entry, focusing on health condition, intervention, sample size, outcomes, and eligibility criteria. To answer our first question regarding compliance, we audited each article to identify the presence of a registry code. For our second question, we performed a side-by-side comparison of nine essential elements from the WHO trial registration data set (including primary outcomes, sample size, and eligibility criteria) to identify discrepancies between the registry and the final publication. Finally, to address our third question, we used chi-square tests to determine whether study characteristics, such as country of origin or subspecialty, were associated with higher rates of reporting shifts. Most orthopaedic RCTs published in leading journals complied with registration standards, with 95% (309 of 324) having an identifiable registry entry. However, 2% (8 of 324) were published without any registry identifier or justification for its absence, and 2% (7 of 324) were identified as long-term follow-up visits that did not have unique prospective entries. Major discrepancies between registered protocols and published manuscripts were frequent. Discrepancies in the sample size occurred in 33% (102 of 309) of trials. Discrepancies in the primary outcome occurred in 25% (78 of 309) of trials. Discrepancies in the secondary outcome occurred in 60% (185 of 309) of trials. Discrepancies in the inclusion criteria occurred in 33% (102 of 309) of trials. Discrepancies in the exclusion criteria occurred in 53% (165 of 309) of trials. Trials conducted in the United States or as multicenter international collaborations were more likely to update their final results in the registry compared with single-country trials conducted outside the US (37% versus 10%; p < 0.001). No other study characteristics, including publication year or subspecialty, were associated with the presence of reporting discrepancies. Prospective registration has become the standard for RCTs in high-impact orthopaedic journals. However, our findings suggest that a gap still exists between having a registry and the accuracy of the information contained within it. These findings suggest that registration is often treated as a procedural requirement rather than a rigorous commitment to a fixed study protocol. The orthopaedic research community should adopt stricter standards for trial registration, reporting, and verification of registry entries to reduce undisclosed protocol changes and improve confidence in published evidence.

Health care technology assessment and transfer

Background:Clinical trials evaluating different management strategies for pulmonary sarcoidosis may measure different outcomes. This heterogeneity in outcomes can lead to waste in research due to the inability to compare and combine data. Core outcome sets (COS) have the potential to address this issue and here we describe a systematic review of outcomes as the first step in the development of a COS for pulmonary sarcoidosis research.Methods:A search of clinical trial registries for phase II, III and IV trials of pulmonary sarcoidosis was undertaken along with a rapid review of the patient perspective literature. Each study was screened for eligibility and outcomes extracted verbatim from the registry entry or publication then reviewed, grouped and categorised using the COMET taxonomy.Results:36 trial registry entries and 6 studies on patients’ perspective of pulmonary sarcoidosis were included reporting 56 and 82 unique outcomes respectively across 23 domains. The most frequently reported outcome domain was “respiratory, thoracic and mediastinal outcomes”. However, the patients’ perspective literature identified outcomes in the “personal circumstances” and “societal/carer burden” domains that were not reported in any of the included trial registrations.Conclusions:Using both clinical trial registry data and published literature on patients’ perspective has allowed rapid review of outcomes measured and reported in pulmonary sarcoidosis research. The use of multiple sources has led to the development of a comprehensive list of outcomes that represents the first step in the development of a COS for use in future pulmonary sarcoidosis research.

To identify investigated interventions for COVID-19 prevention or treatment via trial registry entries on planned or ongoing randomised clinical trials. To assess these registry entries for recruitment status, planned trial size, blinding and reporting of mortality. We identified trial registry entries systematically via the WHO International Clinical Trials Registry Platform and 33 trial registries up to June 23, 2020. We included relevant trial registry entries for randomized clinical trials investigating medical preventive, adjunct or supportive therapies and therapeutics for treatment of COVID-19. Studies with non-random and single-arm design were excluded. Trial registry entries were screened by two authors independently and data were systematically extracted. We included 1303 trial registry entries from 71 countries investigating 381 different single interventions. Blinding was planned in 47% of trials. Sample size was >200 participants in 40% of trials and a total of 611,364 participants were planned for inclusion. Mortality was listed as an outcome in 57% of trials. Recruitment was ongoing in 54% of trials and completed in 8%. Thirty-five percent were multicenter trials. The five most frequent investigational categories were immune modulating drugs (266 trials (20%)), unconventional medicine (167 trials (13%)), antimalarial drugs (118 trials (9%)), antiviral drugs (100 trials (8%)) and respiratory adjuncts (78 trials (6%)). The five most frequently tested uni-modal interventions were: chloroquine/hydroxychloroquine (113 trials with 199,841 participants); convalescent plasma (64 trials with 11,840 participants); stem cells (51 trials with 3,370 participants); tocilizumab (19 trials with 4,139 participants) and favipiravir (19 trials with 3,210 participants). An extraordinary number of randomized clinical trials investigating COVID-19 management have been initiated with a multitude of medical preventive, adjunctive and treatment modalities. Blinding will be used in only 47% of trials, which may have influence on future reported treatment effects. Fifty-seven percent of all trials will assess mortality as an outcome facilitating future meta-analyses.

AimTo identify investigated interventions for COVID-19 prevention or treatment via trial registry entries on planned or ongoing randomised clinical trials. To assess these registry entries for recruitment status, planned trial size, blinding and reporting of mortality.MethodsWe identified trial registry entries systematically via the WHO International Clinical Trials Registry Platform and 33 trial registries up to June 23, 2020. We included relevant trial registry entries for randomized clinical trials investigating medical preventive, adjunct or supportive therapies and therapeutics for treatment of COVID-19. Studies with non-random and single-arm design were excluded. Trial registry entries were screened by two authors independently and data were systematically extracted.ResultsWe included 1303 trial registry entries from 71 countries investigating 381 different single interventions. Blinding was planned in 47% of trials. Sample size was >200 participants in 40% of trials and a total of 611,364 participants were planned for inclusion. Mortality was listed as an outcome in 57% of trials. Recruitment was ongoing in 54% of trials and completed in 8%. Thirty-five percent were multicenter trials. The five most frequent investigational categories were immune modulating drugs (266 trials (20%)), unconventional medicine (167 trials (13%)), antimalarial drugs (118 trials (9%)), antiviral drugs (100 trials (8%)) and respiratory adjuncts (78 trials (6%)). The five most frequently tested uni-modal interventions were: chloroquine/hydroxychloroquine (113 trials with 199,841 participants); convalescent plasma (64 trials with 11,840 participants); stem cells (51 trials with 3,370 participants); tocilizumab (19 trials with 4,139 participants) and favipiravir (19 trials with 3,210 participants).ConclusionAn extraordinary number of randomized clinical trials investigating COVID-19 management have been initiated with a multitude of medical preventive, adjunctive and treatment modalities. Blinding will be used in only 47% of trials, which may have influence on future reported treatment effects. Fifty-seven percent of all trials will assess mortality as an outcome facilitating future meta-analyses.

This study aims to review the development of health technology assessment (HTA), including the socioeconomic context, outputs, and policy utilization in the Thai setting. This study was conducted through extensive document reviews including these published in both domestic and international literature. Evidence suggests that contextual elements of the health system, especially the country's economic status and health financing reforms, as well as their effects on government budgeting for medical and public health services, played an important role in the increasing needs and demands for HTA information among policy makers. In the midst of substantial economic growth during the years 1982 to 1996, several studies reported the rapid diffusion and poor distribution of health technologies, and inequitable access to high-cost technology in public and private hospitals. At the same time, economic analysis and its underpinning concept of efficiency were suggested by groups of scholars and health officials to guide national policy on the investment in health technology equipment. Related research and training programs were subsequently launched. However, none of these HTA units could be institutionalized into national bodies. From 1997 to 2005, an economic recession, followed by the introduction of a universal health coverage plan, triggered the demands for effective measures for cost containment and prioritization of health interventions. This made policy makers and researchers at the Ministry of Public Health (MOPH) pay increasing attention to economic appraisals, and several HTA programs were established in the Ministry. Despite the rising number of Thai health economic publications, a major problem at that period involved the poor quality of studies. Since 2006, economic recovery and demands from different interests to include expensive technologies in the public health benefit package have been crucial factors promoting the role of HTA in national policy decisions. Meanwhile, HTA capacity has been strengthened through the establishment of many health economic and HTA initiatives. An illustration of the work and contributions of the Health Intervention and Technology Assessment Program (HITAP) is provided. In this phase, HTA policy integration has been enhanced through different mechanisms and organizations. Over the past two decades a notable progression has been made in relation to the capacity building of HTA research and its policy utility in Thailand. Such development has been shaped by multiple factors. It is anticipated that experience gained among academics, health officials, and civil society organizations will be helpful not only in sustaining the momentum but also in improving formal HTA systems in the future.

Capacity is limited in the developing world to conduct cost-effectiveness analysis (CEA) of health interventions. In Thailand, there have been concerted efforts to promote evidence-based policy making, including the introduction of economic appraisals within health technology assessment (HTA). This paper reviews the experience of this lower middle-income country, with an emphasis on the creation of the Health Intervention and Technology Assessment Program (HITAP), including its mission, management structures and activities. Over the past 3 decades, several HTA programmes were implemented in Thailand but not sustained or developed further into a national institute. As a response to increasing demands for HTA evidence including CEA information, the HITAP was created in 2007 as an affiliate unit of a semi-autonomous research arm of the Ministry of Public Health. An advantage of this HTA programme over previous initiatives was that it was hosted by a research institute with long-term experience in conducting health systems and policy research and capacity building of its research staff, and excellent research and policy networks. To deal with existing impediments to conducting health economics research, the main strategies of the HITAP were carefully devised to include not only capacity strengthening of its researchers and administrative staff, but also the development of essential elements for the country's health economic evaluation methodology. These included, for example, methodological guidelines, standard protocols and benchmarks for resource allocation, many of which have been adopted by national policy-making bodies including the three major public health insurance plans. Networks and collaborations with domestic and foreign institutes have been sought as a means of resource mobilization and exchange. Although the HITAP is well financed by a number of government agencies and international organizations, the programme is vulnerable to shortages of qualified research staff, as most staff work on a part-time or temporary basis. To enhance the utilization of its research findings by policy makers, practitioners and consumers, the HITAP has adopted the principles of technical excellence, policy relevance, transparency, effective communication and participation of key stakeholders. These principles have been translated into good practice at every step of HTA management. In 2007 and 2008, the HITAP carried out assessments of a wide range of health products, medical procedures and public health initiatives. Although CEA and other economic evaluation approaches were employed in these studies, the tools and underlying efficiency goal were considered inadequate to provide complete information for prioritization. As suggested by official stakeholders, some of the projects investigated broader issues of management, feasibility, performance and socio-political implications of interventions. As yet, it is unclear what role HITAP research and associated recommendations have played in policy decisions. It is hoped that the lessons drawn on the creation of the HITAP and its experience during the first 2 years, as well as information on its main strategies and management structures, may be helpful for other resource-constrained countries when considering how best to strengthen their capacity to conduct economic appraisals of health technologies and interventions.

IntroductionWith ever increasing burden of disease and limited resources, health technology assessment (HTA) is required for efficient resources allocation and priority setting in healthcare. The objective of this study was to establish the baseline HTA evidence generation and use in Zimbabwe.MethodsIn 2019, we convened a stakeholder workshop on HTA at the University of Zimbabwe. Presentations on HTA processes, current healthcare reimbursement model, priority setting in the Ministry of Health and selection of medicines into the treatment guidelines, were done by the experts. We adapted the Health Intervention and Technology Assessment Program questionnaire for situational analysis of HTA introduction at national level and administered it among the workshop participants. We report the baseline information on HTA situation, the need, demand and supply of HTA in Zimbabwe obtained from the presentations and responses from workshop participants.ResultsA total of 33 participants attended the workshop. Participants indicated that there is no formal HTA agency or process in Zimbabwe. The selection of medicines into treatment guideline is determined by disease burden, safety, efficacy and cost data, and it is done by a group of experts. The Association of Healthcare Funders of Zimbabwe (AFHOZ) reported that private healthcare funders use resource-based relative value scale system to determine tariffs and reimbursement levels. The regulator requires safety, efficacy and product quality data for the registration of medicines. Transparency in decision-making, registration of heath technology and formulation of essential medicines and treatment guidelines were reported as the major needs of HTA. The major users of HTA outputs were reported as medicines regulator, AFHOZ and Ministry of Health. Key suppliers of HTA evidence are academic and clinical research institutions and healthcare workers. Lack of training in health economics was cited as the major challenge to supply of HTA evidence.ConclusionsThere is a need to institute a formal, systematic and transparent processes of determining value of health technologies.

Publication of complete trial results is essential if people are to be able to make well-informed decisions about health care. Selective reporting of randomised controlled trials (RCTs) is a common problem. To systematically review studies of cohorts of RCTs to compare the content of trial reports with the information contained in their protocols, or entries in a trial registry. We conducted electronic searches in Ovid MEDLINE (1950 to August 2010); Ovid EMBASE (1980 to August 2010); ISI Web of Science (1900 to August 2010) and the Cochrane Methodology Register (Issue 3, 2010), checked reference lists, and asked authors of eligible studies to identify further studies. Studies were not excluded based on language of publication or our assessment of their quality. Published or unpublished cohort studies comparing the content of protocols or trial registry entries with published trial reports. Data were extracted by two authors independently. Risk of bias in the cohort studies was assessed in relation to follow up and selective reporting of outcomes. Results are presented separately for the comparison of published reports to protocols and trial registry entries. We included 16 studies assessing a median of 54 RCTs (range: 2 to 362). Twelve studies compared protocols to published reports and four compared trial registry entries to published reports. In two studies, eligibility criteria differed between the protocol and publication in 19% and 100% RCTs. In one study, 16% (9/58) of the reports included the same sample size calculation as the protocol. In one study, 6% (4/63) of protocol-report pairs gave conflicting information regarding the method of allocation concealment, and 67% (49/73) of blinded studies reported discrepant information on who was blinded. In one study unacknowledged discrepancies were found for methods of handling protocol deviations (44%; 19/43), missing data (80%; 39/49), primary outcome analyses (60%; 25/42) and adjusted analyses (82%; 23/28). One study found that of 13 protocols specifying subgroup analyses, 12 of these 13 trials reported only some, or none, of these. Two studies found that statistically significant outcomes had a higher odds of being fully reported compared to nonsignificant outcomes (range of odds ratios: 2.4 to 4.7). Across the studies, at least one primary outcome was changed, introduced, or omitted in 4-50% of trial reports. Discrepancies between protocols or trial registry entries and trial reports were common, although reasons for these were not discussed in the reports. Full transparency will be possible only when protocols are made publicly available or the quality and extent of information included in trial registries is improved, and trialists explain substantial changes in their reports.

Introduction: To make more efficient use of limited resources, Vietnam incorporated health technology assessment (HTA) into the decision-making process for the health insurance benefit package in 2014. We evaluated progress in HTA institutionalization in Vietnam based on the theoretical framework developed by the National Institute for Health and Care Excellence and the Health Intervention and Technology Assessment Program, identified negative and conducive factors for HTA development, and finally suggested policy recommendations that fit the Vietnamese context. Methods: Semi-structured in-depth qualitative interviews were conducted between December 2017 and March and April 2018 with a purposive sample of 24 stakeholders involved in decision-making for health insurance reimbursement. We employed thematic analysis to examine themes within the data. Results: Despite a variety of activities (e.g., training and advising/mentoring) and a substantial level of output (e.g., policy statements, focal points assigned, and case studies/demonstration projects), Vietnam has not yet reached the policy decision stage based on HTA with scientific integrity and active stakeholder participation. Most respondents, except some clinicians, supported the use of HTA. The lack of capacity of human resources in the government sector and academia, the limited data infrastructure, the absence of guidelines, the government’s interest in immediate budget-saving, and public resistance were identified as barriers to the advancement of HTA. Conclusions: A structured data repository, guidelines based on the Vietnamese context for both policy decision-making at the central level and daily clinical decision-making at the micro-level, and integration of a participatory process into HTA are suggested as priorities for HTA institutionalization in Vietnam.

Regarding Health Technology Assessment (HTA) represents a form of policy research which considers the short and long term consequences of health technologies. The first steps towards HTA were taken in Romania more than 15 years ago, but the effects seem insignificant even though they have never been measured. After 2011 HTA re-entered on the Romanian health care reforms’ agenda and it started the design of the legal framework in order to develop the HTA system. The scope of this article is to briefly present some key principles and aspects which could represent the fundamentals of a quick development of HTA in the following years. This article is not meant to present an evaluation of the HTA steps which have been done so far. In order to develop a HTA system there is a need for the involvement of key policy making stakeholders who have to express their willingness and demand for the usage of data generated by the HTA system for the decision-making process. Also, there is a need of resources (human, financial, organisational) and for capacity-building of institutions who will offer the ground to develop the HTA. In Romania, the willingness to develop the HTA system is expressed in the health policies (specifically in the Government Program for 2013-2016), but there is still a lot to do in terms of designing the legal framework and the development of required institutions for an HTA system. In this context, there are four key aspects which could facilitate the introduction and functioning of the HTA in Romania, grouped under the „Four P” (Partnership, Pragmatism, Predictability and Praise), aspects detailed in the context of Romanian health care system. The application of these „Four P” has to be coupled with the health policies in order to integrate the HTA system with the continuous reform of the health services: public financing, basic package, evaluation of quality etc. And to all these aspects there is a need to associate change champions, leaders who believe in HTA and who could become pioneers and development catalysts of HTA in Romania if they get support from the political environment. Keywords: HTA - Health Technology Assessment, health policy, health reform, resources allocation.

. As such, it aims to contribute to the NIHR's mission of improving the health and wealth of the nation by funding research that assesses the clinical and cost-effectiveness of healthcare treatments in comparison with the current best alternative(s), and that is therefore immediately useful to patients, clinical practice, and policy or decision-makers. More information about the programme can be found on the NIHR website. A logic model is a visual way of showing how an activity, programme or intervention is expected to work and bring about the benefits and changes it intends to achieve. By summarising the core elements, a logic model can be used to support programme planning, implementation and evaluation. NIHR logic models representin a linear flow diagramthe key activities, outputs, outcomes and impacts of each funding programme as a series of logical steps.

Some considerations on target estimands for health technology assessment.