Methodological Quality Of Economic Evaluations Research Articles

Cost-effectiveness analysis of physical activity interventions for people with schizophrenia or bipolar disorder: systematic review.

Clinical guidelines recommend providing physical activity interventions (PAIs) to people with schizophrenia or bipolar disorder for weight management. However, the cost-effectiveness of PAIs is unknown. To evaluate the availability and methodological quality of economic evaluations of PAIs for people with schizophrenia or bipolar disorder. Four databases (MEDLINE, Embase, PsycInfo and Scopus) were searched on 5 July 2022. Based on the retrieved studies, forward and backward citation searches were conducted. Two reviewers independently selected studies for inclusion. Study quality was assessed using the Drummond checklist. Review results were presented using narrative synthesis. Fourteen articles reporting nine studies were included. All included studies assessed PAIs within a multicomponent lifestyle intervention. Mixed findings were reported on the cost-effectiveness of multicomponent lifestyle intervention: three studies reported it as cost-effective; four studies reported it as not cost-effective; and two studies did not conclude whether it was cost-effective or not. Very limited evidence suggests that certain patient subgroups might be more likely to benefit from multicomponent lifestyle interventions with a PAI component: men; individuals with comorbid type 2 diabetes; and individuals who have been psychiatric hospital in-patients for ≥1 year. The quality of included studies ranged from moderate to high. The current economic evidence suggests that not all modalities of multicomponent lifestyle intervention including a PAI component are cost-effective for people with schizophrenia or bipolar disorder; and not all people with schizophrenia or bipolar disorder would benefit equally from the intervention. Future research is urgently needed to identify the cost-effective modality of PAI for different patient subgroups.

Cost-Effectiveness of Arthroplasty Management in Hip and Knee Osteoarthritis: a Quality Review of the Literature

Musculoskeletal disorders are a leading contributor to healthcare resource use and reduced quality of life, particularly in osteoarthritis of the hip/knee. Consequently, the volume of arthroplasty procedures continues to rise and there is a critical need to identify interventions that provide value-based care. Economic evaluations that assess both cost and health outcomes simultaneously allow clinicians and policymakers to make informed decisions regarding cost-effective interventions. The purpose of this review was to summarize and evaluate the methodological quality of economic evaluations of hip/knee interventions in arthroplasty. We identified 384 studies. Two-thirds did not meet the criteria for full economic evaluations (compared costs independently). Several studies (28%) inappropriately concluded interventions were cost-effective. Using a validated quality assessment tool, we determined the quality of full economic evaluations in hip/knee arthroplasty is generally good; however, important areas for quality improvement are related to reporting perspective, costing methodology, and funding. Value-based care in arthroplasty relies on the quality of the literature. Although the quality is generally good, few full economic evaluations are published in arthroplasty. Yet, these are necessary to appropriately infer intervention cost-effectiveness. Unfortunately, clinical decisions continue to be made using highly cited low-quality or partial economic evaluations.

Methodological Quality of Economic Evaluations in Integrated Care: Evidence from a Systematic Review.

Introduction:The aim of this review is to systematically assess the methodological quality of economic evaluations in integrated care and to identify challenges with conducting such studies.Theory and methods:Searches of grey-literature and scientific papers were performed, from January 2000 to December 2018. A checklist was developed to assess the quality of economic evaluations. Authors’ statements of challenges encountered during their evaluations were qualitatively coded.Results:Forty-four articles were eligible for inclusion. The review found that study design, measurement of cost and outcomes, statistical analysis and presentation of data were the areas with most quality variation. Authors identified challenges mostly related to time horizon of the evaluation, inadequate or lack of comparator group, contamination bias, and a post-hoc evaluation culture.Discussion:Our review found significant differences in quality, with some studies showing poor methodological rigor; challenging conclusions on the cost-effectiveness of integrated care.Conclusion:It is essential for evaluators to use best-practice standards when planning and conducting economic evaluations, in order to build a reliable evidence base for decision-making in integrated care.

Assessment of the Methodological Quality of Economic Evaluations: Main Mistakes

Results of the assessment of methodological quality of economic evaluations (EE), submitted to justify the inclusion/exclusion of drugs in the lists of drugs for medical use (list of vital and essential drugs, additional medication supply list and expensive drug list), are given in the article. The study was conducted in 2018 in the FSBI “Center for healthcare quality assessment and control” of the Ministry of Health of the Russian Federation. Assessment was performed based on 12 criteria in accordance with the requirements to the methodological quality of economic evaluations, described in the draft Order of the Government of the Russian Federation, No. 871, and then were approved in the Order of the Government of the Russian Federation, No. 871, state on 20.11.2018. The distribution of EE based on the number of criteria they did not meet, and also the frequency of made mistakes for each criterion are presented. Main reasons for EE not meeting the requirements to methodological quality, prescribed by the Order of the Government of the Russian Federation, No. 871, are analyzed.

Assessing the Methodological Quality of Economic Evaluations in Integrated Care: Evidence from a Systematic Review

Introduction: In an era of fiscal constraint, economic evaluations are increasingly playing a role in resource allocation decisions to invest in integrated care IC approaches. Posing significant ethical concerns are healthcare investments informed by misleading or biased evidence due to poor methodological quality. We conducted a systematic review to assess the quality of economic evaluations in IC interventions, and identified the challenges associated with the evaluations. Theory/Methods: English publications that reported the concepts of “integrated care” and “economic evaluations”, were searched from the period 2000 to August 2017 in the following databases: PubMed, EMBASE, CINAHL, Web of Science and Scopus. The methodological rigor of the studies was assessed independently by two reviewers using a 30 item-check list adapted from the Consolidated Health Economic Evaluation Reporting Standards CHEERS. Descriptive statements about the challenges experienced when evaluating IC interventions, as reported by authors, were also thematically analysed. Results: We included 40 studies in this review, however, approximately half were considered full economic evaluations; often as part of randomized control trials. Four studies that stated that their research objectives were to examine the cost-effectiveness of the IC interventions, did not perform an incremental cost-effectiveness ratio or a net benefit regression analysis. Of the cohort studies that performed cost comparisons, only one study controlled for potential confounding differences between the intervention and standard care. Valuation of the effects of the IC intervention in the full economic evaluations was often measured using quality-adjusted-life years. In some scenarios where this was challenging to ascertain; particularly in large cohort studies, natural units were used, such as estimated depression-free days. There was variability in the perspectives used, with a minority of the evaluations considering the societal impacts, such as cost shifting between sectors or the economic burden on caregivers. The long-term costs and effects of the interventions were limited in the analyses due to the short observation periods range of 3 months to 5 years. However, some studies attempted to address this through modelling estimated lifetime costs and health outcomes associated with the intervention. Discussions: The complex nature of IC can challenge the application of traditional economic evaluation, making it important to identify these methodological gaps and learn from best-practices. However, other factors may explain the heterogeneous quality of the studies. These may include; the need for further guidance on the methods, the substantial human and financial resources required for data collection and conducting the evaluations, and a lack of economic evaluation culture. Conclusions: This review intended to highlight some of the gaps and opportunities for applying the fundamentals of economic evaluation to IC interventions. Benchmarking economic evaluations against guidelines such as the CHEERS checklist can be a potential catalyst for more consistent and higher quality evidence for decision makers. Limitations: A limitation is the subjective interpretations of the quality of the studies by the reviewers, which could influence how they were rated based on the criteria. However, a major strength is that two-blinded reviewers assessed the studies and based the quality scoring on a consensus.

The insights of health and welfare professionals on hurdles that impede economic evaluations of welfare interventions

ABSTRACTBackground: Four hurdles associated with economic evaluations in welfare interventions were identified and discussed in a previous published literature review. These hurdles include (i) ‘Ignoring the impact of condition-specific outcomes’, (ii) ‘Ignoring the impact of QoL externalities’, (iii) ‘Calculation of costs from a too narrow perspective’ and (iv) ‘The lack of well-described & standardized interventions’. This study aims to determine how healthcare providers and social workers experience and deal with these hurdles in practice and what solutions or new insights they would suggest.Methods: Twenty-two professionals of welfare interventions carried out in Flanders, were interviewed about the four described hurdles using a semi-structured interview. A thematic framework was developed to enable the qualitative analysis. The analysis of the semi-structured interviews was facilitated through the use of the software program QRS NVivo 10.Results: The interviews revealed a clear need to tackle these hurdles. The interviewees confirmed that further study of condition-specific outcomes in economic evaluations are needed, especially in the field of mental health and stress. The proposed dimensions for the condition-specific questionnaires varied however between the groups of interviewees (i.e. general practitioners vs social workers). With respect to QoL externalities, the interviewees confirmed that welfare interventions have an impact on the social environment of the patient (friends and family). There was however no consensus on how this impact of QoL externalities should be taken into account in welfare interventions. Professionals also suggested that besides health care costs, the impact of welfare interventions on work productivity, the patients’ social life and other items should be incorporated. Standardization appears to be of limited added value for most of the interviewees because they need a certain degree of freedom to interpret the intervention. Furthermore, the target population of the interventions is diverse which requires a tailor-made approach.Conclusion: This qualitative research demonstrated that these hurdles occur in practice. The proposed solutions for these hurdles can contribute to the improvement of the methodological quality of economic evaluations of welfare interventions.

Assessment of Methodological Quality of Economic Evaluations in Belgian Drug Reimbursement Applications

ObjectivesThis paper aims to assess the methodological quality of economic evaluations included in Belgian reimbursement applications for Class 1 drugs.Materials and MethodsFor 19 reimbursement applications submitted during 2011 and Spring 2012, a descriptive analysis assessed the methodological quality of the economic evaluation, evaluated the assessment of that economic evaluation by the Drug Reimbursement Committee and the response to that assessment by the company. Compliance with methodological guidelines issued by the Belgian Healthcare Knowledge Centre was assessed using a detailed checklist of 23 methodological items. The rate of compliance was calculated based on the number of economic evaluations for which the item was applicable.ResultsEconomic evaluations tended to comply with guidelines regarding perspective, target population, subgroup analyses, comparator, use of comparative clinical data and final outcome measures, calculation of costs, incremental analysis, discounting and time horizon. However, more attention needs to be paid to the description of limitations of indirect comparisons, the choice of an appropriate analytic technique, the expression of unit costs in values for the current year, the estimation and valuation of outcomes, the presentation of results of sensitivity analyses, and testing the face validity of model inputs and outputs. Also, a large variation was observed in the scope and depth of the quality assessment by the Drug Reimbursement Committee. ConclusionsAlthough general guidelines exist, pharmaceutical companies and the Drug Reimbursement Committee would benefit from the existence of a more detailed checklist of methodological items that need to be reported in an economic evaluation.

Cost Effectiveness of Medication Adherence-Enhancing Interventions: A Systematic Review of Trial-Based Economic Evaluations

In light of the pressure to reduce unnecessary healthcare expenditure in the current economic climate, a systematic review that assesses evidence of cost effectiveness of adherence-enhancing interventions would be timely. Our objective was to examine the cost effectiveness of adherence-enhancing interventions compared with care as usual in randomised controlled trials, and to assess the methodological quality of economic evaluations. MEDLINE, PsycInfo, EconLit and the Centre for Reviews and Dissemination databases were searched for randomised controlled trials reporting full economic evaluations of adherence-enhancing interventions (published up to June 2013). Information was collected on study characteristics, cost effectiveness of treatment alternatives, and methodological quality. A total of 14 randomised controlled trials were included. The quality of economic evaluations and the risk of bias varied considerably between trials. Four studies showed incremental cost-effectiveness ratios (ICERs) below the willingness-to-pay threshold. Few studies seemed to evaluate interventions that successfully changed adherence. Only 14 randomised controlled trials examined the cost effectiveness of adherence interventions. Despite that some studies showe favourable ICERs, the overall quality of studies was modest and the economic perspectives applied were frequently narrow. To demonstrate that adherence interventions can be cost effective, we recommend that proven-effective adherence programmes are subjected to comprehensive economic evaluations.

PHP129 - Assessment of Methodological Quality of Economic Evaluations in Belgian Drug Reimbursement Applications

Objectives This paper aims to assess the methodological quality of economic evaluations included in Belgian reimbursement applications for Class 1 drugs. Materials and Methods For 19 reimbursement applications submitted during 2011 and Spring 2012, a descriptive analysis assessed the methodological quality of the economic evaluation, evaluated the assessment of that economic evaluation by the Drug Reimbursement Committee and the response to that assessment by the company. Compliance with methodological guidelines issued by the Belgian Healthcare Knowledge Centre was assessed using a detailed checklist of 23 methodological items. The rate of compliance was calculated based on the number of economic evaluations for which the item was applicable. Results Economic evaluations tended to comply with guidelines regarding perspective, target population, subgroup analyses, comparator, use of comparative clinical data and final outcome measures, calculation of costs, incremental analysis, discounting and time horizon. However, more attention needs to be paid to the description of limitations of indirect comparisons, the choice of an appropriate analytic technique, the expression of unit costs in values for the current year, the estimation and valuation of outcomes, the presentation of results of sensitivity analyses, and testing the face validity of model inputs and outputs. Also, a large variation was observed in the scope and depth of the quality assessment by the Drug Reimbursement Committee. Conclusions Although general guidelines exist, pharmaceutical companies and the Drug Reimbursement Committee would benefit from the existence of a more detailed checklist of methodological items that need to be reported in an economic evaluation.

A review of economic evaluations of behavior change interventions: setting an agenda for research methods and practice

The objective of this study was to review methodological quality of economic evaluations of lifestyle behavior change interventions (LBCIs) and to examine how they address methodological challenges for public health economic evaluation identified in the literature. Pubmed and the NHS economic evaluation database were searched for published studies in six key areas for behavior change: smoking, physical activity, dietary behavior, (illegal) drug use, alcohol use and sexual behavior. From included studies (n = 142), we extracted data on general study characteristics, characteristics of the LBCIs, methodological quality and handling of methodological challenges. Economic evaluation evidence for LBCIs showed a number of weaknesses: methods, study design and characteristics of evaluated interventions were not well reported; methodological quality showed several shortcomings and progress with addressing methodological challenges remained limited. Based on the findings of this review we propose an agenda for improving future evidence to support decision-making. Recommendations for practice include improving reporting of essential study details and increasing adherence with good practice standards. Recommendations for research methods focus on mapping out complex causal pathways for modeling, developing measures to capture broader domains of wellbeing and community outcomes, testing methods for considering equity, identifying relevant non-health sector costs and advancing methods for evidence synthesis.

Evaluación de la calidad metodológica de la literatura en evaluación económica en salud en Colombia: una revisión sistemática

Economic evaluation is a tool that can provide useful information to decision-makers in health. The methodological quality of Colombian studies has not been assessed in a systematic way. To assess the methodological quality of economic evaluations in health care in Colombia. Systematic review of economic studies that assessed both costs and effectiveness of at least two technologies related to a decision problem in Colombia. Search was performed in international and Colombian databases and was completed with manual searches and contact with authors. Data was synthesized in tables that included relevant information about the studies. Methodological quality was evaluated using a predefined instrument. Searches were performed between January and February. 48 studies were included in the review. Perspective of the study, incremental analyzes and description of alternatives were usually well specified. However, more than half of the articles did not state clearly the time horizon or discount rate and most studies did not address equity and implementation issues. Management of uncertainty was also problematic. Economic evaluation in health care in Colombia has grown considerably in recent years. However, methods vary considerably between studies and therefore their usefulness for decisionmaking in health is limited. It is necessary to standardize methods in order to generate evidence of higher quality to support decisions within the Colombian health care system.

Specific guidelines for assessing and improving the methodological quality of economic evaluations of newborn screening.

BackgroundEconomic evaluation of newborn screening poses specific methodological challenges. Amongst others, these challenges refer to the use of quality adjusted life years (QALYs) in newborns, and which costs and outcomes need to be considered in a full evaluation of newborn screening programmes. Because of the increasing scale and scope of such programmes, a better understanding of the methods of high-quality economic evaluations may be crucial for both producers/authors and consumers/reviewers of newborn screening-related economic evaluations. The aim of this study was therefore to develop specific guidelines designed to assess and improve the methodological quality of economic evaluations in newborn screening.MethodsTo develop the guidelines, existing guidelines for assessing the quality of economic evaluations were identified through a literature search, and were reviewed and consolidated using a deductive iterative approach. In a subsequent test phase, these guidelines were applied to various economic evaluations which acted as case studies.ResultsThe guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening are organized into 11 categories: “bibliographic details”, “study question and design”, “modelling”, “health outcomes”, “costs”, “discounting”, “presentation of results”, “sensitivity analyses”, “discussion”, “conclusions”, and “commentary”.ConclusionsThe application of the guidelines highlights important issues regarding newborn screening-related economic evaluations, and underscores the need for such issues to be afforded greater consideration in future economic evaluations. The variety in methodological quality detected by this study reveals the need for specific guidelines on the appropriate methods for conducting sound economic evaluations in newborn screening.

Methodological Quality of Economic Evaluations of New Pharmaceuticals in the Netherlands

In the Netherlands, decisions about the reimbursement of new pharmaceuticals are based on cost effectiveness, as well as therapeutic value and budget impact. Since 1 January 2005, drug manufacturers are formally required to substantiate the cost effectiveness of drugs that have therapeutic added value in comparison with existing ones through pharmacoeconomic evaluations. Dutch guidelines for pharmacoeconomic research provide methods guidance, ensuring consistency in both the evidence and the decision-making process about drug reimbursement. This study reviewed the methodological quality of all 21 formally required pharmacoeconomic evaluations of new pharmaceuticals between 1 January 2005 and 1 October 2008, and verified whether these evaluations complied with pharmacoeconomic guidelines. Data on the quality of the pharmacoeconomic evaluations were extracted from the pharmacoeconomic reports published by the Dutch Health Care Insurance Board (CVZ). The Board's newsletters provided information on the advice to, and reimbursement decisions made by, the Dutch Minister of Health. All data extraction was carried out by two independent reviewers, and descriptive analyses were conducted. The methodological quality was sound in only 8 of the 21 pharmacoeconomic evaluations. In most cases, the perspective of analysis, the comparator drugs, and the reporting of both total and incremental costs and effects were correct. However, drug indication, form (i.e. cost utility/cost effectiveness) and time horizon of the evaluations were frequently flawed. Moreover, the costs and effects of the pharmaceuticals were not always analysed correctly, and modelling studies were often non-transparent. Twelve drugs were reimbursed, and nine were not. The compliance with pharmacoeconomic guidelines in economic evaluations of new pharmaceuticals can be improved. This would improve the methodological quality of the pharmacoeconomic evaluations and ensure consistency in the evidence and the decision-making process for drug reimbursement in the Netherlands.

Economic evaluations of occupational health interventions from a corporate perspective – a systematic review of methodological quality

Using a standardized quality criteria list, we appraised the methodological quality of economic evaluations of occupational safety and health (OSH) interventions conducted from a corporate perspective. The primary literature search was conducted in Medline and Embase. Supplemental searches were conducted in the Cochrane NHS Economic Evaluation Database, the National Institute for Occupational Safety and Health (NIOSH) database, the Ryerson International Labour, Occupational Safety and Health Index, scans of reference lists, and researchers' own literature database. Independently, two researchers selected articles based on title, keywords, and abstract, and if needed, fulltext. Disagreements were resolved by a consensus procedure. Articles were selected based on seven criteria addressing study population, type of intervention, comparative intervention, outcome, costs, language, and perspective. Two reviewers independently judged methodological quality using the Consensus on Health Economic Criteria (CHEC-list), a 19-item standardized quality criteria list. Disagreements in judgment were also resolved by consensus. Data were analyzed descriptively. A total of 34 studies were included. Of these, only 44% of the studies met more than 50% of the quality criteria. Of the 19 quality criteria, 8 were met by 50% or more of the studies. The 11 least fulfilled criteria related to (i) performance of a sensitivity analysis, (ii) selection of perspective, (iii) description of study population, (iv) discussion of generalizability, (v) description of competing alternatives, (vi) presentation of the research question, (vii) measurement of outcomes, (viii) measurement of costs, (ix) valuation of costs, (x) declaration of researchers' independence, and (xi) discussion of ethical and distributional issues. Apart from a few exceptions, the overall methodological quality of the economic evaluations of OSH interventions from a corporate perspective was poor. As such, there is a risk of biased results. The quality of future evaluations needs to be improved to increase the validity of their conclusions and recommendations.

A Systematic Review of Occupational Health and Safety Interventions With Economic Analyses

We reviewed the occupational health and safety intervention literature to synthesize evidence on financial merits of such interventions. A literature search included journal databases, existing systematic reviews, and studies identified by content experts. Studies meeting inclusion criteria were assessed for quality. Evidence was synthesized within industry-intervention type clusters. We found strong evidence that ergonomic and other musculoskeletal injury prevention interventions in manufacturing and warehousing are worth undertaking in terms of their financial merits. We also found strong evidence that multisector disability management interventions are worth undertaking. While the economic evaluation of interventions in this literature warrants further expansion, we found a sufficient number of studies to identify strong, moderate, and limited evidence in certain industry-intervention clusters. The review also provided insights into how the methodological quality of economic evaluations in this literature could be improved.

The Methodological Quality of Economic Evaluations of Guideline Implementation into Clinical Practice: A Systematic Review of Empiric Studies

Objective Despite the emphasis on efficiency of health-care services delivery, there is an imperfect evidence base to inform decisions about whether and how to develop and implement guidelines into clinical practice. In general, studies evaluating the economics of guideline implementation lack methodological rigor. We conducted a systematic review of empiric studies to assess advances in the economic evaluations of guideline implementation. Methods The Cochrane Effective Professional and Organisational Change Group specialized register and the MEDLINE database were searched for English publications between January 1998 and July 2004 that reported objective effect measures and implementation costs. We extracted data on study characteristics, quality of study design, and economic methodology. It was assessed whether the economic evaluations followed methodological guidance. Results We included 24 economic evaluations, involving 21 controlled trials and three interrupted time series designs. The studies involved varying settings, targeted professionals, targeted behaviors, clinical guidelines, and implementation strategies. Overall, it was difficult to determine the quality of study designs owing to poor reporting. In addition, most economic evaluations were methodologically flawed: studies did not follow guidelines for evaluation design, data collection, and data analysis. Conclusion The increasing importance of the value for money of providing health care seems to be reflected by an increase in empiric economic evaluations of guideline implementation. Because of the heterogeneity and poor methodological quality of these studies, however, the resulting evidence is still of limited use in decision-making. There seems to be a need for more methodological guidance, especially in terms of data collection and data synthesis, to appropriately evaluate the economics of developing and implementing guidelines into clinical practice.

Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria

The aim of the Consensus on Health Economic Criteria (CHEC) project is to develop a criteria list for assessment of the methodological quality of economic evaluations in systematic reviews. The criteria list resulting from this CHEC project should be regarded as a minimum standard. The criteria list has been developed using a Delphi method. Three Delphi rounds were needed to reach consensus. Twenty-three international experts participated in the Delphi panel. The Delphi panel achieved consensus over a generic core set of items for the quality assessment of economic evaluations. Each item of the CHEC-list was formulated as a question that can be answered by yes or no. To standardize the interpretation of the list and facilitate its use, the project team also provided an operationalization of the criteria list items. There was consensus among a group of international experts regarding a core set of items that can be used to assess the quality of economic evaluations in systematic reviews. Using this checklist will make future systematic reviews of economic evaluations more transparent, informative, and comparable. Consequently, researchers and policy-makers might use these systematic reviews more easily. The CHEC-list can be downloaded freely from http://www.beoz.unimaas.nl/chec/.