Methodological Quality Of Guidelines Research Articles

Clinical practice guidelines on newborn hearing screening: A systematic quality appraisal using the AGREE II instrument

IntroductionSeveral guidelines and consensus statements have been produced and disseminated for the detection and management of newborn hearing loss. However, to date, the quality and methodologic rigor of these screening and management protocols have not been appraised. ObjectiveTo identify and evaluate existing guidelines and consensus statements for the detection and management of neonatal hearing loss. MethodsA comprehensive search of EMBASE, MEDLINE/PubMed, SCOPUS and grey literature sources was conducted until August 2020. The quality of these guidelines was assessed by four independent reviewers using the Appraisal of Guidelines for Research and Evaluation, 2nd edition (AGREE II). Domain scores were considered satisfactory quality if they scored >60%, and intraclass correlation coefficients (ICC) were calculated to assess agreement among the appraisers. ResultsTwelve guidelines were assessed for critical evaluation. Only two guidelines were classified as ‘high quality’, and the remaining were ‘average’ or ‘low quality’. The ‘Scope and Purpose’ domain achieved the highest mean score (91.3% ± 5.8%), and lowest was ‘Rigor of Development’ (35.8% ± 19.1%). ICC analysis showed good to very good agreement across all domains (0.63–0.95). ConclusionThese findings highlight the variability in methodologic quality of guidelines and consensus statement for the detection and management of neonatal hearing loss. These results may help to improve the reporting of future guidelines and guide the selection and use of these guidelines in clinical practice.

Appraisal of the current guidelines for management of cholangiocarcinoma-using the Appraisal of Guidelines Research and Evaluation II (AGREE II) Instrument.

Cholangiocarcinoma (CC) is the second most common primary liver tumour. High-quality guidelines are essential for effective patient stratification and individualised treatment. This study aimed to appraise the methodological quality of existing guidelines for the resection of CC using the Appraisal of Guidelines for Research & Evaluation (AGREE II) instrument. A systematic search of the literature in Cochrane, PubMed, Google Scholar, and Embase was performed. Assessment of the clinical practice guidelines (CPGs) and consensuses was performed using the AGREE II instrument by four clinicians experienced in surgical practice and the AGREE II appraisal method. Literature searches identified 13 guidelines of highly variable quality according to the AGREE II criteria. The guidelines scored well in certain domains such as scope & purpose (median score across all guidelines; 65%), clarity of presentation (76%), and editorial independence (56%). However, they scored poorly for applicability (13%), rigour of development (30%), and stakeholder involvement (39%). None of the 13 guidelines was recommended universally for use without modification. Overall, the methodological quality of guidelines on the surgical management of CC is poor. Future updates should address and modify shortcomings detected by the AGREE II instrument, thereby facilitating better patient stratification and individualised treatment strategies.

A technical appraisal of guidelines for the management of skin rash in patients on chemotherapy and targeted therapy

BackgroundSkin rash remains one of the most prevalent and troublesome clinical problems experienced by patients on chemotherapy and targeted therapy. To ensure high-quality care, guidelines are seen as the best guidance. Considering the quality of guidelines varies greatly, a systematical appraisal of the methodological quality of guidelines for the management of skin rash in patients on chemotherapeutic drugs and targeted anticancer therapies was undertaken, in order to identify appropriate ones for healthcare professionals.MethodsA systematic search of databases and Internet was conducted to obtain pertinent guidelines. Two reviewers independently assessed the eligibility of guidelines according to the inclusion criteria. Then the guidelines included were appraised by three researchers with the methodological quality of eligible guideline using Appraisal of Guidelines for Research and Evaluation II (AGREEII).ResultsTotally nineteen guidelines met the inclusion criteria. The quality ranged from good to acceptable in scope and purpose (mean: 78.80%, range: 66.67–94.44%) and clarity of presentation domains (mean: 85.38%, 75.00–91.67%), but not in stakeholder involvement (mean: 50.15%, range: 36.11–75.00%), rigor of development (mean: 23.65%, range: 6.25–70.83%), applicability (mean: 23.96%, range: 4.17–52.08%), and editorial independence domains (mean: 45.18%, range: 0.00–87.50%). Overall, two guidelines were classified as “recommended”.ConclusionsOnly two guidelines were recommended to manage skin rash in patients on chemotherapy and targeted therapies, most guidelines issued were of low to moderate quality. Thus, more attention should be paid to the methodological quality of guideline development in this field.

Improving the adaptability of WHO evidence-informed guidelines for nutrition actions: results of a mixed methods evaluation

BackgroundOver the past decade, the World Health Organization (WHO) has implemented a standardized, evidence-informed guideline development process to assure technically sound and policy-relevant guidelines. This study is an independent evaluation of the adaptability of the guidelines produced by the Evidence and Programme Guidance unit, at the Department of Nutrition for Health and Development (NHD). The study systematizes the lessons learned by the NHD group at WHO.MethodsWe used a mixed methods approach to determine the adaptability of the nutrition guidelines. Adaptability was defined as having two components; methodological quality and implementability of guidelines. Additionally, we gathered recommendations to improve future guideline development in nutrition actions for health and development. Data sources for this evaluation were official documentation and feedback (both qualitative and quantitative) from key stakeholders involved in the development of nutrition guidelines. The qualitative data was collected through a desk review and two waves of semi-structured interviews (n = 12) and was analyzed through axial coding. Guideline adaptability was assessed quantitatively using two standardized instruments completed by key stakeholders. The Appraisal Guideline for Research and Evaluation questionnaire, version II was used to assess guideline quality (n = 6), while implementability was assessed with the electronic version of the GuideLine Implementability Appraisal (n = 7).ResultsThe nutrition evidence-informed guideline development process has several strengths, among them are the appropriate management of conflicts of interest of guideline developers and the systematic use of high-quality evidence to inform the recommendations. These features contribute to increase the methodological quality of the guidelines. The key areas for improvement are the limited implementability of the recommendations, the lack of explicit and precise implementation advice in the guidelines and challenges related to collaborative work within interdisciplinary groups.ConclusionsOverall, our study found that the nutrition evidence-informed guidelines are of good methodological quality but that the implementability requires improvement. The recommendations to improve guideline adaptability address the guideline content, the dynamics shaping interdisciplinary work, and actions for implementation feasibility. As WHO relies heavily on a standardized procedure to develop guidelines, the lessons learned may be applicable to guideline development across the organization and to other groups developing guidelines.

The Scientific Basis of Guideline Recommendations on Sugar Intake: A Systematic Review.

The relationship between sugar and health is affected by energy balance, macronutrient substitutions, and diet and lifestyle patterns. Several authoritative organizations have issued public health guidelines addressing dietary sugars. To systematically review guidelines on sugar intake and assess consistency of recommendations, methodological quality of guidelines, and the quality of evidence supporting each recommendation. MEDLINE, EMBASE, and Web of Science (1995 to September 2016); guideline registries; and gray literature (bibliographies, Google, and experts). Guidelines addressing sugar intake that reported their methods of development and were published in English between 1995 and 2016. Three reviewers independently assessed guideline quality using the Appraisal of Guidelines for Research and Evaluation, 2nd edition (AGREE II), instrument. To assess evidence quality, articles supporting recommendations were independently reviewed and their quality was determined by using GRADE (Grading of Recommendations Assessment, Development and Evaluation) methods. The search identified 9 guidelines that offered 12 recommendations. Each of the reviewed guidelines indicated a suggested decrease in the consumption of foods containing nonintrinsic sugars. The guidelines scored poorly on AGREE II criteria, specifically in rigor of development, applicability, and editorial independence. Seven recommendations provided nonquantitative guidance; 5 recommended less than 25% to less than 5% of total calories from nonintrinsic sugars. The recommendations were based on various health concerns, including nutrient displacement, dental caries, and weight gain. Quality of evidence supporting recommendations was low to very low. The authors conducted the study independent of the funding source, which is primarily supported by the food and agriculture industry. Guidelines on dietary sugar do not meet criteria for trustworthy recommendations and are based on low-quality evidence. Public health officials (when promulgating these recommendations) and their public audience (when considering dietary behavior) should be aware of these limitations. Technical Committee on Dietary Carbohydrates of the North American branch of the International Life Sciences Institute. (PROSPERO: CRD42015029182).

Quality appraisal of clinical practice guidelines on glioma

Clinical practice guidelines (CPGs) play an important role in healthcare. The guideline development process should be precise and rigorous to ensure that the results are reproducible and not vague. To determine the quality of guidelines, the Appraisal of Guidelines and Research and Evaluation (AGREE) instrument was developed and introduced. The aim of the present study was to assess the methodological quality of clinical practice guidelines on glioma. Eight databases (including MEDLINE and Embase) were searched till to August, 2013. The methodological quality of the guidelines was assessed by four authors independently using the AGREE II instrument. Fifteen relevant guidelines were included from 940 citations. The overall agreement among reviewers was moderate (intra-class correlation coefficient = 0.83; 95% confidence interval [CI], 0.66-0.92). The mean scores were moderate for the domains "scope and purpose" (59.54) and "clarity of presentation" (65.46); however, there were low scores for the domains "stakeholder involvement" (43.80), "rigor of development" (39.01), "applicability" (31.89), and "editorial independence" (30.83). Only one third of the guidelines described the systematic methods for searching, and nearly half of the (47%) guidelines did not give a specific recommendation. Only four of 15 described a procedure for updating the guideline; meanwhile, just six guidelines in this field can be considered to be evidence-based. The quality and transparency of the development process and the consistency in the reporting of glioma guidelines need to be improved. And the quality of reporting of guidelines was disappointing. Many other methodological disadvantages were identified. In the future, glioma CPGs should be based on the best available evidence and rigorously developed and reported. Greater efforts are needed to provide high-quality guidelines that serve as a useful and reliable tool for clinical decision-making in this field.

Methodological quality of guidelines in gastroenterology.

Clinical guidelines are a common feature in modern endoscopy practice and they are being produced faster than ever. However, their methodological quality is rarely assessed. This study evaluated the methodological quality of current clinical guidelines in the field of gastroenterology, with an emphasis on endoscopy. Practice guidelines published by the American College of Gastroenterology (ACG), American Gastroenterological Association (AGA), American Society for Gastrointestinal Endoscopy (ASGE), European Society of Gastrointestinal Endoscopy (ESGE), British Society of Gastroenterology (BSG), National Institute for Health and Care Excellence (NICE), and the Scottish Intercollegiate Guidelines Network (SIGN) were searched between September and October 2012 and evaluated using the AGREE II (Appraisal of Guidelines for Research and Evaluation) instrument (23 items, scores 1 - 7 for each item; higher scores mean better quality). A total of 100 guidelines were assessed. The mean number of items scoring 6 or 7 per guideline was 9.2 (out of 23 items). Overall, 99 % of guidelines failed to include the target population in the development process, and 96 % did not report facilitators and barriers to guideline application. In addition, 86 % did not include advice or tools, and 94 % did not present monitoring or auditing criteria. The global methodological quality of clinical guidelines in the field of gastroenterology is poor, particularly regarding involvement of the target population in the development of guidelines and in the provision of clear suggestions to practitioners.

Quality assessment of guidelines on thromboprophylaxis in orthopaedic surgery

We evaluated the quality of guidelines on thromboprophylaxis in orthopaedic surgery by examining how they adhere to validated methodological standards in their development. A structured review was performed for guidelines that were published between January 2005 and April 2013 in medical journals or on the Internet. A pre-defined computerised search was used in MEDLINE, Scopus and Google to identify the guidelines. The AGREE II assessment tool was used to evaluate the quality of the guidelines in the study. Seven international and national guidelines were identified. The overall methodological quality of the individual guidelines was good. 'Scope and Purpose' (median score 98% interquartile range (IQR)) 86% to 98%) and 'Clarity of Presentation' (median score 90%, IQR 90% to 95%) were the two domains that received the highest scores. 'Applicability' (median score 68%, IQR 45% to 75%) and 'Editorial Independence' (median score 71%, IQR 68% to 75%) had the lowest scores. These findings reveal that although the overall methodological quality of guidelines on thromboprophylaxis in orthopaedic surgery is good, domains within their development, such as 'Applicability' and 'Editorial Independence', need to be improved. Application of the AGREE II instrument by the authors of guidelines may improve the quality of future guidelines and provide increased focus on aspects of methodology used in their development that are not robust.

Methodological quality of English-language genetic guidelines on hereditary breast-cancer screening and management: an evaluation using the AGREE instrument

BackgroundWe examined the methodological quality of guidelines on syndromes conferring genetic susceptibility to breast cancer.MethodsPubMed, EMBASE, and Google were searched for guidelines published up to October 2010. All guidelines in English were included. The Appraisal of Guidelines, Research and Evaluation (AGREE) instrument was used to assess the quality of the guidelines, and their reported evidence base was evaluated.ResultsThirteen guidelines were deemed eligible: seven had been developed by independent associations, and the other six had national/state endorsements. Four guidelines performed satisfactorily, achieving a score of greater than 50% in all six AGREE domains. Mean ± SD standardized scores for the six AGREE domains were: 90 ± 9% for 'scope and purpose', 51 ± 18% for 'stakeholder involvement', 55 ± 27% for 'rigour of development', 80 ± 11% for 'clarity and presentation', 37 ± 32% for 'applicability', and 47 ± 38% for 'editorial independence'. Ten of the thirteen guidelines were found to be based on research evidence.ConclusionsGiven the ethical implications and the high costs of genetic testing for hereditary breast cancer, guidelines on this topic should provide clear and evidence-based recommendations. Our analysis shows that there is scope for improving many aspects of the methodological quality of current guidelines. The AGREE instrument is a useful tool, and could be used profitably by guidelines developers to improve the quality of recommendations.

Quality of Low Back Pain Guidelines Improved

Systematic review of clinical guidelines. To assess the methodological quality of clinical guidelines for the management of acute and chronic low back pain (LBP) in primary care and compare their recommendations. A guideline evaluation performed in 2004 concluded that the quality and transparency of the development process and consistency in the reporting of primary care guidelines for LBP need to be improved. At present, several guidelines have been revised and new guidelines are published. We evaluated if the quality of guidelines has improved. Guidelines published since 2004 were selected by electronically searching in MEDLINE, Cochrane Back Review Group database, Guideline Clearing House, Google, and contacting experts. The methodological quality of the guidelines was assessed by 2 authors independently, using the Appraisal of Guidelines, Research, and Evaluation in Europe instrument. Also, the diagnostic and therapeutic recommendations were compared. Fourteen guidelines were included. In general, the quality was satisfactory. The guidelines had best scores on clarity and presentation. The domain scores of scope and purpose were often moderate due to the absence of description of the clinical questions. The domain of stakeholder involvement scored moderate, mostly because guidelines were not tested among target users. Domains that had generally low scores were applicability and editorial independence. Four guidelines scored low on the rigor of development, but the other guidelines scored high on this domain.The diagnostic and therapeutic recommendations in the guidelines for acute LBP were mainly comparable while the recommendations for the management of chronic LBP varied widely. Compared to the quality assessment performed in 2004, the average quality of guidelines has improved. However, guideline developers should still improve the quality transparency of the development process. Especially the applicability of guidelines and the editorial independence need to be ensured in future guidelines.

Do Guidelines for the Diagnosis and Monitoring of Diabetes Mellitus Fulfill the Criteria of Evidence-Based Guideline Development?

Although the methodological quality of therapeutic guidelines (GLs) has been criticized, little is known regarding the quality of GLs that make diagnostic recommendations. Therefore, we assessed the methodological quality of GLs providing diagnostic recommendations for managing diabetes mellitus (DM) and explored several reasons for differences in quality across these GLs. After systematic searches of published and electronic resources dated between 1999 and 2007, 26 DM GLs, published in English, were selected and scored for methodological quality using the AGREE Instrument. Subgroup analyses were performed based on the source, scope, length, origin, and date and type of publication of GLs. Using a checklist, we collected laboratory-specific items within GLs thought to be important for interpretation of test results. The 26 diagnostic GLs had significant shortcomings in methodological quality according to the AGREE criteria. GLs from agencies that had clear procedures for GL development, were longer than 50 pages, or were published in electronic databases were of higher quality. Diagnostic GLs contained more preanalytical or analytical information than combined (i.e., diagnostic and therapeutic) recommendations, but the overall quality was not significantly different. The quality of GLs did not show much improvement over the time period investigated. The methodological shortcomings of diagnostic GLs in DM raise questions regarding the validity of recommendations in these documents that may affect their implementation in practice. Our results suggest the need for standardization of GL terminology and for higher-quality, systematically developed recommendations based on explicit guideline development and reporting standards in laboratory medicine.

Schizophrenia practice guidelines: International survey and comparison

Schizophrenia guidelines differ considerably in methodology and content. To systematically compare national schizophrenia guidelines from different countries. An international survey was conducted on guideline development and a methodological comparison was made using a validated guideline appraisal instrument (the Appraisal Guideline Research and Evaluation Europe). The methodological quality of many schizophrenia guidelines was at best moderate. Few guidelines had included key stakeholders in their development process. Although pharmacotherapy recommendations were similar, there were strong variations in the type of psychosocial interventions recommended. The methodological quality of guidelines has a strong influence on their applicability. However, the lack of financial means to develop and implement guidelines is a serious problem. Independent international organisations could contribute to defining a core set of unbiased schizophrenia treatment recommendations. In countries with a shortage of resources, this could be a basis for adaptation to different cultural and economic backgrounds in collaboration with stakeholders.

Quality of Guidelines for the Laboratory Management of Diabetes Mellitus

BACKGROUND: There is increasing concern about the quality and reliability of practice guidelines, especially in the field of laboratory medicine, as most recommendations are developed by clinical specialty societies, often without involving laboratory professionals. Little information is available on the methodological quality of guidelines for the use of laboratory investigations in the care of specific diseases. We describe a pilot assessment of the most well‐known guidelines for the diagnosis and monitoring of diabetes mellitus (DM). METHODS: Practice guidelines on DM published in English between 1999 and 2005 April were identified by systematic searching in Medline and international guideline databases. Fifty four DM guidelines were retrieved, of which 29 met our inclusion criteria. The four most widely used international guidelines (WHO, ADA, NACB, NICE) were selected for a critical appraisal of their methodological quality. This was carried out by seven independent assessors using a validated checklist, the AGREE Instrument. Twenty three guideline attributes arranged in six independent domains were investigated and the mean scores of assessors for each attribute and the aggregated scores for each domain were calculated. Cronbach's alpha and interclass correlations were calculated to measure internal consistency and reliability within each domain. The four guidelines were compared using one‐way ANOVA and ANOVA using repeated measurements. RESULTS: The selected four guidelines on DM have significant shortcomings in demonstrating and/or reporting multidisciplinary stakeholder involvement in the guideline development process, evidence‐based methodology for formulating recommendations, applicability of statements, and disclosing any conflicts of interest or reporting editorial independence. CONCLUSIONS: Poor quality and lack of explicitness of recommendations in laboratory medicine call for methodological standards of guideline development and reporting, and for an international collaboration of guideline development activities, to increase the internal and external validity of recommendations in laboratory practice.

Analysis and evaluation of current guidelines for diagnosis and treatment of depressive disorders:

Depressive disorders are of high clinical and economic relevance in outpatient and inpatient care. For the treatment of depression effective measures are available. However, these methods often are not used because depressive disorders are underdetected or underestimated in their severity. To improve diagnostic and therapeutic processes in the last years different national and international organizations developed guidelines for diagnosis and treatment of depressive disorders (e. g. AHCPR, APA, German Association of Psychiatrists). These guidelines differ substantially in their goals, content, size, specifically in their methodological quality. The Association of the Scientific Medical Societies in Germany (AWMF) and the German Agency for Quality in Medicine (AQUMED) developed a checklist to evaluate the methodological quality of guidelines. This manual was used to describe and analyze the most relevant and actual six guidelines for diagnosis and treatment of depressive disorders. Recommendations for further improvement and implementation of these guidelines are given.