Delphi Scoring Research Articles

Choosing Wisely and Promoting High-Value Care and Staff Safety During the COVID-19 Pandemic in a Large Safety Net System.

As the COVID-19 pandemic brought surges of hospitalized patients, it was important to focus on reducing overuse of tests and procedures to not only reduce potential harm to patients but also reduce unnecessary exposure to staff. The objective of this study was to create a Choosing Wisely in COVID-19 list to guide clinicians in practicing high-value care at our health system. A Choosing Wisely in COVID-19 list was developed in October 2020 by an interdisciplinary High Value Care Council at New York City Health + Hospitals, the largest public health system in the United States. The first phase involved gathering areas of overuse from interdisciplinary staff across the system. The second phase used a modified Delphi scoring process asking participants to rate recommendations on a 5-point Likert scale based on criteria of degree of evidence, potential to prevent patient harm, and potential to prevent staff harm. The top 5 recommendations included avoiding tracheal intubation without trial of noninvasive ventilation (4.4); not placing routine central venous catheters (4.33); avoiding routine daily laboratory tests and batching laboratory draws (4.19); not ordering daily chest radiographs (4.17); and not using bronchodilators in the absence of reactive airway disease (4.13). We successfully developed Choosing Wisely in COVID-19 recommendations that focus on evidence and preventing patient and staff harm in a large safety net system to reduce overuse.

Choosing Wisely in Adult Hospital Medicine: Co-creation of New Recommendations for Improved Healthcare Value by Clinicians and Patient Advocates.

The American Board of Internal Medicine Foundation's Choosing Wisely campaign has resulted in a vast number of recommendations to reduce low-value care. Implementation of these recommendations, in conjunction with patient input, remains challenging. To create updated Society of Hospital Medicine Adult Hospitalist Choosing Wisely recommendations that incorporate patient input from inception. This was a multi-phase study conducted by the Society of Hospital Medicine's High Value Care Committee from July 2017 to January 2020 involving clinicians and patient advocates. Phase 1 involved gathering low-value care recommendations from patients and clinicians across the USA. Recommendations were reviewed by the committee in phase 2. Phase 3 involved a modified Delphi scoring in which 7 committee members and 7 patient advocates voted on recommendations based on strength of evidence, potential for patient harm, and relevance to either hospital medicine or patients. A patient-friendly script was developed to allow advocates to better understand the clinical recommendations. A total of 1265 recommendations were submitted by clinicians and patients. After accounting for similar suggestions, 283 recommendations were categorized. Recommendations with more than 10 mentions were advanced to phase 3, leaving 22 recommendations for the committee and patient advocates to vote upon. Utilizing a 1-5 Likert scale, the top combined recommendations were reducing use of opioids (4.57), improving sleep (4.52), minimizing overuse of oxygen (4.52), reducing CK-MB use (4.50), appropriate venous thromboembolism prophylaxis (4.43), and decreasing daily chest x-rays (4.43). Specific voting categories, along with the use of patient-friendly language, allowed for the successful co-creation of recommendations.

Identifying core competencies for practicing public health professionals: results from a Delphi exercise in Uttar Pradesh, India

BackgroundEnsuring the current public health workforce has appropriate competencies to fulfill essential public health functions is challenging in many low- and middle-income countries. The absence of an agreed set of core competencies to provide a basis for developing and assessing knowledge, skills, abilities, and attitudes contributes to this challenge. This study aims to identify the requisite core competencies for practicing health professionals in mid-level supervisory and program management roles to effectively perform their public health responsibilities in the resource-poor setting of Uttar Pradesh (UP), India.MethodsWe used a multi-step, interactive Delphi technique to develop an agreed set of public health competencies. A narrative review of core competency frameworks and key informant interviews with human resources for health experts in India were conducted to prepare an initial list of 40 competency statements in eight domains. We then organized a day-long workshop with 22 Indian public health experts and government officials, who added to and modified the initial list. A revised list of 54 competency statements was rated on a 5-point Likert scale. Aggregate statement scores were shared with the participants, who discussed the findings. Finally, the revised list was returned to participants for an additional round of ratings. The Wilcoxon matched-pairs signed-rank test was used to identify stability between steps, and consensus was defined using the percent agreement criterion.ResultsStability between the first and second Delphi scoring steps was reached in 46 of the 54 statements. By the end of the second Delphi scoring step, consensus was reached on 48 competency statements across eight domains: public health sciences, assessment and analysis, policy and program management, financial management and budgeting, partnerships and collaboration, social and cultural determinants, communication, and leadership.ConclusionsThis study produced a consensus set of core competencies and domains in public health that can be used to assess competencies of public health professionals and revise or develop new training programs to address desired competencies. Findings can also be used to support workforce development by informing competency-based job descriptions for recruitment and performance management in the Indian context, and potentially can be adapted for use in resource-poor settings globally.

전문가 델파이 및 토지이용점수화 접근법을 활용한 지역 산림생태계서비스 공급량 지도화 연구

전문가 델파이 및 토지이용점수화 접근법을 활용한 지역 산림생태계서비스 공급량 지도화 연구

Risk factors of preoperative Hirschsprung-associated enterocolitis

BackgroundHirschsprung-associated enterocolitis (HAEC) is a life-threatening complication of Hirschsprung disease (HSCR), that might occur preoperatively. We investigated the risk factors of preoperative HAEC.MethodWe retrospectively reviewed all medical records of HSCR patients admitted at Dr. Sardjito Hospital, Indonesia from March 2012 until March 2015. Diagnosis of HAEC was determined using the Delphi scoring system.ResultsSixty-one HSCR patients were involved in this study, of whom 48 were males and 13 females. Eighteen percent (11/61) patients had a preoperative HAEC. The most common findings of the HAEC score found in our patients were distended abdomen (100%) and dilated loops of bowel (100%), followed by lethargy (72.7%), cut-off sign in rectosigmoid with absence of distal air (72.7%), leukocytosis (72.7%), and shift to left (63.6%). There was no association between gender, age of HSCR diagnosis, early/late diagnosis during neonatal period, aganglionosis type, albumin level nor body mass index with preoperative HAEC (p = 0.69, 0.76, 0.33, 1.0, 0.86, and 0.50, respectively). In addition, the maternal age, gestational age, and maternal education level also did not correlate with the development of preoperative HAEC (p = 0.71, 0.59, and 0.32, respectively).ConclusionThe incidence of preoperative HAEC in our hospital is considered relatively moderate, with the most common findings of distended abdomen and dilated loops of bowel. None of the identified risk factors have an association with the development of HAEC in our patients.

Development of an item bank to measure factual disease and treatment related knowledge of rheumatoid arthritis patients in the treat to target era

ObjectiveTo develop a Disease and treatment associated Knowledge in RA item bank (DataK-RA) based on item response theory. MethodsInitial items were developed from a systematic review. Rheumatology professionals identified relevant content trough a RAND modified Delphi scoring procedure and consensus meeting. RA patients provided additional content trough a focus group. Patients and professionals rated readability, feasibility and comprehensiveness of resulting items. Cross-sectional data were collected to evaluate psychometric properties of the items. ResultsData of 473 patients were used for item reduction and calibration. Twenty items were discarded based on corrected item-total point biserial correlation <0.30. Confirmatory factor analysis with weighted least squares estimation on the polychoric correlation matrix suggested good fit for a unidimensional model for the remaining 42 items (CFI 0.97 TLI=0.97, RMSEA=0.02, WRMR=0.97), supporting the proposed scoring procedure. Scores were highly reliable and normally distributed with minimal ceiling (1.8%) and no floor effects. 75% of tested hypotheses about the association of DataK-RA scores with related constructs were supported, indicating good construct validity. ConclusionDataK-RA is a psychometrically sound item bank. Practice implicationsDataK-RA provides health professionals and researchers with a tool to identify and target patients’ information needs or to assess effects of educational efforts.

Building quality indicators for medication use in people aging with intellectual disabilities and behaviour disorders

Objectives: The quality of healthcare provided to people with intellectual disabilities, is under scrutiny. The aim of this project was to build quality indicators (QIs) that could be used to measure the quality of the medication use process for people with intellectual disabilities and behaviour disorders.Methods: Candidate Qls were developed using literature and guideline review and expert evaluation. The candidate Qls were submitted to a 32-member panel of multidisciplinary experts, from Ireland and the UK. The modified Delphi panel included psychiatrists, pharmacists, nursing personnel, speech and language therapists, geriatricians, psychologists and representation from academia, regulation and administration. Over 2 rounds the QIs were rated for importance, scientific soundness and feasibility on a 9 point likert scale. Consensus criteria were preset.Results: A total of 38 candidate QIs underwent Delphi scoring. QIs addressed patient experience, access to care, continuity of care, equity, patient safety, effectiveness, appropriateness and assessment. Following round one, 23 QIs were accepted, none were rejected and 15 were equivocal. A total of 30 candidate QIs were accepted following round 2. Selection criteria were raised to differentiate hierarchy of final list.Conclusions: This project was the first attempt to develop QIs for the medication use process in people with intellectual disabilities and behaviour disorders. They infer a judgement about the quality of care provided in the medication use process. They may become important definitions of quality care for people with intellectual disabilities and behaviour disorders and be useful for quality improvement in this vulnerable population.

Identifying Children’s Health Care Quality Measures for Medicaid and CHIP: An Evidence-Informed, Publicly Transparent Expert Process

ObjectiveTo describe the process used to identify the recommended core set of quality measures as mandated by the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA) and provide an overview of the measures selected. MethodsIn May 2009, the multidisciplinary Agency for Healthcare Research and Quality (AHRQ) National Advisory Council for Healthcare Research and Quality Subcommittee on Children’s Healthcare Quality Measures for Medicaid and CHIP Programs (SNAC) was formed. The SNAC established criteria to evaluate quality measures on the basis of their validity, feasibility, and importance. Subsequently, AHRQ implemented a measure nomination process. Nominators supplied key information related to measure validity, feasibility, and importance. Oxford Centre for Evidence-Based Medicine (CEBM) criteria were used to assess evidence supporting the validity/scientific soundness of nominated measures. SNAC members applied an adaptation of the RAND-UCLA modified Delphi process to all nominated measures. Measures passing the Delphi process were further assessed on the basis of criteria pertaining to legislative priorities. ResultsSeventy of 119 nominated measures met criteria for validity, feasibility, and importance according to Delphi scoring. After further prioritization, 25 measures were recommended for the initial core set. Twelve of the recommended measures focus on preventive care and health promotion including prenatal/perinatal care (4), well-child care (1), immunizations (2), screening for: developmental delays (1), obesity (1), and sexually transmitted infections (1), and receipt of preventive dental services (2). Five acute care measures were recommended which focus on management of upper respiratory illnesses (2), receipt of acute care dental services (1), emergency department utilization (1), and inpatient rates of central line associated bloodstream infections (1). Five of the recommended measures focus on chronic care, specifically asthma (1), attention-deficit/hyperactivity disorder (1), diabetes (1), and care for children with mental health conditions (2). Two of the measures focus on family experiences with care, and one of the measures assesses utilization of outpatient primary care services. Thirteen (52%) of the measures were derived from the National Committee for Quality Assurance (NCQA) Healthcare Effectiveness Data and Information Set (HEDIS). Eighteen of the measures were supported by relatively high levels of evidence (Oxford CEBM grade A or B). ConclusionsAn open national public process combined with an evidence-informed evaluation methodology resulted in identification of a balanced, grounded, and parsimonious core set of measures that should become feasible to implement on a widespread scale over time.

European Leukemianet (ELN) Project Diagnostic Platform (WP10): Final Results of the First Study of the European Morphology Consensus Panel.

Background: Microscopic peripheral blood and bone marrow cell evaluation remains a milestone in the diagnostic of hematology. Many factors contribute to the lack of standardization of this diagnostic test, such as differences in the procedure, staining, degree of skill in interpretation and glossary. However the new WHO classification highlights the importance of morphological aspects, quantitative as well qualitative, for a better stratification of patients in the diagnosis of haematological malignancies, particularly myeloid malignancies and above all myelodysplasia (MDS). In this high technology era, we have the opportunity to exchange, via the internet, images and information without geographic limitation, sparing time and resources. This represents a great opportunity to try to get a consensus for blood cell morphology. The European LeukemiaNet (ELN) Network of excellence is an EU project funded by the 6th FP, involving 174 centres from 28 countries. Its major goal is the construction of a cooperative network for improving leukemia diagnosis, care and research. Within the activities of the Diagnostic Platform (WP10), focused on Flow Cytometric and Morphological panels, a European Morphology Consensus Faculty (EMCF), composed of 21 expert morphologists from 13 European countries, has been organized with the following goals:to harmonize the identification of hematological cells for a common European morphological diagnostic pathway, in terms of cell lineage, maturation level, normality/abnormality and glossary,to take into account specific national skills, competences and methods,to provide the patients with the same morphological diagnosis all over Europe.The first step was to create a consensus-based cell library of meaningful blood cell images identified and named by top level European morphologists and agreed by EMCF, as a valuable tool of traininig. We are presenting the final results of the first study of the EMCF.Methods: From May to July 2007 we collected from all the EMCF members 164 images containing 438 labelled blood cells with the following distribution: Granulocytic series n=126 (28.5%), Erythroid series n=77 (17.5%), Monocytic series n=35 (8%), Lymphoid series n=107 (24.,5%), Megakaryocytic series n=23 (5.,5%), Blasts (not otherwise specified) n=29 (6.5%) Other n= 41 (9.5%). All the images were uploaded to a restricted web page together with an Excel file containing the author's proposal for each cell. Faculty Members were asked to fill the Excel file with their agreed or personal alternative definition for each labelled cell. On January 2008 cells lacking a full agreement were submitted to a Delphi scoring procedure: a minimum of 3 agreements was needed for a cell to be scored in the Delphi questionnaire. By July 2008 we have received the Delphi files for the final evaluation of the study results.Evaluation rules: Full consensus: cells with an agreement of at least 17 out of 21 (>80%) EMFC members. Delphi questionnaire for all the cells with an agreement <17. Moreover we decided to include in the Delphi scoring procedure n=37 cells with an agreement ≥ 17/21 associated with same alternative proposal from 3 or 4 Faculty members (14% out of all the cells with an agreement ≥ 17/21).Results: We reached a full agreement (consensus ≥17/21) on > 59% of all submitted cells. The main discrepancies in the morphological consensus concern the groups of blast and monocytic series, while for the other groups of cells the distribution is similar. After the Delphi scoring we reached an agreement on all the 216 submitted cells. The EMCF created a new category “Cell to delete” for 8 cells due to a not perfect quality of images. This image library is composed of 430 cells now identified through a consensus method by a European Morphology Consensus Faculty and the cells will be uploaded onto the ELN web site, with a clear identification of those cells identified with a Delphi scoring procedure. Moreover cell names are harmonized and this represents the first morphological European glossary.Future developments: The second extended Faculty including morphologists from 16 European countries has been formed with the aim to apply the consensus glossary to identify a new set of cells submitted by the authors without any suggestions. A meeting is planned to discuss the results.