Probing the power of Apollo : methodological challenges and opportunities of the Delphi methods for developing rankings

Quantifying the development of agreement among experts in Delphi studies

Defining and measuring urban sustainability in Europe: A Delphi study on identifying its most relevant components

The effects of feeding back experts’ own initial ratings in Delphi studies: A randomized trial

Fetal heart rate baseline computation with a weighted median filter

Knowledge capacity plays a vital role in building community resilience to disasters. However, the problem is that there is no resilience framework that integrates the knowledge creation process. This article introduces a new framework for increasing community resilience based on knowledge creation theory (KCT). This research aims to define the elements that support the Knowledge Creation for Community Resilience (KCCR) and to gain consensus from experts on these factors. This study was conducted using semi-structured interviews with five panellists and three rounds of Delphi technique to determine the assessment of 26 factors (including six additional factors) that have been identified by experts (30, 18 and 11 experts in rounds I, II and III, sequentially). The data analysis was carried out in several stages, and included Spearman’s Rank Correlation Coefficient, consensus appraisal and interrater agreement (IRA) statistical evaluation. The result of the agreement level (AL) analysis shows that the majority of the constructs (96.15%) are in the ‘moderate strong’ category. This study shows that there is a significant consensus (with IRA index [awg(1)] ranging from 0.529 to 1), and panellists confirm the significance of all the key constructs. Consensus was gained from experts on seven elements that support the KCCR. This study establishes a systematic, operational and multidimensional KCCR framework that combines the concepts of knowledge creation, community resilience and disaster preparedness. This framework can be used as a qualitative instrument or guidance to build community resilience based on knowledge creation and a quantitative tool for measuring community resilience in facing disasters.

The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment

BackgroundInteroperable electronic health record (EHR) solutions are currently being implemented in Canada, as in many other countries. Understanding EHR users’ perspectives is key to the success of EHR implementation projects. This Delphi study aimed to assess in the Canadian context the applicability, the importance, and the priority of pre-identified factors from a previous mixed-methods systematic review of international literature.MethodsA three-round Delphi study was held with representatives of 4 Canadian EHR user groups defined as partners of the implementation process who use or are expected to use EHR in their everyday activity. These groups are: non-physician healthcare professionals, health information professionals, managers, and physicians. Four bilingual online questionnaire versions were developed from factors identified by the systematic review. Participants were asked to rate the applicability and the importance of each factor. The main outcome measures were consensus and priority. Consensus was defined a priori as strong (≥ 75%) or moderate (≥ 60-74%) according to user groups’ level of agreement on applicability and importance, partial (≥ 60%) when participants agreed only on applicability or importance, or as no consensus (< 60%). Priority for decision-making was defined as factors with strong consensus with scores of 4 or 5 on a five-point Likert scale for applicability and importance.ResultsThree Delphi rounds were completed by 64 participants. Levels of consensus of 100%, 64%, 64%, and 44% were attained on factors submitted to non-physician healthcare professionals, health information professionals, managers, and physicians, respectively. While agreement between and within user groups varied, key factors were prioritized if they were classified as strong (≥ 75% from questionnaire answers of user groups), for decision-making concerning EHR implementation. The10 factors that were prioritized are perceived usefulness, productivity, motivation, participation of end-users in the implementation strategy, patient and health professional interaction, lack of time and workload, resources availability, management, outcome expectancy, and interoperability.ConclusionsAmongst all factors influencing EHR implementation identified in a previous systematic review, ten were prioritized through this Delphi study. The varying levels of agreement between and within user groups could mean that users’ perspectives of each factor are complex and that each user group has unique professional priorities and roles in the EHR implementation process. As more EHR implementations in Canada are completed it will be possible to corroborate this preliminary result with a larger population of EHR users.

Previous studies have identified eight parameters as being the most influential on the Icelandic economy in the light of prevailing climate change. This study aims to investigate these economic parameters in relation to the Arctic Region and the Icelandic economy. The study is rooted in the theoretical framework of decision analysis and is intended to contribute to strategic and tactical decision making in the context of Iceland and developments resulting from climate changes in the Arctic Region. The study results could eventually lead do a “What if” decision model, where multiple scenarios can be analyzed by inserting/entering different probabilities of a particular outcome. The parameters under screening/review are interdependent, and to construct a decision model these interdependencies must be assessed. The Delphi method was employed, along with the cross-impact analysis, to shed light on the interconnections between eight of these parameters. Diverse views are revealed concerning the impacts between parameters, although a consensus was reached in the second round of the survey. In some instances, the level of agreement was high, while other estimates showed a wide range of responses. The diversity in answers suggests a higher level of uncertainty regarding some parameters and events compared to others. The survey also indicates how challenging the Delphi and cross-impact methods are in terms of the involvement of experts. However, when data is scarce, these methods provide significant insight.

The world is rapidly urbanizing which 68% of its population is expected to live in urban areas by 2050. Likewise, secondary cities of Cambodia are rapidly emerging while the capital is the largest city with a population of more than two million. Improving urban sustainability is, therefore, necessary for the world, as well as Cambodia. Thus, Cambodia has launched clean city standard indicators, proposed sectoral green city indicators, and adapted one target of global sustainable development goal 11 (UN SDG 11), to improve its urban quality and sustainability. However, using these indicators is not sufficient towards achieving urban sustainability because these indicators are limited in social and economic dimensions. Hence, this study aims to develop all dimensional indicators of sustainability based on all targets of UN SDG 11 with the above indicators. This study focused on the priorities of indicators in Cambodia verified and prioritized by Delphi and analytic hierarchy process (AHP) techniques. Then, a priority-based urban sustainability index for Cambodia was formed based on the concept of sustainability in developing countries. Finally, the standard scores were applied to comparatively assess the sustainability of capital and emerging secondary cities of Cambodia based on the 2018 Commune Database. Through this application, the study also sought to find out whether the priority weights of indicators are necessary for the comparative assessment. The results showed that the sustainability levels of Phnom Penh and Sihanoukville were found to be strong in all environmental, social, and economic dimensions. Battambang is also strong although economic sustainability is slightly lower than the average. Siem Reap is low in economic sustainability level while Poi Pet is remarkably low in environmental and social sustainability. Furthermore, the ranks of sustainability levels of the five cities based on weighted scores are different from their ranks based on unweighted scores. Therefore, this study confirms that priority weights of indicators are necessary for the comparative assessment towards improving the accuracy of the comparison.

BackgroundThe life expectancy of people with severe mental illness (SMI) is shorter than those without SMI, with multimorbidity and poorer physical health contributing to health inequality. Screening tools could potentially assist the optimisation of medicines to protect the physical health of people with SMI. The aim of our research was to design and validate a medicines optimisation tool (OPTIMISE) to help clinicians to optimise physical health in people with SMI.MethodsA review of existing published guidelines, PubMed and Medline was carried out. Literature was examined for medicines optimisation recommendations and also for reference to the management of physical illness in people with mental illness. Potential indicators were grouped according to physiological system. A multidisciplinary team with expertise in mental health and the development of screening tools agreed that 83 indicators should be included in the first draft of OPTIMISE. The Delphi consensus technique was used to develop and validate the contents. A 17-member multidisciplinary panel of experts from the UK and Ireland completed 2 rounds of Delphi consensus, rating their level of agreement to 83 prescribing indicators using a 5-point Likert scale. Indicators were accepted for inclusion in the OPTIMISE tool after achieving a median score of 1 or 2, where 1 indicated strongly agree and 2 indicated agree, and 75th centile value of ≤ 2. Interrater reliability was assessed among 4 clinicians across 20 datasets and the chance corrected level of agreement (kappa) was calculated. The kappa statistic was interpreted as poor if 0.2 or less, fair if 0.21–0.4, moderate if 0.41–0.6, substantial if 0.61–0.8, and good if 0.81–1.0.ResultsConsensus was achieved after 2 rounds of Delphi for 62 prescribing indicators where 53 indicators were accepted after round 1 and a further 9 indicators were accepted after round 2. Interrater reliability of OPTIMISE between physicians and pharmacists indicated a substantial level of agreement with a kappa statistic of 0.75.ConclusionsOPTIMISE is a 62 indicator medicines optimisation tool designed to assist decision making in those treating adults with SMI. It was developed using a Delphi consensus methodology and interrater reliability is substantial. OPTIMISE has the potential to improve medicines optimisation by ensuring preventative medicines are considered when clinically indicated. Further research involving the implementation of OPTIMISE is required to demonstrate its true benefit.Trial registrationThis article does not report the results of a health care intervention on human participants.

Oral health promotion initiatives must be evidence-based and consistent with broader health messaging. The Oral Health Messages for the Australian Public were first produced in 2009 and sought to enable a focused, and strategic approach to oral health promotion in Australia. As the evidence base and needs of the Australian population have since changed, this consensus statement was updated in 2022-2023. The process of updating the messages consisted of 3 phases (preparatory phase, the Delphi technique, final revision phase). The preparatory phase included public and expert consultation, an umbrella review of published scientific literature and review of available recommendations, policies and guidelines. The Delphi technique used in this study was guided by Guidance on Conducting and REporting DElphi Studies (CREDES) and included 2 voting rounds. There were 70 experts in round 1 and 60 experts in round 2. Delphi participants comprised of experts from a variety of fields to ensure diversity and inclusion, balance expertise and maximize stakeholder representation. Consensus was defined as 75% agreement. A total of 11 messages were included in the 2022 update of Oral Health Messages for Australia. The updated oral health messages will support oral health promotion policy and activity at both individual and population level to improve the oral health of Australians.

Development of an educational program in pelvic floor health for women: A consensus using the Delphi method

There is a need for valid methods to assess the readiness for clinical practice of recently graduated physicians. To develop these methods, it is relevant to know the general features of trainees' performance that facilitate supervisors' trust in their ability to perform critical clinical tasks. To discover such essential facets of competence (FOCs), based on the opinion of experienced physician educators. We conducted a Delphi study, consisting of 2 rounds, among 18 experienced physician educators in the Netherlands. Mean, standard deviation, level of agreement, and skewness were calculated for the importance of FOCs for making entrustment decisions. The study yielded a list of 25 FOCs. In the first round, means were between 6.50 and 7.00 on a 7-point Likert scale (SD, 0.42-2.18); in the second round, means ranged from 5.45 to 6.90 (SD, 0.3-2.02). The level of agreement was high for 92% of the FOCs in the first round and 100% of the FOCs in the second round. Our Delphi study found consensus among experts about FOCs that are important for clinical entrustment decisions.

There is little known about private practice nursing as an area of advanced practice. As more nurses take the option to develop private practice, the experiences of and influences on nurses currently in private practice might be a useful guide to the pitfalls and difficulties which might be encountered. In addition, an understanding of the experiences of and influences might assist nursing organizations and health services to provide support to nurses who play an integral part in health care delivery in the community. A research study was undertaken utilizing a two-round Delphi Technique to elicit and assess consensus on the reasons for nurses going into business and the experiences they encountered in becoming and being a nurse entrepreneur. The study instrument in round one comprised a questionnaire with statement headings inviting opinions on the influences and experiences of nurses in business. In the second round, levels of agreement were elicited from responders on collated opinions from round one, including statements formed from comments written in round one. The initial questionnaire also included closed questions to obtain a profile of nurses in private practice. Responders were 59 nurses in private practice in round one and 54 nurses in round two. The themes raised could be grouped under headings of influences, advantages/disadvantages, education/experience, skills/knowledge, characteristics and barriers. The level of agreement on the themes was reasonably high. Dissent occurred on issues of increased income, professional image and support structures. This Delphi study has identified key areas of consensus on the experiences of nurses in private practice who have extended their career into the business arena. It has also identified areas in which further work needs to be carried out to understand this work of nurse entrepreneurs.

BackgroundCurrently, there is no consensus on the optimal management of women with low prognosis in ART. In this Delphi consensus, a panel of international experts provided real-world clinical perspectives on a series of literature-supported consensus statements regarding the overall relevance of the POSEIDON criteria for women with low prognosis in ART.MethodsUsing a Delphi-consensus framework, twelve experts plus two Scientific Coordinators discussed and amended statements and supporting references proposed by the Scientific Coordinators (Round 1). Statements were distributed via an online survey to an extended panel of 53 experts, of whom 36 who voted anonymously on their level of agreement or disagreement with each statement using a six-point Likert-type scale (1 = Absolutely agree; 2 = More than agree; 3 = Agree; 4 = Disagree; 5 = More than disagree; 6 = Absolutely disagree) (Round 2). Consensus was reached if > 66% of participants agreed or disagreed.ResultsThe extended panel voted on seventeen statements and subcategorized them according to relevance. All but one statement reached consensus during the first round; the remaining statement reached consensus after rewording. Statements were categorized according to impact, low-prognosis validation, outcomes and patient management. The POSEIDON criteria are timely and clinically sound. The preferred success measure is cumulative live birth and key management strategies include the use of recombinant FSH preparations, supplementation with r-hLH, dose increases and oocyte/embryo accumulation through vitrification. Tools such as the ART Calculator and Follicle-to-Oocyte Index may be considered. Validation data from large, prospective studies in each POSEIDON group are now needed to corroborate existing retrospective data.ConclusionsThis Delphi consensus provides an overview of expert opinion on the clinical implications of the POSEIDON criteria for women with low prognosis to ovarian stimulation.