Two-round Modified Delphi Process Research Articles

A Blueprint for Multi-use Disease Modeling in Health Economics: Results from Two Expert-Panel Consultations.

The current use of health economic decision models in HTA is mostly confined to single use cases, which may be inefficient and result in little consistency over different treatment comparisons, and consequently inconsistent health policy decisions, for the same disorder. Multi-use disease models (MUDMs) (other terms: generic models, whole disease models, disease models) may offer a solution. However, much is uncertain about their definition and application. The current research aimed to develop a blueprint for the application of MUDMs. We elicited expert opinion using a two-round modified Delphi process. The panel consisted of experts and stakeholders in health economic modelling from various professional backgrounds. The first questionnaire concerned definition, terminology, potential applications, issues and recommendations for MUDMs and was based on an exploratory scoping review. In the second round, the panel members were asked to reconsider their input, based on feedback regarding first-round results, and to score issues and recommendations for priority. Finally, adding input from external advisors and policy makers in a structured way, an overview of issues and challenges was developed during two team consensus meetings. In total, 54 respondents contributed to the panel results. The term 'multi-use disease models' was proposed and agreed upon, and a definition was provided. The panel prioritized 10 potential applications (with comparing alternative policies and supporting resource allocation decisions as the top 2), while 20 issues (with model transparency and stakeholders' roles as the top 2) were identified as challenges. Opinions on potential features concerning operationalization of multi-use models were given, with 11 of these subsequently receiving high priority scores (regular updates and revalidation after updates were the top 2). MUDMs would improve on current decision support regarding cost-effectiveness information. Given feasibility challenges, this would be most relevant for diseases with multiple treatments, large burden of disease and requiring more complex models. The current overview offers policy makers a starting point to organize the development, use, and maintenance of MUDMs and to support choices concerningwhich diseases and policy decisions they will be helpful for.

Developing quality indicators for cancer hospitals in China: a national modified Delphi process

ObjectiveAlthough demand and supply of cancer care have been rapidly increasing in recent decades, there is a lack of systemic quality measurement for cancer hospitals in China. This study aimed...

A sociotechnical framework to assess patient-facing eHealth tools: results of a modified Delphi process

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

Development of a research mentorship guide and consensus statement for low- and middle-income countries: Results of a modified Delphi process.

Institutional research mentorship is a form of mentorship whereby institutions foster mentor-mentee relationships. Research mentorship improves research effectiveness and supports relationships. However, resources are needed in order to institutionalize research mentorship tailored to low- and middle- income countries (LMICs). The aim of this study was to develop a consensus document on institutionalizing research mentorship through a modified Delphi process as part of the practical guide development process. This study used a two-round modified Delphi process, which is an iterative, structured approach of consensus decision making. Each participant was asked about a series of items related to research mentorship using Likert scale questions. Agreement for each item was pre-defined as ≥80% of participants rating the item as "agree" or "strongly agree." The items that reached agreement, were then discussed during round two at an in-person conference in Ethiopia. A separate group of individuals only participated virtually. For the final consensus survey, response rates and commenting rates (participants who wrote two or more comments) were compared among conference and non-conference participants. The Delphi process led to the inception of three main themes in terms of developing research mentorship: leveraging existing resources, measuring and evaluating institutional mentorship, and encouraging a research mentorship life cycle. During the virtual first round, 59% (36/61) participants who were emailed completed the survey. In the second round, conference participants had a response rate of 79% (11/14) compared to non-conference participants with a response rate of 45% (21/47). Conference participants had a 100% (11/11) commenting rate whereas non-conference participants had a 38% (8/21) commenting rate. This study achieved consensus in both survey rounds for all 35 items on the consensus document. The data suggest that an in-person conference may increase participant engagement. The consensus developed through a modified Delphi method directly informed a practical guide on institutionalizing research mentorship in LMICs.

Clinical indicators for diabetic eyecare delivered by optometrists in Australia: a Delphi study

ABSTRACT Clinical relevance Valid and updated clinical indicators can serve as important tools in assessing and improving eyecare delivery. Background Indicators for diabetic eyecare in Australia were previously developed from guidelines published before 2013 and then used to assess the appropriateness of care delivery through a nationwide patient record card audit (the iCareTrack study). To reflect emerging evidence and contemporary practice, this study aimed to update clinical indicators for optometric care for people with type 2 diabetes in Australia. Methods Forty-five candidate indicators, including existing iCareTrack and new indicators derived from nine high-quality evidence-based guidelines, were generated. A two-round modified Delphi process where expert panel members rated the impact, acceptability, and feasibility of the indicators on a 9-point scale and voted for inclusion or exclusion of the candidate indicators was used. Consensus on inclusion was reached when the median scores for impact, acceptability, and feasibility were ≥7 and >75% of experts voted for inclusion. Results Thirty-two clinical indicators with high acceptability, impact and feasibility ratings (all median scores: 9) were developed. The final indicators were related to history taking (n = 12), physical examination (n = 8), recall period (n = 5), referral (n = 5), and patient education/communication (n = 2). Most (14 of 15) iCareTrack indicators were retained either in the original format or with modifications. New indicators included documenting the type of diabetes, serum lipid level, pregnancy, systemic medications, nephropathy, Indigenous status, general practitioner details, pupil examination, intraocular pressure, optical coherence tomography, diabetic retinopathy grading, recall period for high-risk diabetic patients without retinopathy, referral of high-risk proliferative retinopathy, communication with the general practitioner, and patient education. Conclusion A set of 32 updated diabetic eyecare clinical indicators was developed based on contemporary evidence and expert consensus. These updated indicators inform the development of programs to assess and enhance the eyecare delivery for people with diabetes in Australia.

Development of consensus-driven SPIRIT and CONSORT extensions for early phase dose-finding trials: the DEFINE study

BackgroundEarly phase dose-finding (EPDF) trials are crucial for the development of a new intervention and influence whether it should be investigated in further trials. Guidance exists for clinical trial protocols and completed trial reports in the SPIRIT and CONSORT guidelines, respectively. However, both guidelines and their extensions do not adequately address the characteristics of EPDF trials. Building on the SPIRIT and CONSORT checklists, the DEFINE study aims to develop international consensus-driven guidelines for EPDF trial protocols (SPIRIT-DEFINE) and reports (CONSORT-DEFINE).MethodsThe initial generation of candidate items was informed by reviewing published EPDF trial reports. The early draft items were refined further through a review of the published and grey literature, analysis of real-world examples, citation and reference searches, and expert recommendations, followed by a two-round modified Delphi process. Patient and public involvement and engagement (PPIE) was pursued concurrently with the quantitative and thematic analysis of Delphi participants’ feedback.ResultsThe Delphi survey included 79 new or modified SPIRIT-DEFINE (n = 36) and CONSORT-DEFINE (n = 43) extension candidate items. In Round One, 206 interdisciplinary stakeholders from 24 countries voted and 151 stakeholders voted in Round Two. Following Round One feedback, one item for CONSORT-DEFINE was added in Round Two. Of the 80 items, 60 met the threshold for inclusion (≥ 70% of respondents voted critical: 26 SPIRIT-DEFINE, 34 CONSORT-DEFINE), with the remaining 20 items to be further discussed at the consensus meeting. The parallel PPIE work resulted in the development of an EPDF lay summary toolkit consisting of a template with guidance notes and an exemplar.ConclusionsBy detailing the development journey of the DEFINE study and the decisions undertaken, we envision that this will enhance understanding and help researchers in the development of future guidelines. The SPIRIT-DEFINE and CONSORT-DEFINE guidelines will allow investigators to effectively address essential items that should be present in EPDF trial protocols and reports, thereby promoting transparency, comprehensiveness, and reproducibility.Trial registrationSPIRIT-DEFINE and CONSORT-DEFINE are registered with the EQUATOR Network (https://www.equator-network.org/).

Strategies for implementing pet robots in care homes and nursing homes for residents with dementia: protocol for a modified Delphi study

BackgroundPet robots are a type of technology-based innovation that have shown positive psychosocial benefits for people with dementia in residential facilities, such as improving mood and social interaction and reducing agitation. Nevertheless, little is known about how pet robots can be implemented in care homes and nursing homes for dementia care in real-world practice. The objectives of this study are to (1) identify contextualised implementation strategies for implementing pet robots into care homes and nursing homes for dementia care and (2) achieve consensus on the most relevant strategies.MethodThis study is informed by a preceding scoping review and qualitative study, which used the Consolidated Framework of Implementation Research (CFIR) to identify multi-level determinants of implementation (i.e. barriers and facilitators). We will use the CFIR-ERIC matching tool to identify relevant implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy to address these determinants. Data from the scoping review and qualitative study will be used to contextualise the generic ERIC strategies for our setting. After that, a group of key stakeholders will be consulted to further contextualise and refine these strategies. Next, a two-round modified Delphi process will be conducted. Fifty-four international expert participants including healthcare professionals and organisational leaders from care homes and nursing homes and academic researchers will be recruited through purposive sampling. During the first Delphi round, participants will be invited to rate the relevance of each implementation strategy on a 9-point Likert scale and provide comments or suggestions. Descriptive statistics will be used to identify whether consensus has been obtained. Inductive qualitative content analysis will be used to analyse and summarise textual responses for any new statements suggested by participants. Statements that do not reach consensus and new statements suggested in round 1 will be taken to the next round, which will follow the same rating process.DiscussionThis study will identify strategies for implementing pet robots in care homes and nursing homes for residents with dementia, which will have practical utility for clinicians, organisations and researchers. It will also demonstrate the practical application (and adaptation) of the CFIR-ERIC tool to identify and contextualise ERIC strategies.

Design and content validation of an instrument measuring user perception of the persuasive design principles in a breastfeeding mHealth app: A modified Delphi study

BackgroundPersuasive design principles are increasingly employed in mHealth apps for motivating users and promoting healthy behaviours among individuals. However, how the persuasive design principles are perceived by the mHealth app users remains unclear. AimTo develop and validate the content validity of an instrument designed to measure the user’s perceptions of the persuasive design principles assimilated in a breastfeeding mHealth app. MethodsA critical review of the literature guided the development of the research instrument. The developed instrument was refined and validated through a two-round modified Delphi process. Ten experts drawn from academic and clinical settings evaluated the items through two content validity indicators, relevance, and clarity and provided narrative feedback. The content validity of the instrument was determined by calculating the Content Validity Index (CVI). Content validity indicators at the scale-level (S-CVI) and item-level (I-CVI) were calculated. ResultsThe results demonstrated high content validity index of individual items in the instrument. All items in the instrument reached an excellent I-CVI ≥ 0.78 for both relevance and clarity except one item. The overall content validity index of the instrument using the average approach was moderate to high (S-CVI/Ave was 0.89 and 0.92 for relevance and clarity, respectively). The developed research instrument is composed of 5 constructs representing 24 items: (1) primary task support, (2) dialogue support, (3) system credibility support, (4) social support and (5) perceived persuasiveness. ConclusionsUsing a modified Delphi approach, the development and validation of the research instrument demonstrated moderate to high content validity for measuring users’ perceptions of the persuasive design principles assimilated in a breastfeeding mHealth app. The evidence from this study supports that the research instrument is valid, relevant, and clear.

Capturing employer's perceptions of key variables in exceptional new practitioners' success by using a modified Delphi process

IntroductionThe primary aim was to determine the key variables in a new practitioner's success from the perspective of pharmacy employers. A secondary aim was to describe the use of a modified Delphi process to collect quality information from external constituents. MethodsA historical review of the literature was conducted to ensure the inquiry extended existing thinking and to aid the development of questions. An asynchronous, online two-round modified Delphi process was used to gather input and develop consensus. The findings were verified by crosswalking with the Habits of Mind. Additionally, the strategies for inviting panelists, practitioner response rates, and the amount and quality of the data collected were examined. ResultsThe historical review yielded 25 articles that were categorized by disciplines studied, methods used, and characteristics found. Fourteen seasoned employers participated. Nine themes describing the behaviors and practices of exceptional new practitioners were identified. These themes were categorized into three groups: Change-Leader, Self-Manager, or Relationship-Builder. Additionally, all the themes mapped to multiple Habits of Mind. As a method for constituent input, the Delphi process yielded employer response rates per school ranging from 10% to 67% and produced five single-spaced pages of text (2124 words) for analysis. ConclusionsThese descriptions of behaviors and practices of exceptional new practitioners build upon existing literature and provide additional guidance for professional development curricula. Additionally, an asynchronous, online Delphi panel can be a useful method to glean important insights from experienced pharmacy employers.

NeoCheck: a prescription-screening tool to optimise pharmacotherapy for hospitalised neonates.

To develop a screening tool to optimise neonatal drug prescription, which is often based on low-quality evidence. Neonatal pharmacotherapy recommendations were identified by literature review and synthesised into NeoCheck tool statements. In a two-round modified Delphi process, experts from Swiss neonatal intensive care units (NICUs) rated their agreement with individual statements using a five-point Likert scale (5 = totally agree). Statements with >65% scores ≥4 in round 1 and >75% scores ≥4 in round 2 were selected. We identified 1375 clinical guidelines via literature review. After synthesis, 158 statements were submitted to 23 experts (1 clinical pharmacist, 22 neonatologists; 65% with >10 years neonatology practice) from 10 Swiss NICUs. Nineteen items did not reach the agreement threshold and were eliminated in the second Delphi round. The final NeoCheck tool comprises 141 statements in 11 medical domains concerning 49 neonatal diseases. Most (79%) statements concern all neonates, 13% concern preterm (<37 weeks gestational age) infants and 3% concern very preterm (<32 weeks gestational age) infants CONCLUSIONS: NeoCheck is the first prescription-screening tool developed to optimise neonatal pharmacotherapy. In a future prospective study, its effect on NICU prescription optimisation and the quality of care will be assessed.

Ghent Older People's Prescriptions Community Pharmacy Screening (GheOP3S)-Tool Version 2: Update of a Tool to Detect Drug-Related Problems in Older People in Primary Care.

BackgroundThe Ghent Older People’s Prescriptions community Pharmacy Screening (GheOP3S)-tool was developed in 2016 as a screening tool to detect drug-related problems (DRPs) and to help in performing medication reviews in older people (≥ 65 years).ObjectiveThis study aimed to revise and update the GheOP3S-tool.MethodsUsers’ comments were collected to improve the usability and appropriateness of the original GheOP3S-tool, followed by a two-round modified Delphi process according to the RAND/UCLA appropriateness method. This included a literature review, a round zero meeting, a first written round (with 15 international and multidisciplinary experts) and a second face-to-face round (with 11 experts) to change, delete or add GheOP3S-criteria. An additional third round with 14 community pharmacists was organised to preserve criteria applicable in the current community pharmacy practice.ResultsThe updated GheOP3S-tool consists of five lists of DRPs and a new addendum containing medications that should be avoided or used with caution in older people with reduced renal function. During the first two rounds, related criteria were grouped, 14 criteria were added and 17 criteria were deleted from the original tool. All criteria were deemed applicable in round 3. This led to a final tool (version 2) with 64 GheOP3S-criteria.ConclusionGheOP3S-criteria were revised and updated according to experts’ agreement on their clinical relevance and recent scientific evidence. Future studies should investigate the impact of pharmacist-led medication reviews with GheOP3S-tool version 2 on clinical, humanistic and economic outcomes in primary care.Supplementary InformationThe online version contains supplementary material available at 10.1007/s40266-021-00862-6.

Using the Delphi technique to determine objectives and topical outline for a pharmaceutical care course: an experience from the Cuban higher education system

BackgroundBeing pharmaceutical care one of the four areas defined by the International Pharmaceutical Federation’s Global Competence Framework, the curriculum redesigned scheme is a priority task to perform a pharmaceutical workforce capable to contribute significantly to the appropriate use of medicines. Therefore, the pharmacy curriculum should be adapted, in order to provide pharmacists with new knowledge and skills to provide pharmaceutical care services. This study used a modified Delphi technique to define objectives and topical outlines for a pharmaceutical care course on a pharmacy curriculum.MethodsA modified Delphi process was used to determine a consensus among proposed course objectives and topical outlines. The preliminary phase of the study included a compilation of prospective objectives and outline topics on which to structure informational flow through the Delphi. A two-round modified Delphi process were completed by the participants in the study. The Delphi questionnaire was organized using six domains: the theoretical program foundation; recommended for teaching literature; instructional and educational objectives of the program (course structure); teaching methods; knowledge, skills and professional values considered; and students’ performance assessments. Nineteen items for evaluation within the referred domains were considered.ResultsConsensus was achieved among 15 participants regarding 10 objectives, and eleven topical outlines related to pharmaceutical care teaching in an undergraduate pharmacy course. Despite this favorable valuation and considering the qualitative evaluations provided by the participants, it was believed appropriate to analyze the recommendation for the inclusion of literature for the teaching of the course in Spanish language (73%). It resulted in a project proposal for the elaboration of a book by a group of authors from all the faculties of pharmacy in the country.ConclusionA Delphi expert panel achieved consensus on topical outline and objectives for a pharmaceutical care course. The results of this study can be used to underline the didactic guidance for pharmaceutical care teaching and learning useful for future pharmacy curriculum upgrades.

Development of quality indicators for the diagnosis and management of achalasia

The management of achalasia has improved due to diagnostic and therapeutic innovations. However, variability in care delivery remains and no established measures defining quality of care for this population exist. We aimed to use formal methodology to establish quality indicators for achalasia patients. Quality indicator concepts were identified from the literature, consensus guidelines and clinical experts. Using RAND/University of California, Los Angeles (UCLA) Appropriateness Method, experts in achalasia independently ranked proposed concepts in a two-round modified Delphi process based on 1) importance, 2) scientific acceptability, 3) usability, and 4) feasibility. Highly valid measures required strict agreement (≧ 80% of panelists) in the range of 7-9 for across all four categories. There were 17 experts who rated 26 proposed quality indicator topics. In round one, 2 (8%) quality measures were rated valid. In round two, 19 measures were modified based on panel suggestions, and experts rated 10 (53%) of these measures as valid, resulting in a total of 12 quality indicators. Two measures pertained to patient education and five to diagnosis, including discussing treatment options with risk and benefits and using the most recent version of the Chicago Classification to define achalasia phenotypes, respectively. Other indicators pertained to treatment options, such as the use of botulinum toxin for those not considered surgical candidates and management of reflux following achalasia treatment. Using a robust methodology, achalasia quality indicators were identified, which can form the basis for establishing quality gaps and generating fully specified quality measures.

Identification and Prioritization of Clinical Decision Support Functionalities Built Within A Computerized Provider Order Entry System

Background: Clinical decision support (CDS) functionalities in Computerized Provider Order Entry system (CPOE) need to be identified by the institutional healthcare providers in developing countries. In this regard, CDS functionalities should be a priority for the execution in CPOE. Objectives: Thus, our study was done to identify and prioritize the CDS functionalities in CPOE. Methods: A Two-round Modified Delphi process was used. Firstly, a systematic search was conducted in electronic databases from the date of database inception to February 2019 for identifying CDS functionalities integrated into CPOE. Studies were retrieved from databases, including PubMed, Embase, ProQuest, Scopus, Web of Science, Cochrane, Science Direct, ACM digital library, and IEEE Xplore Digital Library. Secondly, the Modified Delphi method was carried out in 2019 to provide contextual priorities regarding CDS functionalities in CPOE in two iterative rounds. A total of 12 experts working in the three Intensive Care Units (ICUs) with more than three years of experience with homegrown CPOE in Shiraz Nemazee Teaching Hospital, including two clinical pharmacists, two health information management faculty member, four cases with critical care fellowship, and four critical care experts participated in the study. The ≥ 66.6% agreement was considered as the consensus level. SPSS software version 24 was used for statistical analysis. Results: Totally, 327 studies, which met eligibility criteria were found. A number of 60 potential CDS functionalities in CPOE were identified from eligible studies. Also, 13 out of 60 CDS functionalities reached high priority consensus after 2 iterative Delphi rounds, including drug-allergy checking (83.3%), basic dosing guidance (75%), single dosing checking (66.7%), duplicate therapy checking (66.7%), drug-pregnancy alerts (75%), time-based alerts (66.7%), alert for deep vein thrombosis prophylaxis (66.7%), alerts for duplicate medication order checking (66.7%), drug-drug interaction checking (75%), intelligent dosing guidance based on the patients’ characteristics (66.7%), renal-drug problems checking (83.3%), drug-disease interaction checking (66.7%), and displaying medication/test cost (75%). Conclusions: Our study identified high-priority CDS functionalities to be considered in the CPOE system from the viewpoint of multidisciplinary experts, especially in Iran. Results of this study may be beneficial to plan, design, and implement CDS functionalities in CPOE in the ICU.

Pharmacotherapy in paediatric heart failure: a Delphi process.

Little evidence exists to support pharmacotherapeutic strategies for heart failure management in paediatrics. A recent Europe-wide survey suggests that this translates into substantial variability in clinical practice. To conduct a formal discussion among an expert group of paediatric cardiology physicians on controversial aspects regarding the pharmacotherapy of children heart failure, facilitate consensus, and highlight areas of agreement and disagreement. A two-round modified Delphi process was conducted between July and August 2015. Topics addressed were predominantly selected from the results of a previous Europe-wide survey. Fourteen statements were presented for discussion grouped under three categories; Angiotensin-converting-enzyme-inhibitors: Considerations for optimal dosage; Angiotensin-converting-enzyme-inhibitors for the management of CHDs; Neurohumoral antagonists for the management of dilated cardiomyopathy-related heart failure. A total of 13 paediatricians dedicated to cardiology from across Europe and the United States of America completed the study; of them, 92% had a working experience in the field of more than 10 years and were working in a specific paediatric cardiology unit. Agreement on the acceptance/rejection of 11 statements was achieved. Results show agreement on the importance of a set of topics relevant to the standardisation of the therapy as well as consensus upon specific therapeutic attitudes. We have found areas of common thinking and motivation, which can provide a means of triggering scientific collaboration. Our results might also contribute to disseminate available paediatric evidence and promote reducing unjustified variability in everyday practice. Until solid evidence is available, other research methods can contribute to advancing the goal of safe and effective paediatric heart failure pharmacotherapy.

The medication discrepancy taxonomy (MedTax): The development and validation of a classification system for medication discrepancies identified through medication reconciliation

Medication discrepancies directly impact patient safety and can adversely impact quality of care and resource utilization at transitions of care. To develop a common nomenclature and taxonomy for classifying and reporting medication discrepancies and to assess the content validity and reliability of the taxonomy. The taxonomy was developed following a multi-stage process. The content of the taxonomy was then assessed using expert opinion through a two-round modified Delphi process. The expert panel comprised 10 experts who were selected based on pre-defined selection criteria. Six experienced pharmacists were then invited to classify medication discrepancies from a number of fictitious cases (adapted from authentic cases) using the taxonomy. The medication discrepancy taxonomy (MedTax) comprises 12 main types and 28 sub-types of discrepancies. A set of operational instructions and definitions to aid the use of the taxonomy was formulated. The overall Average content validity index (Ave-CVI) was 0.93 and interrater reliability was 0.67 (multirater κfree), indicating substantial agreement. An excellent internal consistency of the taxonomy was established (Kuder-Richardson Formula 20 (KR-20) = 0.92). A content valid and reliable taxonomy for classifying medication discrepancies was developed. The MedTax may be used to classify medication discrepancies identified following medication reconciliation services. The clear and consistent reporting of medication discrepancies arising from medication reconciliation services may be of value to policy makers, healthcare professionals and researchers, when evaluating such services. The MedTax was designed to fill an essential void in global endeavors to reinforce standardization of medication reconciliation practices and to improve medication safety across transitions of care.

Determining predictors of sepsis at triage among children under 5 years of age in resource-limited settings: A modified Delphi process.

Sepsis is a life-threatening dysfunction of the immune system leading to multiorgan failure that is precipitated by infectious diseases and is a leading cause of death in children under 5 years of age. It is necessary to be able to identify a sick child at risk of developing sepsis at the earliest point of presentation to a healthcare facility so that appropriate care can be provided as soon as possible. Our study objective was to generate a list of consensus-driven predictor variables for the derivation of a prediction model that will be incorporated into a mobile device and operated by low-skilled healthcare workers at triage. By conducting a systematic literature review and examination of global guideline documents, a list of 72 initial candidate predictor variables was generated. A two-round modified Delphi process involving 26 experts from both resource-rich and resource-limited settings, who were also encouraged to suggest new variables, yielded a final list of 45 predictor variables after evaluating each variable based on three domains: predictive potential, measurement reliability, and level of training and resources required. The final list of predictor variables will be used to collect data and contribute to the derivation of a prediction model.

Predictor variables for post-discharge mortality modelling in infants: a protocol development project.

Over two-thirds of the five million annual deaths in children under five occur in infants, mostly in developing countries and many after hospital discharge. However, there is a lack of understanding of which children are at higher risk based on early clinical predictors. Early identification of vulnerable infants at high-risk for death post-discharge is important in order to craft interventional programs. To determine potential predictor variables for post-discharge mortality in infants less than one year of age who are likely to die after discharge from health facilities in the developing world. A two-round modified Delphi process was conducted, wherein a panel of experts evaluated variables selected from a systematic literature review. Variables were evaluated based on (1) predictive value, (2) measurement reliability, (3) availability, and (4) applicability in low-resource settings. In the first round, 18 experts evaluated 37 candidate variables and suggested 26 additional variables. Twenty-seven variables derived from those suggested in the first round were evaluated by 17 experts during the second round. A final total of 55 candidate variables were retained. A systematic approach yielded 55 candidate predictor variables to use in devising predictive models for post-discharge mortality in infants in a low-resource setting.

An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process.

BackgroundThe number of e-mental health apps is increasing rapidly. Studies have shown that the use of some apps is beneficial, whereas others are ineffective or do not meet users’ privacy expectations. Individuals and organizations that curate, recommend, host, use, or pay for apps have an interest in categorizing apps according to the consensus criteria of usability and effectiveness. Others have previously published recommendations for assessing health-related apps; however, the extent to which these recommendations can be generalized across different population groups (eg, culture, gender, and language) remains unclear. This study describes an attempt by Canadian stakeholders to develop an e-mental health assessment framework that responds to the unique needs of people living in Canada in an evidence-based manner.ObjectiveThe objective of our study was to achieve consensus from a broad group of Canadian stakeholders on guiding principles and criteria for a framework to assess e-mental health apps in Canada.MethodsWe developed an initial set of guiding principles and criteria from a rapid review and environmental scan of pre-existing app assessment frameworks. The initial list was refined through a two-round modified Delphi process. Participants (N=25) included app developers and users, health care providers, mental health advocates, people with lived experience of a mental health problem or mental illness, policy makers, and researchers. Consensus on each guideline or criterion was defined a priori as at least 70% agreement. The first round of voting was conducted electronically. Prior to Round 2 voting, in-person presentations from experts and a persona empathy mapping process were used to explore the perspectives of diverse stakeholders.ResultsOf all respondents, 68% (17/25) in Round 1 and 100% (13/13) in Round 2 agreed that a framework for evaluating health apps is needed to help Canadian consumers identify high-quality apps. Consensus was reached on 9 guiding principles: evidence based, gender responsive, culturally appropriate, user centered, risk based, internationally aligned, enabling innovation, transparent and fair, and based on ethical norms. In addition, 15 informative and evaluative criteria were defined to assess the effectiveness, functionality, clinical applicability, interoperability, usability, transparency regarding security and privacy, security or privacy standards, supported platforms, targeted users, developers’ transparency, funding transparency, price, user desirability, user inclusion, and meaningful inclusion of a diverse range of communities.ConclusionsCanadian mental health stakeholders reached the consensus on a framework of 9 guiding principles and 15 criteria important in assessing e-mental health apps. What differentiates the Canadian framework from other scales is explicit attention to user inclusion at all stages of the development, gender responsiveness, and cultural appropriateness. Furthermore, an empathy mapping process markedly influenced the development of the framework. This framework may be used to inform future mental health policies and programs.

Selecting candidate predictor variables for the modelling of post-discharge mortality from sepsis: a protocol development project.

Post-discharge mortality is a frequent but poorly recognized contributor to child mortality in resource limited countries. The identification of children at high risk for post-discharge mortality is a critically important first step in addressing this problem. The objective of this project was to determine the variables most likely to be associated with post-discharge mortality which are to be included in a prediction modelling study. A two-round modified Delphi process was completed for the review of a priori selected variables and selection of new variables. Variables were evaluated on relevance according to (1) prediction (2) availability (3) cost and (4) time required for measurement. Participants included experts in a variety of relevant fields. During the first round of the modified Delphi process, 23 experts evaluated 17 variables. Forty further variables were suggested and were reviewed during the second round by 12 experts. During the second round 16 additional variables were evaluated. Thirty unique variables were compiled for use in the prediction modelling study. A systematic approach was utilized to generate an optimal list of candidate predictor variables for the incorporation into a study on prediction of pediatric post-discharge mortality in a resource poor setting.