The Delphi method and its application in medical education research

Objective To establish a standardized and feasible family health assessment index system based on the family practice system in Shanghai urban communities. Methods On the basis of literature review and special topic discussion, a preliminary index system framework was developed and an expert consultation questionnaire was compiled. Twenty one experts of general practice, medical specialties, health administration and public health management were invited and two rounds of improved Delphi expert consultation were conducted. The results of statistical analysis were summarized and the evaluation index system was established. Results The evaluation index system included 5 first-level indicators, 17 second-level indicators and 63 third-level indicators. The weight coefficients of first-level indicators in basic information, family structure, family function, family resources and health status of the family members were 0.199 6, 0.197 5, 0.201 7, 0.185 0, 0.216 2, respectively. In the two rounds of consultation, the positive coefficient of experts was both 100%, the degree of expert authority was 0.867 and 0.880, and the coordination coefficient was 0.294 and 0.415, respectively (P<0.05). The operational coordination coefficients were 0.320 and 0.402 (P<0.05), respectively. Conclusion The family health assessment index system constructed by the Delphi expert consultation method can provide reference for family doctors to carry out family health assessment in urban communities. Key words: Delphi technique; Health assessment; Family doctor system service

To constitute school sanitation standard using modified Delphi method, and to explore the feasibility and the predominance of Delphi method in the constitution of school sanitation standard. Two rounds of expert consultations were adopted in this study. The data were analyzed with SPSS15.0 to screen indices of school sanitation standard. Thirty-two experts accomplished the 2 rounds of consultations. The average length of expert service was (24.69 ±8.53) years. The authority coefficient was 0.729 ±0.172. The expert positive coefficient was 94.12% (32/34) in the first round and 100% (32/32) in the second round. The harmonious coefficients of importance, feasibility and rationality in the second round were 0.493 (P<0.05), 0.527 (P<0.01), and 0.535 (P<0.01), respectively, suggesting unanimous expert opinions. According to the second round of consultation, 38 indices were included in the framework. Theoretical analysis, literature review, investigation and so on are generally used in health standard constitution currently. Delphi method is a rapid, effective and feasible method in this field.

Objective To establish the benign prostatic hyperplasia evaluation index system for nursing quality. Methods Indicators framework for perioperative nursing quality evaluation index system was built up by the theoretical research, clinical research, expert meetings approach firstly, then the Delphi method was used to screen and verify the indicators. Results Two rounds of consultation questionnaires were collected with rate of 86.0%, 95.3% respectively, the first round of consultation had the retention of 13 indicators, removed 4 indicators of those. After the second round, the average score of important target was no less than 8.0, and more than half of indicators had a full mark, coefficient of variation targets no higher than 0.20 were up to 100%. Coefficient of concordance of two rounds were 0.241 and 0.433 respectively. Conclusions After two rounds of consultation, there were 13 quality indicators left and experts had well authoritativeness and representative. Key words: Benign prostatic hyperplasia; Nursing care; Quality indicators; Single disease quality management; Delphi method

Objective Based on the theory of the Balanced Score Card framework and using Delphi method and analytic hierarchy process to construct the clinical nursing staff performance appraisal index evaluation system and provide reference for clinical nursing staff post management. Methods According to the theory of Balanced Score Card, constructed the preliminary establishment of index system for performance evaluation of clinical nursing. Used Delphi method to interview 26 nursing experts to select clinical nursing performance evaluation indicator. In the end, applied hierarchical analysis to determine index weight at all levels. Results Conducted two rounds of expert consultation; distributed 2 rounds of questionnaires by 27 and 26; recycled by 26 and 25; the recycling rate was 96.3% and 99.1%. It showed that the active coefficient of experts was higher. The degree of experts′ authority was 0.89, which meaned the higher authority. Through two rounds of expert consultation, the Kendall coordination coefficient was 0.199-0.388. The significant test indicated that it was statistically significant (P < 0.05), which meaned the experts tend to agree. After two rounds of expert consultation, the final clinical nursing staff performance appraisal indicators included 4 first-class indexes, 10 secondary indexes, 24 third-class indexes. Each index weight was determined by the analytic hierarchy process. In primary index, the internal process, learning and growth, performance indicators and the weight of customer satisfaction were 0.3, 0.3, 0.3 and 0.1 respectively. In secondary index weight, the top three were the work′s quality, post, and quantity. In the third class index weight, the top three were undertaking clinical work, annual nurse competence grading and the person-time of critical patients′ management or surgical patient management. Conclusions Based on Delphi method and analytic hierarchy process to determine the clinical nursing staff performance appraisal evaluation index, whic can provide theoretical basis for clinical nursing staff post management. Key words: Performance appraisal; Analytic hierarchy process; Balanced Score Card; Nursing management

Objective To establish a learning evaluation indexes system of flipped teaching method, and provide the basis for evaluating academic performance of each student who majors in nursing. Methods The indexes were determined by Delphi method of two rounds of consultation involving 15 experts, and the weight of every index was set up by analytic hierarchy process. Results The response rates of questionnaire investigation for the two rounds of consultation were 13/15 and 12/13 respectively. The authority value of experts was 0.847. The Kendall concordance coefficients for the indexes of first-level, second-level and third-level were 0.373, 0.258 and 0.595 respectively, P<0.05. Finally, the evaluation indexes system including 3 first-level indexes, 5 second-level indexes, and 6 third-level indexes was set up. Conclusions The results of the two rounds of expert consultation are reliable. The consistency of expert opinions is good. The study provides a scientific and systematic evaluation tool for evaluating nursing undergraduates' performance under flipped teaching. Key words: Students, nursing; Computer-aided teaching; Evaluation indexes

Consensus group methods, such as the Delphi method and nominal group technique (NGT), are used to synthesize expert opinions when evidence is lacking. Despite their extensive use, these methods are inconsistently applied. Their use in medical education research has not been well studied. The authors set out to describe the use of consensus methods in medical education research and to assess the reporting quality of these methods and results. Using scoping review methods, the authors searched the Medline, Embase, PsycInfo, PubMed, Scopus, and ERIC databases for 2009-2016. Full-text articles that focused on medical education and the keywords Delphi, RAND, NGT, or other consensus group methods were included. A standardized extraction form was used to collect article demographic data and features reflecting methodological rigor. Of the articles reviewed, 257 met the inclusion criteria. The Modified Delphi (105/257; 40.8%), Delphi (91/257; 35.4%), and NGT (23/257; 8.9%) methods were most often used. The most common study purpose was curriculum development or reform (68/257; 26.5%), assessment tool development (55/257; 21.4%), and defining competencies (43/257; 16.7%). The reporting quality varied, with 70.0% (180/257) of articles reporting a literature review, 27.2% (70/257) reporting what background information was provided to participants, 66.1% (170/257) describing the number of participants, 40.1% (103/257) reporting if private decisions were collected, 37.7% (97/257) reporting if formal feedback of group ratings was shared, and 43.2% (111/257) defining consensus a priori. Consensus methods are poorly standardized and inconsistently used in medical education research. Improved criteria for reporting are needed.

Objective To construct an index system for evaluating the core competence of clinical teachers, and provide a scientific and objective basis for the selection, training, assessment and management of clinical teachers. Methods Delphi method was used to conduct two rounds of consultation among 21 experts of administrive management, nursing management, nursing education and clinical nursing fields from January 2016 to July 2016. Core competence evaluation index for clinical teachers of specialty nurses was set up and the weights of indexes in all levels were assigned. Results The evaluation indexes finally set up contained 6 first-grade indexes such as clinical practice ability, teaching ability, scientific research ability, professional development ability, organization and management ability and critical thiking ability; 19 second-grade indexes and 52 third-grade indexes. The active coefficients of experts in two rounds were 91.3% and 100%; the mean degree of familiarity of experts was 0.971; the mean authority coefficient was 0.943; and the cooperation indexes of two rounds were 0.319 and 0.323 (P<0.01). Conclusions The opinions of experts are reasonable and the results are credible. Combind with the results of previous studies, this study constructs the index system for evaluating the core competence and enriches the clinical nursing teaching theory. This study provides a solid foundation for further constructions of evaluation systems in the future. Key words: Delphi method; Specialty nurses; Core competence; Clinical teachers; Evaluation index system

Prueba máximo caracteres resumen autor

Objective To construct the gall-stone patients' nursing workload measurement project by Delphi method,so as to measure single disease workload scientifically and reasonably. Methods The gall-stone patients' nursing workload measurement project were designed on the basis of consulting literature and experts discussion, then two rounds of consultations were made with experts by using the Delphi method. Results The valid recovery rate of questionnaires of two rounds consultations were all 100% (22/22). The expert authority coefficient was 0.886 1. Nursing workload measurement project, directly related to the gall bladder calculi 54 nursing projects, including technical 46, unskilled 7; indirectly related to the gall bladder calculi 36 nursing projects, including technical 17, unskilled 19. Conclusions The measurement project of gall-stone patient determined by Delphi method has high reliability, which is significant to the scientific measuring of nursing single disease workload. Key words: Cholecystolithiasis; Delphi technique; Nursing workload; Measurement project

Objective To construct a set of evaluation index system for medical research project, to make the work justice and fairness, to manage the research work scientificly and standardized, to promote scientific research and hospital development closely, so as to better serve the society. Methods Based on the literature review, semi structure interview method and current medical research project evaluation index system and related management methods, a preliminary index system was formed. Using the Delphi method to carry out two rounds anonymous questionnaire consulting, the medical research project acceptance evaluation index system was finalized. Results The evaluation index system of medical research projects includes 5 first-level indexes and 21 second-level indexes.Ranking from high to low is based on the weight coefficient of the size. Conclusions Through the establishment of the index system of medical scientific research projects, the project acceptance management work has become scientifically planned, which guide scientific research project productive and improving the scientific management level and technological innovation ability. Key words: Delphi method; Medical research projects; Acceptance evaluation; Indicator system

Objective We aimed to construct the handbook of self-management for patients with permanent colostomy by using the Delphi method. Methods Based on literature review, 23 experts were consulted twice by using the Delphi method about the related problems of constructing the handbook of self-management for patients with permanent colostomy, and the consulting results were analyzed later. Results The valid response rates of two rounds of expert consultation were 91.3% and 100.0%, respectively. The authority coefficient of the experts (Cr) was 0.879, and coordination coefficients of the experts' opinion (W) were 0.387 and 0.307, respectively. Finally, the handbook of self-management for patients with permanent colostomy was determined by two rounds of expert consultation. Conclusions The handbook of self-management for patients with permanent colostomy, constructed by using the Delphi method, provides a practical and effective tool for the patients with permanent colostomy to manage themselves, so as to provide a guidance and supervision role on self-management, such as stoma management, role management and emotion management, further more, it provides references for the clinical and community nursing of the patients with permanent colostomy. Key words: Permanent colostomy; Self-management; Delphi method

Objective To establish the evaluation index system on maternal health seeking behavior with Delphi method. Methods Based on Andersen health service utilization models, literature analysis was used to establish evaluation index system on maternal health seeking behavior, then Delphi method was carried out to revise index system through two rounds questionnaire consulting from 24 experts. Results The confirmed index system of maternal health seeking behavior included 4 first-level index, 12 second-level index and 57 third-level index.The average authority degree of experts were 0.869, and the coordination coefficients of expert in the total index were 0.195 and 0.166(P<0.01). The weight coefficients of the indicators at first level were 0.1529, 0.3403, 0.2786 and 0.2281, respectively. Conclusion The representativeness, authority degree and coordination of expert are good, and the tentatively established evaluation index system on maternal health seeking behavior is scientific and reasonable. Key words: Delphi method; Maternal; Health seeking behavior; Index screening

Objective Taking into account of the actual scientific research conducted at medical universities in Yunnan Province, Southwest China, to develop an appropriate evaluation index system of scientific research performance. Methods main methodologies and procedures adopted to develop the evaluation index system include literature analysis, Delphi method, Group Analytic Hierarchy Process and Expert authority calculation. Results The evaluation index system of scientific research performance consists of scientific research input index system and output index system. The scientific research input index system includes financial input and human resources input as two primary indicators.While the scientific research output index system is made up of two primary indicators, named quantitative output and qualitative output. There are another six second-level indicators, six third-level indicators and 37 fourth-level indicators to make up of the quantitative output index system. Calculation of the indicator weight will consider the natures of different units or departments, the mixed analysis of quantitative and qualitative methods, as well as one-vote veto decision were adopted to make sure classified assessment were actually implemented during the evaluation process. Conclusions The index system is suggested to be used for evaluating scientific research performance in different colleges or departments. Adopting such system can help the management of scientific research performances at both micro-and meso-levels.It may also inspire more researchers and research management personnel in a broader sense. Key words: Scientific research performance; Evaluation index system; Delphi; Multi-AHP

Objective To build a scientific, comprehensive, operable evaluation indicator system on clinical risk exposure of medical devices from the perspective of medical institutions. It can provide reference and help for medical institutions to carry out medical device risk management activities. Methods Through literature analysis and expert interview, the draft index system was proposed. Select 20 experts and scholars to form a panel of experts. A 2-rounds Delphi survey was made on the proposed indicators. The reliability of this expert consultation was tested by the response rate, the authority coefficient and concordance coefficient of experts. Results The response rates of the 2 rounds were 90%, 100% respectively. After two rounds of expert consultation, expert opinion has been basically concentrated. Ultimately, an index system comprising 4 first-level, 14 second-level, and 48 third-level was built. Conclusion The expert consultation has achieved reliable results. The evaluation indicator system on the clinical application risk of medical device can be used as an evaluation tool for medical device. Key words: Delphi method; Medical device; Clinical application; Risk assessment; Indicator system

Objective To construct an evaluation index system of nursing quality for pressure injury in Operating Room based on Donabedian structure-process-outcome three-dimensional quality assessment model to provide a reference for nursing quality assessment by nursing managers in Operating Room. Methods From March to July 2018, we selected 18 experts such as chairman of Committee of Operating Room of Shanghai Nursing Association, head nurses of Operating Room at ClassⅢ Grade A general hospitals in Shanghai, nurse specialist of pressure injury management team, by purposive sampling as subjects. The consultation form was designed based on search of operation related pressure injury at home and abroad as well as Donabedian structure-process-outcome three-dimensional quality assessment model. Delphi method was used to 2 rounds of expert consultation. Items in all levels were deleted, merged and revised according to expert opinions and to define the evaluation indexes in all levels of nursing quality for pressure injury in Operating Room as well as the weight. Results This study carried out 2 rounds of expert consultation, with 0.989 for the expert judgment coefficient, 0.956 for the expert familiarity coefficient and 0.973 for the expert authority coefficient. The variation coefficients of second-level indexes and third-level indexes ranged from 0.109 to 0.138 in 2 rounds of expert consultation. The evaluation index system of nursing quality for pressure injury in Operating Room included 3 first-level indexes, 9 second-level indexes and 28 third-level indexes. Conclusions This study constructs a scientific and reasonable evaluation index system of nursing quality for pressure injury in Operating Room with the method of Delphi which can be used to nursing quality assessment for pressure injury in Operating Room and is in favor of continuous quality improvement of nursing in Operating Room. Key words: Operating Room; Pressure injury; Nursing quality; Evaluation index; Delphi methods; Expert consultation; Three-dimensional quality assessment model