Educational outcomes when traditional clinical experience is replaced with simulation are beginning to be reported in the literature (Harder, 2010; Schlairet & Pollock, 2010; Sears, Goldsworthy, & Goodman, 2010). As nursing programs increasingly integrate simulation throughout the curriculum and substitute simulation for clinical time, reliable and valid evaluation instruments designed specifically to measure learning outcomes and the effectiveness of the simulation as a teaching strategy are needed (Harder; Kardong-Edgren, Adamson, & Fitzgerald, 2010; Yuan, Williams, & Fang, 2012). Because cognitive, affective, and psychomotor behaviors occur simultaneously in real clinical situations, simulation evaluation tools should measure learning in all three domains. A review of 22 published instruments for evaluation of simulations (Kardong-Edgren et al.) concluded that four tools came close to meeting this criterion, including a tool developed by faculty at Creighton University School of Nursing. The Creighton Simulation Evaluation Instrument (C-SEI) was chosen for this study because it is based on core competencies (critical thinking, communication, assessment, and technical skills) integrated throughout the 1998 version of the American Association of Colleges of Nursing Essentials of Baccalaureate Education for Professional Nursing Practice (Todd, Manz, Hawkins, Parsons, & Hercinger, 2008). Twenty-two behaviors were identified as essential to include in the instrument, which was developed by Todd and colleagues to evaluate a team in simulation. Content validity was determined by an expert panel and by a review of the literature. Interrater reliability, determined by calculating the percentage of agreement between two raters, was reported to be 0.85 to 0.89 for the categories assessment, communication, critical thinking, and technical skills. According to Burns and Grove, with a simple percent of interrater agreement, 90 percent is considered good and 80 percent is considered acceptable (as cited in Todd et al., 2008, p. 12). Todd and colleagues concluded that additional research with different evaluators, varying levels of students, and different scenarios was needed to verify the results. According to Kardong-Edgren and colleagues (2010), one way to move simulation science forward is to provide data for reliability and validity statistics for existing tools, using large sample sizes in more than one geographic location. The purpose of the current study was to assess the degree of agreement between the ratings of student performance during a clinical simulation by critical care course instructors and the course coordinator. Approval was obtained from the study university's Institutional Review Board. Permission was granted from Creighton University School of Nursing to use the C-SEI. Method For the study, the researcher and the critical care course coordinator agreed to utilize the evaluation method established and tested by Creighton University (Todd et al., 2008). The method consists of assigning a zero (does not demonstrate competency) or 1 (demonstrates competency) to specific behaviors. If a particular behavior is not included in a scenario, evaluators have the option of identifying the behavior as not applicable (NA). Scores are based on the percentage of competencies that were successfully demonstrated. For the purpose of this study, a score of 75 percent was considered passing. Guidelines for the development of expected behaviors were followed. A detailed worksheet defining the expected behaviors for each competency was completed by the researcher and the course coordinator. Clinical instructors completed a one-hour training session that included an introduction to the scenario and an explanation of the C-SEI and the scoring method. The group-scoring method represented a change from the school of nursing's usual clinical evaluation, where each student is evaluated individually. …
Read full abstract
Jan 1, 2019
Benjamin Nicholson + 6
Open Access