Online Summative Paramedic Exam (OSPE)–Phase 1: feasibility of using web-based testing tools to develop and administer a final summative exam to paramedic students

Abstract

Introduction: Web-based tools allow multiple institutions to work collaboratively in the development, validation, and administration of exams. No research is available regarding the feasibility of using Web-based testing tools in paramedic programs. Hypothesis: It is feasible to use Web-based tools to develop and administer a summative evaluation for paramedic students. Methods: Emergency medical services (EMS) educators and medical directors from seven geographically diverse paramedic training institutions used an online instructional management system, WebCT, to develop, review, and administer a summative paramedic exam to current students. The 200-item exam was divided into seven topical categories: airway, cardiology, electrocardiogram (ECG) interpretation, trauma, medical, obstetrics-pediatrics, and EMS operations. Each exam item was reviewed by at least four subject-matter experts. A modification of the Angoff method was used to determine the exam cut score. The proctored exam was administered to 95 graduating paramedic students. A brief questionnaire and consent form were administered on completion of all sections of the exam. Results: Of the 95 students tested, 80 (84.2%) consented to sharing their anonymous data for the study. Sixty-four students (80%; 95% CI: 70%–88%) reported taking the exam in a proctored environment. Forty students (50%; 95% CI: 39%–61%) indicated that they would have had equal or less opportunity to cheat than they would have if they had taken the exam on paper. Four students (5%; 95% CI: 1%–12%) indicated they had more opportunity to cheat than if the exam were taken on paper. Seven students (8.8%; 95% CI: 4%–17%) reported that it would have been possible to cheat on this exam. Sixty-two students (77.5%; 95% CI: 68%–86%) indicated that they would recommend using computerized exams in the future. Eighteen students (22.5%; 95% CI: 14%–33%) indicated they did not recommend using computerized exams in the future. Conclusion: It is feasible to develop and administer a summative paramedic exam using Web-based tools. Online delivery allowed multiple geographically diverse institutions to collaborate. The majority of students recommended computerized exams. Few students indicated that this format would provide more opportunity to cheat.