The Development and Evaluation of New Versions of the Written Examination for the American Heart Association Advanced Cardiac Life Support Provider Course

Abstract

Study objective: To create new versions of the written, multiple-choice examination used in the American Heart Association (AHA) Advanced Cardiac Life Support course, evaluate their reliability and difficulty, and then design revised versions with improved reliability and of standardized difficulty. Design: Psychometric evaluation of new versions of the AHA Advanced Cardiac Life Support test and revisions. Setting: AHA Advanced Cardiac Life Support courses. Participants: Candidates for completion of AHA Advanced Cardiac Life Support provider courses in five states. Intervention: The course content was divided into 11 content areas that were weighted for importance and appropriateness for testing in a multiple-choice format. The weights were used to construct a blueprint for a 50-question, multiple-choice examination. Five versions of the examination were then constructed based on the content blueprint, drawing from new questions and expert revision of previously written questions. Reliability and difficulty were assessed using 915 administrations at five different sites nationwide. The initial test versions differed in their degree of difficulty, which was not explained by demographic factors. The results were used to revise three of the versions to improve reliability and equalize difficulty of the versions. Measurements and main results: The final five versions have estimated reliability ranging from Cronbach's α of .62 to .86. Mean scores ranged from 87.4% to 89.1%. Conclusion: After field testing and revision, five examinations with acceptable reliability and roughly equal difficulty were constructed. The new examinations test the participants' knowledge of important aspects of resuscitation science and practice based on a blueprint of the course content. [Billi JE, Gruppen LD, Boxer GJ, Oh M, Donnelly MB, Schroeder P, Littrell KA, Grauer K, Cavallaro DL, Thies W, Membrino GE: The development and evaluation of new versions of the written examination for the American Heart Association Advanced Cardiac Life Support provider course. Ann Emerg Med July 1994;24:26-31.]