Psychometric properties of a novel knowledge assessment tool of mechanical ventilation for emergency medicine residents in the northeastern United States.

Tania D. Strout,Jeremy B. Richards,Susan R. Wilcox,Todd A. Seigel

doi:10.3352/jeehp.2016.13.10

Abstract

Purpose:Prior descriptions of the psychometric properties of validated knowledge assessment tools designed to determine Emergency medicine (EM) residents understanding of physiologic and clinical concepts related to mechanical ventilation are lacking. In this setting, we have performed this study to describe the psychometric and performance properties of a novel knowledge assessment tool that measures EM residents’ knowledge of topics in mechanical ventilation.Methods:Results from a multicenter, prospective, survey study involving 219 EM residents from 8 academic hospitals in northeastern United States were analyzed to quantify reliability, item difficulty, and item discrimination of each of the 9 questions included in the knowledge assessment tool for 3 weeks, beginning in January 2013.Results:The response rate for residents completing the knowledge assessment tool was 68.6% (214 out of 312 EM residents). Reliability was assessed by both Cronbach’s alpha coefficient (0.6293) and the Spearman-Brown coefficient (0.6437). Item difficulty ranged from 0.39 to 0.96, with a mean item difficulty of 0.75 for all 9 questions. Uncorrected item discrimination values ranged from 0.111 to 0.556. Corrected item-total correlations were determined by removing the question being assessed from analysis, resulting in a range of item discrimination from 0.139 to 0.498.Conclusion:Reliability, item difficulty and item discrimination were within satisfactory ranges in this study, demonstrating acceptable psychometric properties of this knowledge assessment tool. This assessment indicates that this knowledge assessment tool is sufficiently rigorous for use in future research studies or for assessment of EM residents for evaluative purposes.

Highlights

Management of mechanical ventilation is an important aspect of caring for intubated, critically ill patients and adhering to evidence-based practices can improve patient outcomes [13]
We report the psychometric characteristics of the knowledge assessment tool, including reliability, item difficulty, and item discrimination, as understanding the performance characteristics of this tool will strengthen its utility in generalized clinical educational settings and future research studies
We describe the psychometric and performance characteristics of a knowledge assessment tool designed to assess Emergency medicine (EM) residents’ understanding of concepts in mechanical ventilation

Summary

Introduction

Management of mechanical ventilation is an important aspect of caring for intubated, critically ill patients and adhering to evidence-based practices can improve patient outcomes [13]. EM residents provide a substantial portion of the clinical care of patients in the ED, including caring for critically ill, mechanically ventilated patients [4,5]. Accurately and reliably determining EM residents’ knowledge and understanding of principles of mechanical ventilation is important for both education and clinical care. Focused and topical education regarding clinically relevant issues in mechanical ventilation has the potential to improve residents’ clinical care and patient outcomes [6,7,8]

Methods

Results

Conclusion