Construct validity of the professional behavior evaluation instrument from the National Standard Paramedic Curriculum

Abstract

Introduction: Professional behavior is difficult to measure and define. The literature does not report any agreed-upon definition of professionalism, and no studies were found that demonstrate valid measurement of this important domain. Hypothesis: This study examined the construct validity of the Professional Behavior Evaluation (PBE) instrument published by the National Standard Paramedic Curriculum in 1998. Methods: Subjects consisted of 222 paramedics in a municipal fire department system serving a population of 1.32 million. The paramedics had a median experience of eight years and a range of service of one to 30 years. All evaluators are practicing paramedics and hold state instructor credentials. They hold faculty positions at the university level and have been teaching emergency medical services for an average of nine years, with teaching experience ranging from two to 22 years. Using a prospective factor analytic design, five paramedic educators assessed the 222 practicing paramedics using the dimensions of integrity, empathy, self-motivation, appearance, self-confidence, communications, time management, teamwork-diplomacy, respect, patient advocacy, and careful delivery of service on a Likert-type scale. The evaluations occurred as the five evaluators participated in eight-hour “ride-alongs” with the subjects during regular shifts. The ride-alongs resulted in a summative rating at the end of the shift using the research instrument. The PBE instrument was modified to a five-point Likert-type scale using 1 = poor, 2 = below average, 3 = average, 4 = above average, and 5 = exemplary as anchors. Each dimension used the standard definition of terms included in the instrument in the curriculum. Results: The data were subjected to principal components factor analysis with Varimax rotation. The analysis identified one factor that explained 65% of the variance. Cronbach's alphas ranged from 0.46 to 0.79. Conclusion: Only one factor emerged from the analysis and is best thought of as a generic type of professional behavior. Additional research is needed to develop theory in professionalism and develop scales to measure other potentially useful dimensions such as professional development, critical thinking, and personal/professional balance.