Using item analysis to evaluate hand hygiene self-assessments at Alberta health services

Abstract

BackgroundIt is important to evaluate the quality of assessments used in education as the evaluation can identify poorly worded questions, or questions which are not contributing to the validity or reliability of the test. The objective of this study was to perform an item analysis on the hand hygiene self-assessments used at Alberta Health Services to determine the assessments’ validity, reliability, and to highlight questions that required rewording or removal. MethodsThe sample included all first completed attempts between April 2018 and December 2020. Components of the analysis included item difficulty, item discrimination, B-index, point biserial correlation, overall discrimination score, distractor efficiency, and Cronbach's alpha. ResultsA total of 650 health care workers completed the assessment. The average item difficulty was 0.76 (0.32-0.99), average discrimination index was 0.23 (0.02-0.65), average B-index was 0.15 (0.02-0.45), and average point biserial correlation was 0.26 (0.07-0.54). Seventeen of 45 items had an overall discrimination score greater than 7 of 9 and 20 of 21 (95%) multiple choice questions had at least 1 non-functioning distractor. The self-assessment had an overall Cronbach's alpha value of 0.68. ConclusionThe item analysis provided robust evidence for improving the validity and reliability of the self-assessment. The steps performed in this item analysis can be useful for educators interested in improving assessments that include different question types.