A comparison of matched and aggregated group outcomes data for evaluating continuing education of hematology and oncology health care professionals.

Abstract

Capturing educational outcomes from health care professionals is often challenging. Therefore, many providers utilize aggregated group data (comparing all preassessments to all postassessments) rather than matched group data (comparing pre- and postassessments only for those learners who completed both). To address the agreement between aggregate and matched outcomes, a preliminary analysis was conducted. Matched and aggregated group data were compared from 4 hematology/oncology education series and 3 satellite symposia. Moore's Level 3 and 4 and outcomes were assessed using an electronic audience response system before and after each activity. Knowledge and competence shifts as well as the response spectrum for both groups were compared. A total of 2953 health care professionals were educated in 7 programs comprising 128 live activities. The programs employed a combined total of 39 practice pattern, knowledge, competence, and self-assessed confidence/competence questions. All knowledge and competence shifts were within 10 absolute percentage points between the matched and aggregated groups with an average difference of 3.4 percentage points. The 39 questions had 185 possible choices and 370 total possible responses. When all responses for the matched and aggregated groups were compared, 95% were within 5 absolute percentage points and 99% were within 10 absolute percentage points. The agreement between the groups was found regardless of program or question type. Overall, the aggregated and matched group results were comparable. Aggregated data may be sufficiently accurate for many program evaluation purposes, depending on the degree of certainty required by the evaluation stakeholders.