The Interactive Media Package for Assessment of Communication and Critical Thinking (IMPACCT©): Testing a Programmatic Online Communication Competence Assessment System

Abstract

IMPACCT is an online survey covering over 40 self-report types of student communication competency, as well as a test of critical thinking based on cognitive problem-solving. The student nominates two peers who rate the student's interpersonal, computer-mediated, group and leadership, and public speaking communication competence. The student takes the self-report survey at Time 1 (T1), and also at a Time 2 (T2). The system generates a printable profile for the student displaying the following percentiles: (a) T1-Self (i.e., how the student sees self's communication skills in various skill domains at time 1), (b) T2-Self (i.e., how the student's scores changed from T1 to T2), (c) Peers (i.e., how self-rated skills compare to averaged ratings by two peers nominated by the student), and (d) Norms (i.e., how self-rated skills compare to everyone else's self-ratings who has taken the survey). The resulting collective data are available to the department to provide evidence of student self-perceived skill deficits at T1, as well as perceived change from T1 (e.g., beginning of course or major) to T2 (e.g., end of course or major), providing quantifiable data for assessment accounting and reporting. The system was tested on a basic course sample of 1,880 freshmen basic course students and 1,999 affiliated peer raters using a new measure of communication competence developed for this project. All subscales were sufficiently reliable. The self-reported motivation, knowledge, and skills constructs accounted for between 68–72% of student self-perceptions of overall communication competence (i.e., appropriateness, effectiveness, clarity, attractiveness, satisfaction). Critical thinking was unrelated, and peer ratings were only modestly related, to student self-perceptions of communication competence. Student self-perceived skills were systematically lower than peer ratings of students. Students consistently perceived that their communication competence and skills increased significantly over the span of the semester. The system is highly scalable and adaptable to multiple curricular configurations, and shows promise in providing a practical solution to departmental assessment needs. The measure also provides a valuable architecture for significant research and theory development in the area of communication competence.