Reliability of the Critical Thinking in Communication Science and Disorders: A Content-Specific Critical Thinking Assessment

Abstract

Introduction: Students and professionals in communication sciences and disorders (CSD) need to exhibit good critical thinking (CT) skills when engaged in clinical tasks. CSD clinicians must make decisions that are free from biases and support their claim with facts. Thus, CSD clinicians need to be trained to question their clinical practices and to skeptically evaluate new practices that develop. A content-specific CT test can help determine if students are developing these skills. However, to date, no such content-specific CT assessment exists for CSD. The purpose of this study was to determine the reliability of the current version of a specific content CT assessment, the Critical Thinking in Communication Sciences and Disorders (CTCSD). Methods: A sample of 150 CSD graduate students enrolled in three programs participated. They completed an online Qualtrics survey that consisted of the CTCSD. They completed the Qualtrics survey twice, once at the beginning of a semester and once at the end. The participant responses were independently scored by two research associates. The data were analyzed for reliability in three ways. Intra-subject reliability was assessed by comparing scores across the two testing sessions. Internal consistency of the items to measure a common construct was assessed using Cronbach’s alpha and Guttman’s Lambda 6. Inter-rater reliability was assessed using Cohen’s Kappa coefficient. In addition, the time used to complete the survey was analyzed. Results: The students from the three programs scored similarly on the CTCSD. High reliability ratings occurred for the intra-subject, internal consistency, and inter-rater measures. Discussion/Conclusion: The results indicate the reliability of the CTCSD. In combination with previous results indicating the face, construct, and criterion validity of the CTCSD, it appears to have psychometric strength. The CTCSD may help academic and clinical faculty select learning activities and focus feedback to their graduate students in order to reinforce skills the students exhibit and to develop other skills.