Verbal Reports as Data for Cognitive Diagnostic Assessment

Abstract

The term cognitive diagnostic assessment (CDA) is used in this chapter to refer to a specific type of student evaluation. Unlike classroom-based tests designed by teachers or large-scale assessments designed by test developers to measure how much an examinee knows about a subject domain, CDAs are designed to measure the specific knowledge structures (e.g., distributive rule in mathematics) and processing skills (e.g., applying the distributive rule in appropriate mathematical contexts) an examinee has acquired. The type of information provided by results from a CDA should answer questions such as the following: Does the examinee know the content material well? Does the examinee have any misconceptions? Does the examinee show strengths for some knowledge and skills but not others? The objective of CDAs, then, is to inform stakeholders of examinees' learning by pinpointing the location where the examinee might have specific problem-solving weaknesses that could lead to difficulties in learning. To serve this objective, CDAs are normally informed by empirical investigations of how examinees understand, conceptualize, reason, and solve problems in content domains (Frederiksen, Glaser, Lesgold, & Shafto, 1990; Nichols, 1994; Nichols, Chipman, & Brennan, 1995). In this chapter, we focus on two methods for making sense of empirical investigations of how examinees understand, conceptualize, reason, and solve problems in content domains. As a way of introduction, we first briefly discuss the importance of CDAs for providing information about examinees' strengths and weaknesses, including the ways in which CDAs differ from traditional classroom-based tests and large-scale tests.