An Interview Test of Critical Thinking

Abstract

THE ABILITY to think critically is generally I considered to be one of the skills teachers should attempt to develop in their students. The desira bility of this skill is reflected in recommenda tions of such authoritative bodies as the Educa tional Policies Commission, the Harvard Fac ulty, and the American Council on Education (1). There have been many attempts to measure the ability to think critically. One of the first major attempts resulted in the construction of the Watson-Glazer Test of Critical Thinking, which at the present time appears to be the major meas urement approach utilized in studies of critical thinking. A more recent test developed by Ennis (2) is intended to obviate some of the limitations inherent in the Watson-Glazer. A problem of interest both to educators and psychologists is the extent to which critical thinking skills manifest themselves beyond the paper and pencil test-taking situation. To study this question, Reilly (4) developed a test designed to measure a student's ability to apply knowledge in circumstances not directly learned in the class room situation. A science scrapbook was prepared containing 40 newspaper clippings matched in content or idea to 40 items on a previously given physics test. The students were asked to agree or disagree with the accuracy of the newspaper clippings. There have been other devices of a test nature somewhat similar to this (3, 1). It will be noted, however, that these attempts, while less obviously purporting to measure criitical think ing, still are limited by the paper and pencil for mat, a limitation which would appear to be rather important. This paper reports an attempt to evaluate criti cal thinking through a structured interview with the student. This approach appears to have the following merits: First, it is not identified as a test and in particular is not identified as directly related to critical thinking or reasoning but merely seeks to examine the student's opinions and the reasons for the opinions which he holds on a particular topic. Second, the topic about which opinions are sought is one which is believed to be of some interest and importance generally and particularly to high school students so that | one might have some confidence in generalizing to other aspects of their every-day life. The interview consists of 12 items pertaining to the topic of sleep learning of which ten are scored. The 12 items along with samples of the scoring guides for each are presented at the end of this article. The interview was administered to 59 high-school juniors and recorded on tape. Each interview took approximately five minutes. The scoring procedure was developed by hav ing three individuals, all with background in logic or critical thinking, independently score a sample of 70 responses. Using a six point scale the raters agreed within a three point range on all responses except four. On these four items two of the judges discussed the items and arrived at the final assignment. The remaining item scores were determined by averaging the judg ments of the three raters. Following this the pro tocols of each of the subjects were scored using the examples so derived. The score for each indi vidual was a summation of his scores on the 12 items. The objectivity of the scoring system was evaluated by having two independent scorers score 47 interviews. A correlation of .85 was at tained between the two sets of scores. The split-half reliability adjusted by the Spearman-Brown formula was found to be .64. In addition to the logical validity implicit in the development of the scoring system, the following validity data were obtained. In each case the standard error of r is .16 or smaller. The correla tion with the Watson-Glazer Test of Critical Thinking was .58. The correlation with the I.D.S. Test of Critical Thinking was .54. The correla tion with teachers' ratings of critical thinking based on one year's experience with the same stu dents during which a concerted attempt to teach critical thinking was integrated into the Ameri can history course resulted in a correlation of .60. An additional attempt at validation consisted of comparing the C.I. scores of one group of students prior to an instructional period in criti cal thinking with the scores of a second group of students who had completed the course work in critical thinking, the expectation being that the scores of the students having completed the