A Functional Item Analysis of Group Performance in General Mathematics

Abstract

This paper proposes to summarize both the I procedure used and the results obtained from the use of a functional item analysis of the re sponses made to the items of the Cornell Math ematics test by 416 freshmen entering the New York State College of Agriculture at Cornell University in September, 1948. The Cornell General Mathematics Test was developed at Cornell University as a test of proficiency in general mathematics and contains forty items arranged in an increasing order of difficulty. It is divided into two parts. Part I is designed to measure ability in fundamental operations, while Part U tests ability in prob lem solving. With but two exceptions, all the items can be solved by a student proficient in arithmetic and elementary algebra. In its present form, the test embodies the results of one revision of the original based up on a previous item analysis which resulted in the elimination of some non-discriminating items. The test is objective in nature and uses the principle of multiple choice with four alter native answers to each item. It is given annu ally to entering freshmen and the results are used for purposes of counseling, sectioning, and research. The test was administered with out time limit to the group from which the ac companying data were compiled. The data here presented are based upon the following procedures. On the basis of scores on the test as a whole, the entire group was ranked and divided into quintiles. An I. B. M. graphic item count was then made for each quintile group, showing the number of respon ses to each alternative answer to each question on the test. From this information, it was then possible to tabulate for each group the exact number of correct and incorrect responses to each test item. From this tabulation it was readily observable that for most of the test it ems, the number of correct answers varied directly in relation to the rank of the group be ing studied. This is especially to be expected since the test had already been subjected to one item analysis for the express purpose of in creasing its internal consistency and hence its validity. For the purposes of this study, the data from only three, the low, middle, and high, groups of the original five groups were used, and a new comparative factor was introduced. I A careful analysis was made of each item in the test for the purpose of determining the fund amental operation or operations involved in its solution. As a result of this analysis of the forty test items, twenty fundamental operations were identified and each test item was listed under the operation or operations which were judged to be primarily involved in its solution. Whenever possible an item was classified un der the one fundamental operation involved. This was possible for nineteen items. When this could not be done, and when two operations seemed to be equally involved, the item was s o classified. This was done in eighteen cases. It was necessary to classify two items under three different operations and one under four opera tions. A miscellaneous classification was set up to take care of three problems involving op erations which did not readily fit into other classifications and for which new classifications did not seem warranted. The next step was to tabulate for each of the three quintile groups the right and wrong ans wers to classified groups of test items, thus making it possible to determine the correct and incorrect responses, by quintile groups, to the test items classified under a particular fun damental operation. Performance by ranked groups on functionally classified test items could then be observed. It could now be deter mined on which types of problems performance was best and where it was poorest. It was still possible to compare groups, and by studying performance on individual problems under dif ferent classifications, it was possible to form many strong inferences as to the exact location of specific difficulties. These data are shown in the accompanying graph (Figure 1) in which all numerals repre sent percentages of correct answers either to classified groups of test items, or in a few in stances, to the test as a whole. It is here pos sible to observe general performance, group performance, and performance by groups in re spect to fundamental operations in general math ematics.