Learning Potential in Analogical Reasoning: Construction of a Rasch-scaled Instrument for Group Wise Assessment

Abstract

The aim of this project is to construct a new learning potential test in analogical reasoning that can be administered both individually and group wise. A first set of items has been created within the framework of Berger’s Master’s thesis (Berger; 2003). In this study (Bosson, 2003), these items were revised and applied in a primary school with 271 children, age between 6 and 13 years. The HART (Hessels Analogical Reasoning Test; Hessels, 2003) is composed of 66 analogical matrices presented in 2 (columns) by 3 (rows) or 3 by 3 format with 6 or 8 response alternatives. Items are distributed across 9 levels of complexity based on number of attributes and relations in the matrices. The test is preceded by an introductory phase, during which four examples are discussed extensively with the children. In this study, the HART is presented in three phases. After the introductory phase, each child had to individually resolve 40 to 60 items (pretest). Then a short training (20-30 minutes) about how to resolve analogical matrices was given to whole classes, followed by a posttest in which the child had to resolved another 20 items (posttest). For a first evaluation of training effects and validity of the HART, the 14 classes were randomly assigned to either the control or the experimental condition. Furthermore, the teachers were asked to give ratings of pupils’ school performance in arithmetic, French, behavior, participation, and application in class.Analysis of pretest results permitted us to construct a generalized Rasch-scale with all 66 items. The theoretical complexity levels of the items largely concurred with their empirical difficulties. These results will allow us, in future research, to construct short tests adapted to the developmental level of each child. With regard to the validity, the low correlations between pretest, posttest and non-cognitive aspects (measured with teacher’s ratings) demonstrated that the test provides a relatively independent evaluation of factors such as behavior and application in class. The effect of training was significant: participants from the experimental group showed higher performance at posttest than those of the control group. Furthermore, the gains realized by the experimental group were not related to age, as all age groups showed to have benefited from the training to the same extent. The residual gains from pre- to posttest showed also to be significantly related to school success in the experimental group only.The results of this study show that the experimental version of the HART allows one to estimate the learning potential of children in mainstream primary education classes. The research will continue with children in special education to evaluate the differential effects of the training and the predictive capacity of the HART, as well as the merits of different testing procedures.