Comparing Two Tests of Formal Reasoning in a College Chemistry Context

Abstract

The Test of Logical Thinking (TOLT) and the Group Assessment of Logical Thinking (GALT) are two of the instruments most widely used by science educators and researchers to measure students’ formal reasoning abilities. Based on Piaget’s cognitive development theory, formal thinking ability has been shown to be essential for student achievement in science and chemistry courses. The major difference between TOLT and GALT is two concrete items relating to conservation of mass and volume, which GALT contains over and above TOLT. The first intent of this study was to determine whether the two concrete items represent an advantage for the general chemistry population in terms of test reliability, discriminatory power, and item bias. Accordingly, TOLT and “TOLT + 2” (TOLT with the extra two concrete items) were each administered to a sample of over 900 students enrolled in first-semester general chemistry. The two concrete items were not observed to provide any advantage, and DIF analysis suggests that TOLT is tenably a less biased test. The second part of the study entailed a direct comparison between TOLT and GALT as intact instruments. GALT was found to have no advantage over TOLT in terms of reliability, discriminatory power, or potential item bias for either general chemistry or preparatory chemistry students.