Abstract
We conducted two studies to replicate and extend, as well as test, the limits of previous findings regarding an apparent disconnect between scientific-reasoning skills in psychological science, on the one hand, and scores on standardized tests of general intelligence, on the other. In Study 1, we examined whether this disconnect would extend beyond psychological science to additional sciences as well, such as nutrition and agriculture. The results did indeed extend, suggesting that scientific reasoning across various natural sciences is comparable to scientific reasoning in psychological science, but different in kind from the reasoning required on conventional standardized tests. In Study 2, we examined whether these findings were linked to the format of presentation of scientific problems. Whereas real scientific-reasoning problems are open-ended, standardized tests tend to use multiple-choice format. We discovered that using multiple-choice format did indeed result in an apparently closer relation of the scientific-reasoning tests to two of the conventional ability measures (SAT Reading and Number Series) but not to two other tests (Letter Sets and SAT Math). Thus, one can increase the correlations of scientific-reasoning tests with at least some standardized ability tests but at the cost of content validity and ecological validity.
Highlights
Graduate schools use a variety of criteria to determine whom to admit
The results suggest that scientific reasoning of the kinds we have assessed in this experiment, whether for psychological science or science in general, measure skills other than those measured by tests of fluid and crystallized abilities
For the free-response items, participants were asked to explain whether or not the conclusions drawn from the results were valid, as in Study 1
Summary
Graduate schools use a variety of criteria to determine whom to admit. But graduate admissions in scientific fields, and in many other fields as well, rely, often quite substantially, on scores of candidates on the Graduate Record Examination (GRE) [1]. Some researchers have argued that tests such as the GRE (as well as the SAT, the ACT, and other related tests) assess a relatively narrow band of analytical-thinking skills [2,3,4], such as inferential reasoning, similar to those on IQ tests. Tests like the ACT, SAT, and GRE provide reasonably good measures of general cognitive ability [5,6,7], but they do not measure the research and teaching skills that are relevant to success in science [8,9]. Investigators have sought to expand the range of assessment for graduate admissions in psychological science and related disciplines, see References [8,9]. Sternberg and Williams [10]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
