Assessing Student Development of Scientific Thinking Skills using the EDAT and ADAT: The Experimental Design and Analysis of Data Ability Tests

Abstract

A significant challenge to measuring undergraduate students’ science thinking skills is defining science reasoning and specifically isolating the various overlapping skills. These include quantitative literacy/numeracy, visual literacy, and the sub skills required for experimental design and data analysis. To this end, we have designed and used the Experimental Design Ability Test (EDAT) and are developing a companion Analysis of Data Ability Test (ADAT). The EDAT is an open‐ended prompt used to reveal students’ ability to design a simple experiment to test a product claim (Sirum and Humburg, 2011). Student responses are scored by looking for the presence or absence of ten basic elements of experimental design. We have recently expanded the scoring rubric of the EDAT to begin to describe the different ways an expert might include these basic experimental design elements versus a novice's approach. The ADAT is a new open‐ended response instrument designed to minimize the threshold requirement for quantitative and visual literacy skills, while specifically assessing students’ reasoning with data. Similar to findings using the EDAT, the ADAT also reveals that up to 85% of undergraduate biology students at all levels have difficulty controlling variables in experimental situations. In addition, similar to previous research by others, the ADAT also reveals that even upper level students have deficiencies in fundamental correlational and proportional reasoning skills when asked to interpret a relatively simple data table (Lawson, 1978).