Prescribing Structure for Validation Arguments: Elemental, Structural, and Ecological Validity

Abstract

ABSTRACTContingent argument-based approaches to validity require a unique argument for each use, in contrast to more prescriptive approaches that identify the common kinds of validity evidence researchers should consider for every use. In this article, we evaluate our use of an approach that is both prescriptive and argument-based to develop a validity argument for a novel instrument.The elemental aspect concerns the items; the structural aspect concerns how the items function together; and the ecological aspect concerns how the resulting instrument is related to the context of use. Elemental, structural, and ecological aspects of validity are considered independently. We illustrate this approach by describing our experience constructing a validity argument for a novel, integrated knowledge and motivation instrument for use in teacher education. For each aspect, we illustrate the approach by discussing theory-based assumptions, articulating inferences based on these assumptions, and describing how we collected and assessed empirical evidence to support the inferences. We conclude with a retrospective analysis resulting in lessons learned for others who may consider a prescriptive, argument-based approach to validity.