Are psychological quantities and measurement relevant in the 21st century?

Abstract

Osborne (2010, p. 1) argues that as quantitative psychologists we must “keep challenging ourselves…and…continue questioning and examining our tacit assumptions”. Whilst I wholeheartedly endorse the critical and optimistic spirit of his article, I find it alarming that he only implicitly directs quantitative psychologists to question and examine their most fundamental tacit assumption. This is the claim, or rather, inadequately tested hypothesis that continuous, psychological quantities exist. Unless we “shrug off the shackles of 20th century methodology and thinking” (Osborne, 2010, p. 3) and seriously address this hypothesis, the relevance of quantitative psychology, and the “measurements” it provides will remain obscure in this century and beyond. The desire to replicate the physical sciences model in psychology has a long history, as do criticisms of this desire, which I cannot do justice in the present commentary (see Michell, 1999). The culmination of this history took place when Stevens (1946)proposed a new definition of measurement, as numerical application according to rule, which would have a profound effect upon the practice of Psychology, including the “explosive progress” that Osborne (2010, p. 2) refers to. But, at what cost has this “progress” come? The fact that Stevens’ definition endures in the mainstream even today is testament to the schism between quantitative psychology and quantitative science more generally. There is only one rule for measurement in quantitative science. This is, the estimation of a magnitude of a quantity by its ratio to a unit of the same quantity (Michell, 1999). This is not a trivial matter, as even Stevens (1946) acknowledged that everything from the calculation of the humble mean and t-test through to the sophisticated multivariate analyses that Osborne (2010) refers to are contingent upon this “kind of” measurement1. Thus, even though our data may be “hard-earned” (p. 3), if they are not derived from defensible measurement practices, then our toil simply leads to questionable inferences and continued scepticism from outside the discipline (Cliff, 1996). Thankfully, a number of quantitative psychologists have eschewed Stevens’ definition and proposed ways to measure psychological attributes in a manner that is consistent with the physical sciences (Luce and Tukey, 1964; Rasch, 1980; Michell, 1990). Axiomatic Conjoint Measurement theory and the class of Rasch models stand out from the crowd as potential tools to be used by psychologists to test their quantitative assumptions, establish standard units of measurement, and bring about defensible methodological practices (see Humphry and Andrich, 2008; Kyngdon, 2008). However, the minimal impact that these approaches have had on mainstream psychology over the past half-century does raise the question of whether they are sufficient or even necessary for addressing the question of psychological quantities. A more fundamental cognitive shift seems to be required whereby the application of such mathematical models is not seen as an end unto itself, as measurement never takes place in a theoretical vacuum. One only needs to take a cursory look at measurement practices in the physical sciences to realize that they are intrinsically linked to what has been described as a “body of workable, quantitative theories and laws” (Michell, 1999, p. 217). This means a body of substantive, quantitative theories and laws and not algebraic ideals. Thus, before we may defensibly apply such models, the burden is upon us as quantitative psychologists to delve into the applied literature to establish and experimentally investigate strong, quantitative psychological theories (Borsboom, 2006). Michell (2009) and Trendler (2009) provide examples of the necessary theoretical analysis and their conclusions are alarming. Firstly, Michell argues that many psychological attributes are necessarily not quantitative, as unlike established physical quantities, the differences between levels of these attributes are not mutually homogeneous. Furthermore, Trendler argues that quantification and measurement will never be possible in psychology, as we will never possess the necessary control over our subject matter. Whilst their arguments are compelling, it is pertinent to note that the systems of measurement in the physical sciences were developed over centuries and necessarily required a large amount of qualitative work to substantiate them (Kuhn, 1961). Thus, it seems premature to conclude that quantification and measurement are, in principle, not possible in psychology. However, this possibility can only be realized (or not) if we stop begging the quantity and measurement questions. So I accept Osborne's (2010) challenges and I add my own (on behalf of the authors cited from whom this commentary is derived). If it is to be shown that Quantitative Psychology is relevant to the 21st century and beyond, then we must provide empirical evidence for the existence of psychological quantities and then establish units by which they can be measured. Hopefully, given Osborne's editorial directive, some of the pages of this new and exciting journal will be dedicated to this most fundamental challenge.