How Do I Know My Data? Let Me Count the Ways

Abstract

Rather than resolving the paradoxes created by the preceding two papers, this article points out why different researchers, applying comparable methodological procedures to the same data, can sometimes come to conflicting conclusions. In a general sense, data are collected to gain theoretical understanding and to make decisions. Theory development involves testing hypotheses about causal relations and estimating parameters for specific models. Decision making also involves creating models. These models, however, serve to transform available information on a particular case to a form that will indicate which of a set of choices applies to the case. The two approaches are easily confused because of their many interrelations. A sound theoretical model, for example, cannot be developed without good decision models (for measurement). A good theoretical model can usually be applied for decision purposes (at a policy level). The development of both kinds of models depends on the existence of nonzero correlations among variables. The development of both theoretical and decision models may involve the same methodological procedure, like regression analysis. What most distinguishes decision models is just the secondary importance of theory in them. A nonzero correlation is the ultimate piece of information that allows one to transmute information about one variable into a categorization with respect to another variable. This, therefore, is the only information of consequence in decision models. Presumably, there is a reason why two variables are correlated, but one does not need to know the reason in order to translate information. The problem of measuring structure, for example, is a decision problem in the sense that it must be decided whether the organization of activities is simple, complex, or at a level in between. As indicators of structure, number of different positions, number of organizational levels, and even organization size might be used (providing it was not intended to study the relation of structure and size later). From a decision-making standpoint, it does not matter whether structure determines these variables, whether they determine structure, or whether they all are correlated spuriously through the action of other variables. It is enough that, given information on the indicator variables, the picture of an organization can be completed by making an estimate on the dependent variable of structure. In a sense, decision models are raw-empiricist mechanisms for filling in information that is missing because it is too expensive to collect, it cannot be obtained by any direct means, or because the information depends on events that have not yet occurred. The Aston group state they are applying correlation and regression analyses without making presumptions about causal dependencies. This implies that they are intent on developing decision models. Thus, if one has no information on structure, their results indicate an approximate classification can be made in terms of size and technology, and size should be weighted much more in the transformation of data than technology, since size is a better indicator of structure. This is a legitimate decision model, and it is essentially free of causal assumptions. It is crucial to realize, however, that since such a decision model was developed without theoretical constraints, it provides little basis for making inferences about the patterns of causality. In particular, it yields no information about the directions of effects, and it does not even necessarily imply that the causal linkage between technology and structure is of less magnitude than that between size and structure (since suppressor variables may be acting within the system). Thus, one way different researchers can ar-