Information Selection and Utilization in Hypothesis Testing: A Comparison of Process-Tracing and Structural Analysis Techniques

Abstract

This paper examines the contributions of process-tracing and structural analysis techniques to our understanding of the cognitive processes underlying hypothesis testing. An experiment designed to compare and contrast the techniques is reported in which participants (N= 72) made diagnostic judgments by selecting information they felt was necessary from computerized information-boards (Mouselab 4.2: Johnson et al., 1989). This innovative methodology enabled us to explore the potential of a multimethod approach in providing insights into how people's understanding of the diagnostic value of conditional probability information in hypothesis testing deviated from the prescriptions of Bayes' theorem. In short, the process-tracing results indicated that, as well as typically selecting information according to a strategy which did not fit the Bayesian prediction, participants generally paid most attention to information directly pertinent to the hypothesis or hypotheses they were explicitly asked to evaluate. Although the structural analysis also implied that participants' integration strategies were not strictly Bayesian, direct comparisons between the data sources highlighted possible dangers associated with relying on each technique in isolation. Most significantly, by synthesizing data between techniques the value of adopting combined process-tracing/structural analysis approaches in decision research was demonstrated.