The Relative Effectiveness of Diagnostic and Positive Test Strategies is Logically Indeterminable

Abstract

Research Into the cognitive processes underlying inductive inference has not only yielded valuable data on the ways in which people go about testing hypotheses; it has also raised important questions as to what constitutes a normative test. One widely accepted assumption regarding test normativeness is the idea that diagnostic test strategy (the method of conducting crucial tests to distinguish between competing hypotheses) is generally superior to positive test strategy (the method of testing for the occurrence of events predicted by a single hypothesis). This seemingly obvious assumption, however, fails to stand up to close inspection. The relative effectiveness of positive and diagnostic test strategies varies with a factor that is, even in principle, unknowable. Specifically, test effectiveness depends upon the logical relationship between the sets of events predicted by the hypotheses under investigation and the set of events that logically follows from the truth; and the tester never knows what this relationship is. Considerations of a test’s potential to falsify, including Bayesian considerations of the probability of predicted events given that a particular hypothesis is not true, are essential to optimal hypothesis testing. However, the construction and consideration of specified alternatives prior to testing is not essential.