Arbitrating norms for reasoning tasks

Abstract

The psychology of reasoning uses norms to categorize responses to reasoning tasks as correct or incorrect in order to interpret the responses and compare them across reasoning tasks. This raises the arbitration problem: any number of norms can be used to evaluate the responses to any reasoning task and there doesn’t seem to be a principled way to arbitrate among them. Elqayam and Evans have argued that this problem is insoluble, so they call for the psychology of reasoning to dispense with norms entirely. Alternatively, Stupple and Ball have argued that norms must be used, but the arbitration problem should be solved by favouring norms that are sensitive to the context, constraints, and goals of human reasoning. In this paper, I argue that the design of reasoning tasks requires the selection of norms that are indifferent to the factors that influence human responses to the tasks—which aren’t knowable during the task design phase, before the task has been given to human subjects. Moreover, I argue that the arbitration problem is easily dissolved: any well-designed task will contain instructions that implicitly or explicitly specify a single determinate norm, which specifies what would count as a solution to the task—independently of the context, constraints, and goals of human reasoning. Finally, I argue that discouraging the use of these a priori task norms may impair the design of novel reasoning tasks.