What makes some problems really hard: Explorations in the problem space of difficulty

Abstract

This paper identifies two sources, one larger, one smaller, of the great difficulty encountered by subjects solving the Chinese Ring puzzle. Almost none of our subjects were able to solve the puzzle within 2 h unless they were given a demonstration of how to move, and even with that help only half of the subjects obtained solutions. Discovering how to make moves, rather than other features of the problem search space, was the source of its inordinate difficulty. Evidence for this comes from isomorphs that were designed to “digitize” the moves, which in the original version have analog qualities. These digital isomorphs were solvable by almost all subjects, with average solution times of 10 to 25 min, depending on isomorph type. The sense in which these altered problems were isomorphic to the original ones is explored. The isomorphism is defined in terms of an external search space consisting of a graph of nodes which represent states of knowledge, and links between those states which represent legal moves. Subjects' representations of the more difficult problems evolve toward isomorphism as they discover what constitutes a move. The digitized problems still required considerable effort for their solution. The difficulty of these problems in digital form is particularly surprising, given that the problem search space is linear: there is no branching. Hence, size of search space (exponential explosion) was not the source of difficulty here. The linearity of the search space did not prevent the subjects from making a large number of moves in reaching a solution. The average number of moves ranged from 150 to 450 for different isomorphs. In addition, the subjects' move behavior was often dichotomous, consisting of a very large number of nonprogressive, often error-prone moves, followed by a very rapid, often error-free movement to the goal. In our examination of the sources of difficulty for this problem, we also studied the transfer of skill between different isomorphs. The investigation of transfer-of-training showed that problem-representational features such as move-operator compatibility, move difficulty, and the presence or absence of move legality cues determined the amount of transfer.