A Computational Theory of Complex Problem Solving Using Latent Semantic Analysis

Abstract

A Computational Theory of Complex Problem Solving Using Latent Semantic Analysis Jose Quesada, Walter Kintsch ([quesadaj, wkintsch]@psych.colorado.edu) Institute of Cognitive Science, University of Colorado, Boulder Boulder, CO 80309-0344 USA Emilio Gomez (egomez@ugr.es) Department of Experimental Psychology University of Granada Campus cartuja, S/N, Granada, Spain Abstract Complex Problem Solving (CPS) is a hybrid between field studies and experimental studies. This paper introduces a new, abstract conceptualization of microworlds research based on two innovations: (1) a problem representation, which treats protocols as objects in a feature space and, (2) a similarity metric which is defined in this problem space. Latent Semantic Analysis (LSA) is used to analyze performance in CPS, using actions or states as units instead of words and trials instead of text passages. Basic examples of applications are provided, and advantages and limitations are discussed. Introduction Many real-word decision making and problem solving situations are (1) dynamic, because early actions determine the environment in which subsequent decision must be made, and features of the task environment may change independently of the solver’s actions; (2) time-dependent, because decisions must be made at the correct moment in relation to environmental demands; and (3) complex, in the sense that most variables are not related to each other in one-to-one manner. In these situations, the problem requires not a single decision, but a long series of decisions which are dependent on one another. For a task that is changing continuously, the same action can be successful at moment t1 and useless at moment t2. However, traditional, experimental problem solving research has focused largely on tasks such as anagrams, concept identification, puzzles, etc. that are not representative of the features described above. In Europe, researchers led by Broadbent (e.g., Broadbent, 1977) in the UK and Dorner (e.g., Dorner, 1975) in Germany, were concerned about that fact and started working on a set of computer-based, experimental tasks that are dynamic, time-dependent, and complex, called Microworlds 1 . The study of microworlds is an example of Complex Problem Solving (e.g., French & Funke, 1995). This term sometimes has other meanings. For example, educational applications created to teach physics (Henderson, Klemes, & Eshet, 2000), simulated words in the early AI programs like the block word of SHRDLU, (Winograd, 1972) and static tasks to study decision making (Green, 2001) have been called Compared to traditional Problem Solving, Complex Problem Solving (CPS) radically changed the kind of phenomena reported, the kind of explanations looked for, and even the kind of data that were generated. However, the results obtained to date are far from being integrated and consolidated. This fact led Funke to affirm that ‘Despite 10 years of research in the area, there is neither a clearly formulated specific theory nor is there an agreement on how to proceed with respect to the research philosophy. Even worse, no stable phenomena have been observed’ (Funke, 1992, p. 25). Almost another 10 years after Funke’s argument, although more empirical research has been conducted in the area, we cannot say that the situation has changed drastically. At this moment, there is no theory able to explain even part of the specific effects that have been described or how they can be generalized. A theory of generalization and similarity is as necessary to psychology as Newton's laws are to physics (Shepard, 1987). However, for CPS there is no common, explicit theory to explain why a complex, dynamic situation is similar to any other situation or how two slices of performance taken from a problem solving task can possibly be compared quantitatively (Klein, Orasanu, Calderwood, & Zsambok, 1993). This lack of formalized, analytical models is slowing down the development of theory in the field. At least two problems make it difficult to apply the classical problem solving approach to CPS, one theoretical and one methodological: (1) The utility of state space representation for tasks with inner dynamics is reduced because in most CPS environments it is not possible to undo the actions. For example, imagine that two participants in Firechief (see below) are in an identical situation (system state) when the trial starts. One of them proceeds to make a control fire on the east side of a fire, while the other one is preparing a control fire on the north front of the fire. After these actions, the system state is no longer identical for them. Now they have to cope with rather different problems. Moreover, if the first participant wants to apply the same technique that Microworlds. However, we are concerned here only with tasks that fulfill the conditions described above.