Judgment Strategies Research Articles

The relation between metastrategic knowledge, strategy use and task performance: Findings and reflections from a numerosity judgement task

In the research literature several positions concerning the role played by metacognition in adaptive strategy choice can be distinguished. While many authors adhere so-called metacognitive models of strategy choice and strategy change, others have questioned the extent to which metacognitive factors are associated with strategy choice and task performance and have proposed alternative theoretical frameworks wherein strategy choices are described in terms of associative models. In the present article we report data coming from a larger research project on the development of children’s numerosity judgement strategies and skills. The experimental task involved judging numerosities of colored blocks presented in a rectangular grid. Participants were 59 second grade and 50 sixth grade children, whose strategic performance data — obtained by means of a systematic analysis of their response-time patterns — were compared with interview data collected at the end of the experiment. The major result of this comparison is that not only the children from the oldest age group, but also the children from the youngest age group showed clear evidence of metacognitive awareness and understanding of different aspects of their strategic performance.

The Use of Cognitive Feedback by Experts and Novices in a Judgment Task

Cognitive feedback (CFB) has been shown to increase performance in Multiple Cue Probability Learning (MCPL) tasks in both real life and experimental situations. Our experiment focuses on investigating how experts and novices behave differently in applying CFB, which is presented in forms of relative importance weights of each of the cues and the display of function forms (Weaver and Steward, 2000). We believe this study provided some interesting insight into how much domain knowledge is necessary to understand and apply CFB. The study examined how well novices and experts are able to process and use different types of feedback (i.e. CFB and Outcome Feedback) and hence determined which type is more useful for novices or for those with prior knowledge of the task. A total of 20 participants, 10 experts and 10 novices, were given a series of baseball statistics and asked to predict how many wins the team would have in a season based on these statistics. Performance measures (e.g. relative weighting accuracy, Lens Model measures), mental workload and judgment strategy were measured. Domain Knowledge was statistically significant for all weights except one less important cue, “stolen bases.” In other words, domain knowledge impacts weighting accuracy greatly. Domain knowledge was also found to have a significant effect on judgment performance (i.e. Lens Model measures), such as accuracy and knowledge. Mental workload was significantly affected by CFB, but none of the other dependent variables were.

Not so fast! (and not so frugal!): rethinking the recognition heuristic

The ‘fast and frugal’ approach to reasoning (Gigerenzer, G., & Todd, P. M. (1999). Simple heuristics that make us smart. New York: Oxford University Press) claims that individuals use non-compensatory strategies in judgment – the idea that only one cue is taken into account in reasoning. The simplest and most important of these heuristics postulates that judgment sometimes relies solely on recognition. However, the studies that have investigated usage of the recognition heuristic have confounded recognition with other cues that could also lead to similar judgments. This paper tests whether mere recognition is actually driving the findings in support of the recognition heuristic. Two studies provide evidence that judgments do not conform to the recognition heuristic when these confounds are accounted for. Implications for the study of simple heuristics are discussed.

Instance‐based learning in dynamic decision making

AbstractThis paper presents a learning theory pertinent to dynamic decision making (DDM) called instancebased learning theory (IBLT). IBLT proposes five learning mechanisms in the context of a decision‐making process: instance‐based knowledge, recognition‐based retrieval, adaptive strategies, necessity‐based choice, and feedback updates. IBLT suggests in DDM people learn with the accumulation and refinement of instances, containing the decision‐making situation, action, and utility of decisions. As decision makers interact with a dynamic task, they recognize a situation according to its similarity to past instances, adapt their judgment strategies from heuristic‐based to instance‐based, and refine the accumulated knowledge according to feedback on the result of their actions. The IBLT's learning mechanisms have been implemented in an ACT‐R cognitive model. Through a series of experiments, this paper shows how the IBLT's learning mechanisms closely approximate the relative trend magnitude and performance of human data. Although the cognitive model is bounded within the context of a dynamic task, the IBLT is a general theory of decision making applicable to other dynamic environments.

Instance-based learning in dynamic decision making

This paper presents a learning theory pertinent to dynamic decision making (DDM) called instance-based learning theory (IBLT). IBLT proposes five learning mechanisms in the context of a decision-making process: instance-based knowledge, recognition-based retrieval, adaptive strategies, necessity-based choice, and feedback updates. IBLT suggests in DDM people learn with the accumulation and refinement of instances, containing the decision-making situation, action, and utility of decisions. As decision makers interact with a dynamic task, they recognize a situation according to its similarity to past instances, adapt their judgment strategies from heuristic-based to instance-based, and refine the accumulated knowledge according to feedback on the result of their actions. The IBLT’s learning mechanisms have been implemented in an ACT-R cognitive model. Through a series of experiments, this paper shows how the IBLT’s learning mechanisms closely approximate the relative trend magnitude and performance of human data. Although the cognitive model is bounded within the context of a dynamic task, the IBLT is a general theory of decision making applicable to other dynamic environments.

Explicit and implicit processes in multicue judgment.

In two experiments, a multicue probability learning task was used to train participants in relating judgments to a criterion, on the basis of several cues that could or could not be relevant. The outcome feedback had 25% added noise to simulate real-world experience-based learning. Judgmental strategies acquired were measured by individual multiple linear regression analyses of a test phase (with no feedback) and were compared with self-ratings of cue relevance. In a third experiment, participants were instructed explicitly on cue relevance, with no training phase. The pattern of results suggested that both implicit and explicit cognitive processes influenced judgments and that they may have been sensitive to different task manipulations in the learning phase. On more complex tasks, despite weak explicit learning, explicit processes continued to influence judgments, producing a decrement in performance. These findings explain why studies of expert judgment often show only moderate levels of self-insight, since people have only partial access to the processes determining their judgments.

Specificity of Malingering Detection Strategies in Older Adults Using the CVLT and WCST

Strategies for the detection of possible malingering have largely taken two forms. First is the development and validation of domain-specific measures of malingering designed specifically for the detection of malingering (e.g., Test Of Memory Malingering, Word Memory Test, and Word Completion Memory Test). The second has been the development and evaluation of performance patterns and cutoff scores applicable to commonly used neuropsychological instruments. Two such instruments that have been examined are the Wisconsin Card Sorting Test (WCST) and the California Verbal Learning Test (CVLT). While several studies propose possible cutoff scores and actuarial judgement strategies for these tests, the specificity of these strategies within older, nonimpaired adults has not been established. Without proper evaluation of potential age-related effects within these strategies, the actual utility of the strategies within suspected malingerers who are older adults is unknown. Therefore, the present study was conducted to evaluate the specificity of the proposed strategies for detecting possible malingering with the WCST and the CVLT in a sample of older, community-dwelling, nonimpaired adults. The results suggest that the currently existing WCST formulas may have limited utility for the detection of malingering with older adults while the CVLT strategies do appear to have potential clinical utility. The potential of these formulas for use with older adults is discussed.

Mathematical modeling of decision making: a soft and fuzzy approach to capturing hard decisions.

This research focuses on a modeling approach and set of mathematical tools that were derived from research on intelligence systems, namely fuzzy system modeling. This study systematically evaluates these tools as an approach for modeling human decision making, contrasting the approach with more traditional methods based on regression. The research was conducted using experts and a simulated task environment related to allocating rewards in the form of merit pay. The results indicate that fuzzy system models generally perform as well as or better than both linear and nonlinear regression methods in terms of model fit. These results are discussed in terms of issues regarding modeling precision versus parsimony, the value of adaptive modeling techniques, empirical versus subjective approaches to model building, and individual differences in judgment strategies. Potential applications of this research include using the modeling approach studied to build higher-fidelity models that yield new insights and a better understanding of decision-making strategies and environments.

Adapting Strategy Choices to Situational Factors: The Effect of Time Pressure on Children’s Numerosity Judgement Strategies

The present study investigated the effect of time pressure on 6th graders' (11-12 years old) numerosity judgement strategies under three different time pressure conditions. Time pressure was manipulated by varying the maximum presentation time of the stimuli on three different levels: 5 s in the severe, 10 s in the moderate, and 20 s in the low time pressure condition. Analysis of the results in terms of the theoretical framework of Lemaire and Siegler (1995) revealed an effect of increasing time pressure on several parameters of strategy use such as participants' strategy repertoire, relative frequency of strategy use and efficiency of strategy execution. Unexpectedly, we did not observe a significant decline in the adaptiveness of strategy choices as a function of item characteristics. Taken as a whole, these results indicate that children are already at a young age able to adapt their strategy use to the external demands of a task.

Flexibility in strategy use: Adaptation of numerosity judgement strategies to task characteristics

In previous investigations we documented that people use several strategies to determine numerosities of blocks that are presented in a square grid. One of these strategies is the clever subtraction strategy, wherein the number of empty squares in the grid is subtracted from the total number of squares in the grid. In the present study we investigated participants' flexibility in strategy use when varying the size of the grids. Results are described in terms of the theoretical framework of Lemaire and Siegler (1995) regarding strategic change, and show that this contextual variable affected the frequency, execution time, and accuracy of subjects' use of the subtraction strategy. The usefulness of this framework for analysing the nature of the adaptation to contextual variations is discussed. From a methodological point of view, this study documents the potential of Beem's (1993, 1999) segmented linear regression models for assessing subjects' strategy use in cognitive tasks.

Can Examinees Use Judgments of Item Difficulty to Improve Proficiency Estimates on Computerized Adaptive Vocabulary Tests?

Recent simulation studies indicate that there are occasions when examinees can use judgments of relative item difficulty to obtain positively biased proficiency estimates on computerized adaptive tests (CATs) that permit item review and answer change. Our purpose in the study reported here was to evaluate examinees' success in using these strategies while taking CATs in a live testing setting. We taught examinees two item difficulty judgment strategies designed to increase proficiency estimates. Examinees who were taught each strategy and examinees who were taught neither strategy were assigned at random to complete vocabulary CATs under conditions in which review was allowed after completing all items and when review was allowed only within successive blocks of items. We found that proficiency estimate changes following review were significantly higher in the regular review conditions than in the strategy conditions. Failure to obtain systematically higher scores in the strategy conditions was due in large part to errors examinees made in judging the relative difficulty of CAT items.

Aesthetic judgments of novel graphic patterns: analyses of individual judgments.

Aesthetic judgments were investigated using a combined nomothetic and idiographic approach. Participants judged novel graphic patterns with respect to their own personal definitions of "beauty." Judgment analysis was employed to derive individual case models of judgment strategies as well as a group model. As predicted, symmetry had the highest correlations with aesthetic judgments of beauty. Stimulus complexity was the second-highest correlate of a positive evaluation. Thus, there was agreement at the group level. The judgment analyses, however, indicated substantial individual differences. These included use of symmetry or complexity cues that were contrary to the main group use, e.g., a few participants considered nonsymmetric patterns more beautiful. These findings suggest that exclusive consideration of the group model would have leveled the individual differences and been misleading. The group model is significant; however, the individual judgment analyses represent individual patterns of judgment in a notedly more accurate way.

Momentary and integrative response strategies in causal judgment.

Associative models of causal learning predict recency effects. Judgments at the end of a trial series should be strongly biased by recently presented information. Prior research, however, presents a contrasting picture of human performance. López, Shanks, Almaraz, and Fernández (1998) observed recency, whereas Dennis and Ahn (2001) found the opposite, primacy. Here we replicate both of these effects and provide an explanation for this paradox. Four experiments show that the effect of trial order on judgments is a function of judgment frequency, where incremental judgments lead to recency while single final judgments abolish recency and lead instead to integration of information across trials (i.e., primacy). These results challenge almost all existing accounts of causal judgment. We propose a modified associative account in which participants can base their causal judgments either on current associative strength (momentary strategy) or on the cumulative change in associative strength since the previous judgment (integrative strategy).

The Development of Judgment and Decision Making During Childhood and Adolescence

Research on adult judgment and decision making has focused on deviations from normative models, demonstrating biases and reliance on heuristic shortcuts, thus presenting a very different picture than developmental theories that describe a unidirectional progression toward greater logic and efficiency. Recent research related to this apparent contradiction indicates that children develop competencies to reason effectively and make normative decisions, but also develop biased judgment strategies that are used inappropriately in some situations. We suggest potential explanations for the findings, highlighting the need to consider models that incorporate development in both experiential and analytic information processing systems.

Episodic and semantic knowledge in emotional self-report: Evidence for two judgment processes.

Three studies involving 3 participant samples (Ns = 39, 55, and 53) tested the hypothesis that people retrieve episodic emotion knowledge when reporting on their emotions over short (e.g., last few hours) time frames, but that they retrieve semantic emotion knowledge when reporting on their emotions over long (e.g., last few months) time frames. Support for 2 distinct judgment strategies was based on judgment latencies (Studies 1 and 2) and priming paradigms (Studies 2 and 3). The authors suggest that self-reports of emotion over short versus long time frames assess qualitatively different sources of self-knowledge.

The fit and implementation of sexual harassment law to workplace evaluations.

Three studies used videotaped harassment complaints to examine the impact of legal standards on the evaluation of social-sexual conduct at work. Study 1 demonstrated that without legal instructions, college students' judgment strategies were highly variable. Study 2 compared 2 current legal standards, the "severity or pervasiveness test" and a proposed utilitarian alternative (i.e., the rational woman approach). Undergraduate participants taking the perspective of the complainant were more sensitive to offensive conduct than were those adopting an objective perspective. Although the utilitarian altemative further increased sensitivity on some measures, it failed to produce a principled judgment strategy. Finally, Study 3 examined the kinds of errors that full-time workers make when applying the "severity or pervasiveness" test to examine more closely the sensitivity of the subjective approach.

SEEING AND SINGING: VISIONS AND HYMNS IN THE BOOK OF AMOS

AbstractIn the present composition of the Book of Amos, a striking relationship can be found between the hymns (Am. iv 13; v 8; ix 5-6) and three of the five vision-reports (vii 1-3, 4-6; ix 1-4). Thus, at least the second part of the Book is presented as a 'progress of doom', in which the prophet is more and more absorbed in the execution of YHWH's judgment, personally and qua communicative strategy. The second pair of vision-reports (vii 7-9; viii 1-3) serve in the current shape of the Book as a framework for the so-called biographical section Am. vii 10-17. This composition is aimed at a reading audience that is exhorted to learn from the bad example of Amaziah and to believe the divine origin of Amos' words.

A study of the use of past experiences in clinical decision making in emergency situations

Making decisions to call emergency assistance to patients is an important dimension of nursing practice. Most usually these decision making situations are uncertain and it is expected nurses rely on past clinical experiences. This study, approved by the ethics committees of both a university and an area health service, aimed to describe nurses’ reliance on past experiences and identify associated judgement strategies (heuristics). Thirty-two registered nurses with five or more years experience were interviewed. Main findings were: nurses did use their past experiences and these experiences were used in the form of the three ‘classic’ heuristics, representativeness, availability and anchoring and adjustment. It can be concluded past experiences are intrinsic to decision making and this has implications for both the clinical components of nursing educational programs and staffing allocations made by administrators. Some nurses, however, did not include referral to past experiences in their decision-making accounts which may be a limitation of the study design.

“That’s Completely Obvious... and Important”: Lay Judgments of Social Psychological Findings

Many social psychologists believe that if research results are obvious, they are unimportant and uninteresting. The current study evaluated lay perceptions of social psychological research findings. Results from three studies reveal differences between lay evaluations of research and scientific evaluations. In Study 1, students with no prior exposure to social psychology judge the most obvious research findings to be the most important. In Study 2, students can predict findings, and the most predictable findings are judged most important. In Study 3, students judge the most obvious findings to be most important to establish with research. Results address the accuracy of lay judgments of research, judgmental strategies, and the process by which social psychologists select research topics.

Cognitive issues in image quality measurement

Designers of imaging systems, image processing algorithms, etc., usually take for granted that methods for assessing perceived image quality produce unbiased estimates of the viewers’ quality impression. Quality judgments, however, are affected by the judgment strategies induced by the experimental procedures. In this paper the results of two experiments are presented illustrating the influence judgment strategies can have on quality judgments. The first experiment concerns contextual effects due to the composition of the stimulus set. Subjects assessed the sharpness of two differently composed sets of blurred versions of one static image. The sharpness judgments for the blurred images present in both stimulus sets were found to be dependent on the composition of the set as well as the scaling technique employed. In the second experiment subjects assessed either the overall quality or the overall impairment of manipulated and standard JPEG-coded images containing two main artifacts. The results indicate a systematic difference between the quality and impairment judgments that could be interpreted as instruction-based different weighting of the two artifacts. Again, some influence of scaling technique was observed. The results of both experiments underscore the important role judgment strategies play in the psychophysical evaluation of image quality. Ignoring this influence on quality judgments may lead to invalid conclusions about the viewers’ impression of image quality.