Multiple-cue Probability Learning Research Articles

Kenneth R. Hammond’s contributions to the study of judgment and decision making

AbstractKenneth R. Hammond (1917–2015) made several major contributions to the science of human judgment and decision making. As a student of Egon Brunswik, he kept Brunswik’s legacy alive – advancing his theory of probabilistic functionalism and championing his method of representative design. Hammond pioneered the use of Brunswik’s lens model as a framework for studying how individuals use information from the task environment to make clinical judgments, which was the precursor to much ‘policy capturing’ and ‘judgment analysis’ research. Hammond introduced the lens model equation to the study of judgment processes, and used this to measure the utility of different forms of feedback in multiple-cue probability learning. He extended the scope of analysis to contexts in which individuals interact with one another – introducing the interpersonal learning and interpersonal conflict paradigms. Hammond developed social judgment theory which provided a comprehensive quantitative approach for describing and improving judgment processes. He proposed cognitive continuum theory which states that quasi-rationality is an important middle-ground between intuition and analysis and that cognitive performance is dictated by the match between task properties and mode of cognition. Throughout his career, Hammond moved easily from basic laboratory work to applied settings, where he resolved policy disputes, and in doing so, he pointed to the dichotomy between theories of correspondence and coherence. In this paper, we present Hammond’s legacy to a new generation of judgment and decision making scholars.

An investigation of the role of some person and situation variables in multiple cue probability learning

Making decisions using judgements of multiple non-deterministic indicators is an important task, both in everyday and professional life. Learning of such decision making has often been studied as the mapping of stimuli (cues) to an environmental variable (criterion); however, little attention has been paid to the effects of situation-by-person interactions on this learning. Accordingly, we manipulated cue and feedback presentation mode (graphic or numeric) and task difficulty, and measured individual differences in working memory capacity (WMC). We predicted that graphic presentation, fewer cues, and elevated WMC would facilitate learning, and that person and task characteristics would interact such that presentation mode compatible with the decision maker's cognitive capability (enhanced visual or verbal WMC) would assist learning, particularly for more difficult tasks. We found our predicted main effects, but no significant interactions, except that those with greater WMC benefited to a larger extent with graphic than with numeric presentation, regardless of which type of working memory was enhanced or number of cues. Our findings suggest that the conclusions of past research based predominantly on tasks using numeric presentation need to be reevaluated and cast light on how working memory helps us learn multiple cue-criterion relationships, with implications for dual-process theories of cognition.

Applying the Multiple Cue Probability Learning

In the present study, we apply the multiple cue probability learning (MCPL) paradigm to examine consumer learning from feedback in repeated trials. This paradigm is useful in investigating consumer learning, especially learning the relationships between the overall quality and attributes. With this paradigm, we can analyze what people learn from repeated trials by using the lens model, i.e., whether it is knowledge or consistency. In addition to introducing this paradigm, we aim to demonstrate that knowledge people gain from repeated trials with feedback is robust enough to weaken one of the most often examined contextual effects, the asymmetric dominance effect. The experiment consists of learning session and a choice task and stimuli are sport rafting boats with motor engines. During the learning session, the participants are shown an option with three attributes and are asked to evaluate its overall quality and type in a number between 0 and 100. Then an expert`s evaluation, a number between 0 and 100, is provided as feedback. This trial is repeated fifteen times with different sets of attributes, which comprises one learning session. Depending on the conditions, the participants do one (low) or three (high) learning sessions or do not go through any learning session (no learning). After learning session, the participants then are provided with either a core or an extended choice set to make a choice to examine if learning from feedback would weaken the asymmetric dominance effect. The experiment uses a between-subjects experimental design (2 ⅹ 3; core set vs. extended set; no vs. low vs. high learning). The results show that the participants evaluate the overall qualities more accurately with learning. They learn the true trade-off rule between attributes (increase in knowledge) and become more consistent in their evaluations. Regarding the choice task, there is a significant decrease in the percentage of choosing the target option in the extended sets with learning, which clearly demonstrates that learning decreases the magnitude of the asymmetric dominance effect. However, these results are significant only when no learning condition is compared either to low or high learning condition. There is no significant result between low and high learning conditions, which may be due to fatigue or reflect the characteristics of learning curve. The present study introduces the MCPL paradigm in examining consumer learning and demonstrates that learning from feedback increases both knowledge and consistency and weakens the asymmetric dominance effect. The latter result may suggest that the previous demonstrations of the asymmetric dominance effect are somewhat exaggerated. In a single choice setting, people do not have enough information or experience about the stimuli, which may lead them to depend mostly on the contextual structure among options. In the future, more realistic stimuli and real experts` judgments can be used to increase the external validity of study results. In addition, consumers often learn through repeated choices in real consumer settings. Therefore, what consumers learn from feedback in repeated choices would be an interesting topic to investigate.

External validity of individual differences in multiple cue probability learning: The case of pilot training

AbstractIndividuals differ in their ability to deal with unpredictable environments. Could impaired performances on learning an unpredictable cue-criteria relationship in a laboratory task be associated with impaired learning of complex skills in a natural setting? We focused on a multiple-cue probability learning (MCPL) laboratory task and on the natural setting of pilot training. We used data from three selection sessions and from the three corresponding selected pilot student classes of a national airline pilot selection and training system. First, applicants took an MCPL task at the selection stage (N = 556; N = 701; N = 412). Then, pilot trainees selected from the applicant pools (N = 44; N = 60; N = 28) followed the training for 2.5 to 3 yrs. Differences in final MCPL performance were associated with pilot training difficulties. Indeed, poor MCPL performers experienced almost twice as many pilot training difficulties as better MCPL performers (44.0% and 25.0%, respectively).

Dual-processes in learning and judgment: Evidence from the multiple cue probability learning paradigm

Multiple cue probability learning (MCPL) involves learning to predict a criterion based on a set of novel cues when feedback is provided in response to each judgment made. But to what extent does MCPL require controlled attention and explicit hypothesis testing? The results of two experiments show that this depends on cue polarity. Learning about cues that predict positively is aided by automatic cognitive processes, whereas learning about cues that predict negatively is especially demanding on controlled attention and hypothesis testing processes. In the studies reported here, negative, but not positive cue learning related to individual differences in working memory capacity both on measures of overall judgment performance and modelling of the implicit learning process. However, the introduction of a novel method to monitor participants’ explicit beliefs about a set of cues on a trial-by-trial basis revealed that participants were engaged in explicit hypothesis testing about positive and negative cues, and explicit beliefs about both types of cues were linked to working memory capacity. Taken together, our results indicate that while people are engaged in explicit hypothesis testing during cue learning, explicit beliefs are applied to judgment only when cues are negative.

Dimensions of Judgment: Factor Analysis of Individual Differences

ABSTRACTIn this paper, we explore a pattern of individual differences in performance on judgment tasks. One hundred participants completed a battery of standard judgment tasks, including multiple cue judgment and tasks based on the “heuristics and biases” and problem‐solving literature. Participants also completed tests of fluid and crystallized intelligence, forward digit span memory, and multiple cue probability learning. Confirmatory factor analyses indicated a two‐factor structure underlying performance on these tasks and tests. The two moderately correlated factors reflected a distinction between tasks requiring accurate judgments based on multiple cues (correspondence tasks) and those requiring coherent judgments (coherence tasks), in which probabilities are compared and combined. Measures of fluid and crystallized intelligence and forward digit span memory were correlated with both factors but were more associated with correspondence than with coherence. Copyright © 2011 John Wiley & Sons, Ltd.

The Role of Working Memory Capacity in Multiple-Cue Probability Learning

Multiple-cue probability learning (MCPL) involves learning to predict a criterion when outcome feedback is provided for multiple cues. A great deal of research suggests that working memory capacity (WMC) is involved in a wide range of tasks that draw on higher level cognitive processes. In three experiments, we examined the role of WMC in MCPL by introducing measures of working memory capacity, as well as other task manipulations. While individual differences in WMC positively predicted performance in some kinds of multiple-cue tasks, performance on other tasks was entirely unrelated to these differences. Performance on tasks that contained negative cues was correlated with working memory capacity, as well as measures of explicit knowledge obtained in the learning process. When the relevant cues predicted positively, however, WMC became irrelevant. The results are discussed in terms of controlled and automatic processes in learning and judgement.

Learning in a changing environment.

Multiple cue probability learning studies have typically focused on stationary environments. We present 3 experiments investigating learning in changing environments. A fine-grained analysis of the learning dynamics shows that participants were responsive to both abrupt and gradual changes in cue-outcome relations. We found no evidence that participants adapted to these types of change in qualitatively different ways. Also, in contrast to earlier claims that these tasks are learned implicitly, participants showed good insight into what they learned. By fitting formal learning models, we investigated whether participants learned global functional relationships or made localized predictions from similar experienced exemplars. Both a local (the associative learning model) and a global learning model (the Bayesian linear filter) fitted the data of the first 2 experiments. However, the results of Experiment 3, which was specifically designed to discriminate between local and global learning models, provided more support for global learning models. Finally, we present a novel model to account for the cue competition effects found in previous research and displayed by some of our participants.

The Effectiveness of Feedback in Multiple-Cue Probability Learning

How effective are different types of feedback in helping us to learn multiple contingencies? This article attempts to resolve a paradox whereby, in comparison to simple outcome feedback, additional feedback either fails to enhance or is actually detrimental to performance in nonmetric multiple-cue probability learning (MCPL), while in contrast the majority of studies of metric MCPL reveal improvements at least with some forms of feedback. In three experiments we demonstrate that if feedback assists participants to infer cue polarity then it can in fact be effective in nonmetric MCPL. Participants appeared to use cue polarity information to adopt a linear judgement strategy, even though the environment was nonlinear. The results reconcile the paradoxical contrast between metric and nonmetric MCPL and support previous findings of people's tendency to assume linearity and additivity in probabilistic cue learning.

Beyond nonutilization: Irrelevant cues can gate learning in probabilistic categorization.

In probabilistic categorization, also known as multiple cue probability learning (MCPL), people learn to predict a discrete outcome on the basis of imperfectly valid cues. In MCPL, normatively irrelevant cues are usually ignored, which stands in apparent conflict with recent research in deterministic categorization that has shown that people sometimes use irrelevant cues to gate access to partial knowledge encapsulated in independent partitions. The authors report 2 experiments that sought support for the existence of such knowledge partitioning in probabilistic categorization. The results indicate that, as in other areas of concept acquisition (such as function learning and deterministic categorization), a significant proportion of participants partitioned their knowledge on the basis of an irrelevant cue. The authors show by computational modeling that knowledge partitioning cannot be accommodated by 2 exemplar models (Generalized Context Model and Rapid Attention Shifts 'N Learning), whereas a rule-based model (General Recognition Theory) can capture partitioned performance. The authors conclude by pointing to the necessity of a mixture-of-experts approach to capture performance in MCPL and by identifying reduction of complexity as a possible explanation for partitioning.

Social influence on predictions of simulated stock prices

AbstractHerding in financial markets refers to that investors are influenced by others. This study addresses the importance of consistency for herding. It is suggested that, in financial markets perceptions of consistency are based on repeated observations over time. Consistency may then be perceived as the agreement across time between investors' predictions. In addition, consistency may be related to variance over time in each investor's predictions. In an experiment using a Multiple Cue Probability Learning paradigm, 96 undergraduates made multi‐trial predictions of future stock prices given information about the current price and the predictions made by five fictitious others. Consistency was varied between the others' predictions (correlation) and within the others' predictions (variance). The results showed that the predictions were significantly influenced by the others' predictions when these were correlated. No effect of variance was observed. Hence, participants were influenced by the others when they were in agreement, regardless of whether they varied their predictions over trials or not. Copyright © 2008 John Wiley & Sons, Ltd.

Choice strategies in multiple-cue probability learning.

Choice strategies for selecting among outcomes in multiple-cue probability learning were investigated using a simulated medical diagnosis task. Expected choice probabilities (the proportion of times each outcome was selected given each cue pattern) under alternative choice strategies were constructed from corresponding observed judged probabilities (of each outcome given each cue pattern) and compared with observed choice probabilities. Most of the participants were inferred to have responded by using a deterministic strategy, in which the outcome with the higher judged probability is consistently chosen, rather than a probabilistic strategy, in which an outcome is chosen with a probability equal to its judged probability. Extended practice in the learning environment did not affect choice strategy selection, contrary to reports from previous studies, results of which may instead be attributable to changes with practice in the variability and extremity of the perceived probabilities on which the choices were based.

Gender differences in the use of high-variance strategies in tournament competition

We report on a series of experiments in which participants first completed a simple game of chance and then competed in a tournament based on a multiple cue probability learning task. The results show that men made riskier choices in the simple game of chance, but women adopted the high-variance strategy more frequently in the tournament. However, after controlling for differences in forecasting skill, we find no significant differences in the rates at which men and women adopted a high-variance strategy. Although altering the difficulty of the forecasting task produces differences in the rate at which participants selected the high-variance strategy, it did not produce a significant difference in the rate at which men and women selected the strategy. Thus, this paper suggests that although women are more risk-averse than men, women are no less likely to adopt a high-variance strategy in a tournament competition.

Missing information in multiple-cue probability learning

In a multiple-cue probability learning task, participants learned to use six discrete symptoms (i.e., cues) to diagnose which of three possible flu strains a hypothetical patient suffered from. For some patients, information regarding the status of certain symptoms was not available. Various possible ways in which the missing cue information might be processed were distinguished and tested in a series of three experiments (Ns = 80, 109, and 61). The results suggest that the judged probability of the outcome variable (i.e., flu strain) was assessed by "filling in" the missing cue information with a mean value based on previous observations. The predictions of other methods of processing missing cue information are inconsistent with the data.

Aging and Probabilistic Learning in Single- and Multiple-Cue Tasks

The objective of the study was to examine some of the conditions under which elderly people are able to learn probabilistic inverse relationships and when this type of learning is no longer possible. Two kind of tasks were used: (a) two single-cue learning tasks with either direct or inverse relationships (the Single-Cue Probability Learning paradigm), and (b) three two-cue learning tasks, one with two direct relationships, one with a combination of direct and inverse relationships, and one with two inverse relationships (the Multiple-Cue Probability Learning paradigm). Four groups of participants were included in the study: young adults (18- to 25-year-olds), adults (40- to 50-year-olds), elderly people (65- to 74-year-olds), and very elderly people (75- to 90-year-olds). It was shown that (a) older adults are able to reject the direct relationship “default” hypothesis and select the inverse relationship hypothesis when outcome feedback contradicting the default hypothesis is given, and provided that the learning setting be a very simple one, involving only one cue; (b) some older adults are able to select the inverse relationship hypothesis provided that the learning setting be a simple one, involving only two inverse relationship cues; and (c) very few older adults are able to select the inverse relationship hypothesis when the learning setting is a complex one, involving two cues with both direct and inverse relationships with the criterion. These results led to the revision of the “gradual decrease of cognitive flexibility in older adults” hypothesis proposed by Chasseigne, Mullet, and Stewart (Acta Psychologic`a, 103, 229–238, 1997).

The Use of Cognitive Feedback by Experts and Novices in a Judgment Task

Cognitive feedback (CFB) has been shown to increase performance in Multiple Cue Probability Learning (MCPL) tasks in both real life and experimental situations. Our experiment focuses on investigating how experts and novices behave differently in applying CFB, which is presented in forms of relative importance weights of each of the cues and the display of function forms (Weaver and Steward, 2000). We believe this study provided some interesting insight into how much domain knowledge is necessary to understand and apply CFB. The study examined how well novices and experts are able to process and use different types of feedback (i.e. CFB and Outcome Feedback) and hence determined which type is more useful for novices or for those with prior knowledge of the task. A total of 20 participants, 10 experts and 10 novices, were given a series of baseball statistics and asked to predict how many wins the team would have in a season based on these statistics. Performance measures (e.g. relative weighting accuracy, Lens Model measures), mental workload and judgment strategy were measured. Domain Knowledge was statistically significant for all weights except one less important cue, “stolen bases.” In other words, domain knowledge impacts weighting accuracy greatly. Domain knowledge was also found to have a significant effect on judgment performance (i.e. Lens Model measures), such as accuracy and knowledge. Mental workload was significantly affected by CFB, but none of the other dependent variables were.

Task difficulty, incentive effects, and the selection of high-variance strategies: an experimental examination of tournament behavior

Previous research on tournaments shows two important results. First, increasing the gap between the payoffs to winners and losers will increase effort and performance. Second, agents with lower capabilities will adopt strategies that increase the variance of their performance. This paper argues that these results are not independent but vary with the difficulty of the task. The results from a series of experiments using a multiple cue probability learning forecasting task confirm that wider payments do not lead to increased performance when the task is simple. In addition, agents in the simple task select high-variance strategies more frequently. The increased use of high-variance strategies in the simple task occurs because agents with low capabilities use the strategy more frequently. This implies that the difficulty of the task must be taken into consideration when designing tournaments.

An evidential support accumulation model of subjective probability

A model of cue-based probability judgment is developed within the framework of support theory. Cue diagnosticity is evaluated from experience as represented by error-free frequency counts. When presented with a pattern of cues, the diagnostic implications of each cue are assessed independently and then summed to arrive at an assessment of the support for a hypothesis, with greater weight placed on present than on absent cues. The model can also accommodate adjustment of support in light of the baserate or prior probability of a hypothesis. Support for alternatives packed together in a “residual” hypothesis is discounted; fewer cues are consulted in assessing support for alternatives as support for the focal hypothesis increases. Results of fitting this and several alternative models to data from four new multiple-cue probability learning experiments are reported.

How well do elderly people cope with uncertainty in a learning task?

The relation between age, task complexity and learning performance in a Multiple Cue Probability Learning task was studied by systematically varying the level of uncertainty present in the task, keeping constant the direction of relationships. Four age groups were constituted: young adults (mean age=21), middle-aged adults (45), elderly people (69) and very elderly people (81). Five uncertainty levels were considered: predictability=0.96, 0.80, 0.64, 0.48, and 0.32. All relationships involved were direct ones. A strong effect of uncertainty on `control', a measure of the subject’s consistency with respect to a linear model, was found. This effect was essentially a linear one. To each decrement in predictability of the task corresponded an equal decrement in participants' level of control. This level of decrement was the same, regardless of the age of the participant. It can be concluded that elderly people cope with uncertainty in probability learning tasks as well as young adults.

A model of probabilistic category learning.

A new connectionist model (named RASHNL) accounts for many "irrational" phenomena found in nonmetric multiple-cue probability learning, wherein people learn to utilize a number of discrete-valued cues that are partially valid indicators of categorical outcomes. Phenomena accounted for include cue competition, effects of cue salience, utilization of configural information, decreased learning when information is introduced after a delay, and effects of base rates. Experiments 1 and 2 replicate previous experiments on cue competition and cue salience, and fits of the model provide parameter values for making qualitatively correct predictions for many other situations. The model also makes 2 new predictions, confirmed in Experiments 3 and 4. The model formalizes 3 explanatory principles: rapidly shifting attention with learned shifts, decreasing learning rates, and graded similarity in exemplar representation.