Ideal Observer Model Research Articles

Visual Attention in Crisis.

Research on visual attention has uncovered significant anomalies, and some traditional methods may have inadvertently probed peripheral vision rather than attention. Vision science needs to rethink visual attention from the ground up. To facilitate this, for a year I banned the word "attention" in my lab. This constraint promoted a more precise discussion of attention-related phenomena, capacity limits, and mechanisms. The insights gained lead me to challenge attributing to "attention" those phenomena that can be better explained by perceptual processes, are predictable by an ideal observer model, or that otherwise may not require an additional mechanism. I enumerate a set of critical phenomena in need of explanation. Finally, I propose a unifying theory in which all perception results from performing a task, and tasks face a limit on complexity.

Changes in neural readout of response magnitude during auditory streaming do not correlate with behavioral choice in the auditory cortex

A fundamental goal of the auditory system is to group stimuli from the auditory environment into a perceptual unit (i.e., "stream") or segregate the stimuli into multiple different streams. Although previous studies have clarified the psychophysical and neural mechanisms that may underlie this ability, the relationship between these mechanisms remains elusive. Here, we recorded multiunit activity (MUA) from the auditory cortex of monkeys while they participated in an auditory-streaming task consisting of interleaved low- and high-frequency tone bursts. As the streaming stimulus unfolded over time, MUA amplitude habituated; the magnitude of this habituation was correlated with the frequency difference between the tone bursts. An ideal-observer model could classify these time- and frequency-dependent changes into reports of "one stream" or "two streams" in a manner consistent with the behavioral literature. However, because classification was not modulated by the monkeys' behavioral choices, this MUA habituation may not directly reflect perceptual reports.

Why fractions are difficult? Modeling optimal and sub-optimal integration strategies of numerators and denominators by educated adults

Many children and educated adults experience difficulties in understanding and manipulating fractions. In this study, we argue that a major cause of this challenge is rooted in the need to integrate information from two separate informational sources (i.e., denominator and numerator) according to a normative arithmetic rule (i.e., division). We contend that in some tasks, the correct arithmetic rule is replaced by an inadequate (sub-optimal) operation (e.g., multiplication), which leads to inaccurate representation of fractions. We tested this conjecture by applying two rigorous models of information integration : (a) functional measurement (Experiments 1-3) and (b) conjoint measurement (Experiment 4-5) to data from number-to-line and comparative judgment tasks. These allowed us to compare participants’ integration strategies with that of an ideal-observer model. Functional measurement analyses on data from the number-to-line task, revealed that participants could represent the global magnitude of proper and improper fractions quite accurately and combine the fractions’ components according to an ideal-observer model. However, conjoint measurement analyses on data from the comparative judgment task, showed that most participants combined these fractions’ components according to a sub-optimal (saturated) observer model, that is inconsistent with an ideal-observer (additive) model. These results support the view that educated adults are capable of extracting multiple types of representations of fractions depending on the task at-hand. These representations can be either accurate and conform with normative arithmetic or approximated and inconsistent with normative arithmetic. The latter may lead to the observed difficulties people experience with fractions.

Auditory learning of recurrent tone sequences is present in the newborn's brain

The seemingly effortless ability of our auditory system to rapidly detect new events in a dynamic environment is crucial for survival. Whether the underlying brain processes are innate is unknown. To answer this question, electroencephalography was recorded while regularly patterned (REG) versus random (RAND) tone sequences were presented to sleeping neonates. Regular relative to random sequences elicited differential neural responses after only a single repetition of the pattern indicating the existence of an innate capacity of the auditory system to detect auditory sequential regularities. We show that the newborn auditory system accumulates evidence only somewhat longer than the minimum amount determined by the ideal Bayesian observer model (the prediction from a variable-order Markov chain model) before detecting a repeating pattern. Thus, newborns can quickly form representations for regular features of the sound input, preparing the way for learning the contingencies of the environment.

Only visible flicker helps flutter: Tactile-visual integration breaks in the absence of visual awareness

Combining information from multiple senses enhances our perception of the world. Whether we need to be aware of all stimuli to benefit from multisensory integration, however, is still under investigation. Here, we tested whether tactile frequency perception benefits from the presence of congruent visual flicker even if the flicker is so rapid that it is perceptually fused into a steady light and therefore invisible. Our participants completed a tactile frequency discrimination task given either unisensory tactile or congruent tactile-visual stimulation. Tactile and tactile-visual test frequencies ranged from far below to far above participants' flicker fusion threshold (determined separately). For frequencies distinctively below their flicker fusion threshold, participants performed significantly better given tactile-visual stimulation than when presented with only tactile stimuli. Yet, for frequencies above their flicker fusion threshold, participants' tactile frequency perception did not profit from the presence of congruent but likely fused and thus invisible visual flicker. The results matched the predictions of an ideal-observer model in which tactile-visual integration is conditional on awareness of both stimuli. In contrast, it was impossible to reproduce the observed results with a model that assumed tactile-visual integration proceeds irrespective of stimulus awareness. In sum, we revealed that the benefits of congruent visual stimulation for tactile flutter frequency perception depend on the visibility of the visual flicker, suggesting that multisensory integration requires awareness.

State-specific alterations in the neural computations underlying inhibitory control in women remitted from bulimia nervosa.

The neurocomputational processes underlying bulimia nervosa and its primary symptoms, out-of-control overeating and purging, are poorly understood. Research suggests that the brains of healthy individuals form a dynamic internal model to predict whether control is needed in each moment. This study tested the hypothesis that this computational process of inhibitory control is abnormally affected by metabolic state (being fasted or fed) in bulimia nervosa. A Bayesian ideal observer model was fit to behavioral data acquired from 22 women remitted from bulimia nervosa and 20 group-matched controls who completed a stop-signal task during two counterbalanced functional MRI sessions, one after a 16 h fast and one after a meal. This model estimates participants' trial-by-trial updating of the probability of a stop signal based on their experienced trial history. Neural analyses focused on control-related Bayesian prediction errors, which quantify the direction and degree of "surprise" an individual experiences on any given trial. Regardless of group, metabolic state did not affect behavioral performance on the task. However, metabolic state modulated group differences in neural activation. In the fed state, women remitted from bulimia nervosa had attenuated prediction-error-dependent activation in the left dorsal caudate. This fed-state activation was lower among women with more frequent past binge eating and self-induced vomiting. When they are in a fed state, individuals with bulimia nervosa may not effectively process unexpected information needed to engage inhibitory control. This may explain the difficulties these individuals have stopping eating after it begins.

Quantum spin models for numerosity perception.

Humans share with animals, both vertebrates and invertebrates, the capacity to sense the number of items in their environment already at birth. The pervasiveness of this skill across the animal kingdom suggests that it should emerge in very simple populations of neurons. Current modelling literature, however, has struggled to provide a simple architecture carrying out this task, with most proposals suggesting the emergence of number sense in multi-layered complex neural networks, and typically requiring supervised learning; while simple accumulator models fail to predict Weber's Law, a common trait of human and animal numerosity processing. We present a simple quantum spin model with all-to-all connectivity, where numerosity is encoded in the spectrum after stimulation with a number of transient signals occurring in a random or orderly temporal sequence. We use a paradigmatic simulational approach borrowed from the theory and methods of open quantum systems out of equilibrium, as a possible way to describe information processing in neural systems. Our method is able to capture many of the perceptual characteristics of numerosity in such systems. The frequency components of the magnetization spectra at harmonics of the system's tunneling frequency increase with the number of stimuli presented. The amplitude decoding of each spectrum, performed with an ideal-observer model, reveals that the system follows Weber's law. This contrasts with the well-known failure to reproduce Weber's law with linear system or accumulators models.

Attractive serial dependence in heading perception from optic flow occurs at the perceptual and postperceptual stages.

Previous work has revealed that the heading perception from optic flow can be either attracted to the straight-ahead direction showing a center bias or repelled away from the previously seen heading (i.e., repulsive serial dependence) after ruling out the center bias accounting for perceptual errors. Recent studies have debated whether the serial dependence occurs at the perceptual or postperceptual stages (e.g., working memory). Our current study reexamined the serial dependence in heading perception and investigated whether the serial dependence occurred at perceptual or postperceptual stages. Additionally, an ideal observer model was developed to explore whether observers optimally combined the straight-ahead direction and previous and current headings to perceive headings. Our results showed that after ruling out the center bias, the perceived heading was biased toward the previous heading, suggesting an attractive serial dependence in heading perception. This attractive serial dependence occurred at both perceptual and postperceptual stages. Importantly, the perceived heading was well predicted by an ideal observer model, suggesting that observers could optimally combine their perceptual observations (current heading) with their prior information about the straight-ahead direction and previous headings to estimate their heading.

Tracking the contribution of inductive bias to individualised internal models.

Internal models capture the regularities of the environment and are central to understanding how humans adapt to environmental statistics. In general, the correct internal model is unknown to observers, instead they rely on an approximate model that is continually adapted throughout learning. However, experimenters assume an ideal observer model, which captures stimulus structure but ignores the diverging hypotheses that humans form during learning. We combine non-parametric Bayesian methods and probabilistic programming to infer rich and dynamic individualised internal models from response times. We demonstrate that the approach is capable of characterizing the discrepancy between the internal model maintained by individuals and the ideal observer model and to track the evolution of the contribution of the ideal observer model to the internal model throughout training. In particular, in an implicit visuomotor sequence learning task the identified discrepancy revealed an inductive bias that was consistent across individuals but varied in strength and persistence.

Explaining human sampling rates across different decision domains

AbstractUndersampling biases are common in the optimal stopping literature, especially for economic full choice problems. Among these kinds of number-based studies, the moments of the distribution of values that generates the options (i.e., the generating distribution) seem to influence participants’ sampling rate. However, a recent study reported an oversampling bias on a different kind of optimal stopping task: where participants chose potential romantic partners from images of faces (Furl et al., 2019). The authors hypothesised that this oversampling bias might be specific to mate choice. We preregistered this hypothesis and so, here, we test whether sampling rates across different image-based decision-making domains a) reflect different over- or undersampling biases, or b) depend on the moments of the generating distributions (as shown for economic number-based tasks). In two studies (N= 208 andN= 96), we found evidence against the preregistered hypothesis. Participants oversampled to the same degree across domains (compared to a Bayesian ideal observer model), while their sampling rates depended on the generating distribution mean and skewness in a similar way as number-based paradigms. Moreover, optimality model sampling to some extent depended on the the skewness of the generating distribution in a similar way to participants. We conclude that oversampling is not instigated by the mate choice domain and that sampling rate in image-based paradigms, like number-based paradigms, depends on the generating distribution.

Generic learning mechanisms can drive social inferences: The role of type frequency

How do we form opinions about typical and morally acceptable behavior in other social groups despite variability in behavior? Similar learning problems arise during language acquisition, where learners need to infer grammatical rules (e.g., the walk/walk-ed past-tense) despite frequent exceptions (e.g., the go/went alternation). Such rules need to occur with many different words to be learned (i.e., they need a high type frequency). In contrast, frequent individual words do not lead to learning. Here, we ask whether similar principles govern social learning. Participants read a travel journal where a traveler observed behaviors in different imaginary cities. The behaviors were performed once by many distinct actors (high type frequency) or frequently by a single actor (low type frequency), and could be good, neutral or bad. We then asked participants how morally acceptable the behavior was (in general or for the visited city), and how widespread it was in that city. We show that an ideal observer model estimating the prevalence of behaviors is only sensitive to the behaviors’ type frequency, but not to how often they are performed. Empirically, participants rated high type frequency behaviors as more morally acceptable more prevalent than low type frequency behaviors. They also rated good behaviors as more acceptable and prevalent than neutral or bad behaviors. These results suggest that generic learning mechanisms and epistemic biases constrain social learning, and that type frequency can drive inferences about groups. To combat stereotypes, high type frequency behaviors might thus be more effective than frequently appearing individual role models.

Analyzing the misperception of exponential growth in graphs

Exponential growth is frequently underestimated, an error that can have a heavy social cost in the context of epidemics. To clarify its origins, we measured the human capacity (N = 521) to extrapolate linear and exponential trends in scatterplots. Four factors were manipulated: the function underlying the data (linear or exponential), the response modality (pointing or venturing a number), the scale on the y axis (linear or logarithmic), and the amount of noise in the data. While linear extrapolation was precise and largely unbiased, we observed a consistent underestimation of noisy exponential growth, present for both pointing and numerical responses. A biased ideal-observer model could explain these data as an occasional misperception of noisy exponential graphs as quadratic curves. Importantly, this underestimation bias was mitigated by participants' math knowledge, by using a logarithmic scale, and by presenting a noiseless exponential curve rather than a noisy data plot, thus suggesting concrete avenues for interventions.

Ideal observer analysis for continuous tracking experiments.

Continuous tracking is a newly developed technique that allows fast and efficient data acquisition by asking participants to “track” a stimulus varying in some property (usually position in space). Tracking is a promising paradigm for the investigation of dynamic features of perception and could be particularly well suited for testing ecologically relevant situations difficult to study with classical psychophysical paradigms. The high rate of data collection may be useful in studies on clinical populations and children, who are unable to undergo long testing sessions. In this study, we designed tracking experiments with two novel stimulus features, numerosity and size, proving the feasibility of the technique outside standard object tracking. We went on to develop an ideal observer model that characterizes the results in terms of efficiency of conversion of stimulus strength into responses, and identification of early and late noise sources. Our ideal observer closely modeled results from human participants, providing a generalized framework for the interpretation of tracking data. The proposed model allows to use the tracking paradigm in various perceptual domains, and to study the divergence of human participants from ideal behavior.

Rat sensitivity to multipoint statistics is predicted by efficient coding of natural scenes.

Efficient processing of sensory data requires adapting the neuronal encoding strategy to the statistics of natural stimuli. Previously, in Hermundstad et al., 2014, we showed that local multipoint correlation patterns that are most variable in natural images are also the most perceptually salient for human observers, in a way that is compatible with the efficient coding principle. Understanding the neuronal mechanisms underlying such adaptation to image statistics will require performing invasive experiments that are impossible in humans. Therefore, it is important to understand whether a similar phenomenon can be detected in animal species that allow for powerful experimental manipulations, such as rodents. Here we selected four image statistics (from single- to four-point correlations) and trained four groups of rats to discriminate between white noise patterns and binary textures containing variable intensity levels of one of such statistics. We interpreted the resulting psychometric data with an ideal observer model, finding a sharp decrease in sensitivity from two- to four-point correlations and a further decrease from four- to three-point. This ranking fully reproduces the trend we previously observed in humans, thus extending a direct demonstration of efficient coding to a species where neuronal and developmental processes can be interrogated and causally manipulated.

Numerosity Perception in Peripheral Vision.

Peripheral vision has different functional priorities for mammals than foveal vision. One of its roles is to monitor the environment while central vision is focused on the current task. Becoming distracted too easily would be counterproductive in this perspective, so the brain should react to behaviourally relevant changes. Gist processing is good for this purpose, and it is therefore not surprising that evidence from both functional brain imaging and behavioural research suggests a tendency to generalize and blend information in the periphery. This may be caused by the balance of perceptual influence in the periphery between bottom-up (i.e., sensory information) and top-down (i.e., prior or contextual information) processing channels. Here, we investigated this interaction behaviourally using a peripheral numerosity discrimination task with top-down and bottom-up manipulations. Participants compared numerosity between the left and right peripheries of a screen. Each periphery was divided into a centre and a surrounding area, only one of which was a task relevant target region. Our top-down task modulation was the instruction which area to attend – centre or surround. We varied the signal strength by altering the stimuli durations i.e., the amount of information presented/processed (as a combined bottom-up and recurrent top-down feedback factor). We found that numerosity perceived in target regions was affected by contextual information in neighbouring (but irrelevant) areas. This effect appeared as soon as stimulus duration allowed the task to be reliably performed and persisted even at the longest duration (1 s). We compared the pattern of results with an ideal-observer model and found a qualitative difference in the way centre and surround areas interacted perceptually in the periphery. When participants reported on the central area, the irrelevant surround would affect the response as a weighted combination – consistent with the idea of a receptive field focused in the target area to which irrelevant surround stimulation leaks in. When participants report on surround, we can best describe the response with a model in which occasionally the attention switches from task relevant surround to task irrelevant centre – consistent with a selection model of two competing streams of information. Overall our results show that the influence of spatial context in the periphery is mandatory but task dependent.

Role of memory in a Bayesian ideal observer model of visual search in natural images

Role of memory in a Bayesian ideal observer model of visual search in natural images

Neural Responses to Bayesian Expectation of Reward are Related to Anxiety and Post-Traumatic Guilt in Trauma-Expose Veterans

Abnormalities in reward responsiveness are common among individuals exposed to trauma and exacerbated by depressive and post-traumatic guilt symptoms. Such alterations may be difficult to identify based on self-report alone. By providing a fine-grained mechanistic framework of explicit and latent cognitive processes, computational models could help identify clinical markers of altered reward processing to in turn improve long-term clinical outcome prediction and treatment recommendations. Here, we apply a Bayesian ideal-observer model to neuroimaging data to investigate neural markers of reward learning abnormalities in combat trauma-exposed Veterans.

Serial dependence and center bias in heading perception from optic flow.

Previous work shows that observers can use information from optic flow to perceive the direction of self-motion (i.e. heading) and that perceived heading exhibits a bias towards the center of the display (center bias). More recent work shows that the brain is sensitive to serial correlations and the perception of current stimuli can be affected by recently seen stimuli, a phenomenon known as serial dependence. In the current study, we examined whether, apart from center bias, serial dependence could be independently observed in heading judgments and how adding noise to optic flow affected center bias and serial dependence. We found a repulsive serial dependence effect in heading judgments after factoring out center bias in heading responses. The serial effect expands heading estimates away from the previously seen heading to increase overall sensitivity to changes in heading directions. Both the center bias and repulsive serial dependence effects increased with increasing noise in optic flow, and the noise-dependent changes in the serial effect were consistent with an ideal observer model. Our results suggest that the center bias effect is due to a prior of the straight-ahead direction in the Bayesian inference account for heading perception, whereas the repulsive serial dependence is an effect that reduces response errors and has the added utility of counteracting the center bias in heading judgments.

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception.

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.

How optimal is word recognition under multimodal uncertainty?

Identifying a spoken word in a referential context requires both the ability to integrate multimodal input and the ability to reason under uncertainty. How do these tasks interact with one another? We study how adults identify novel words under joint uncertainty in the auditory and visual modalities, and we propose an ideal observer model of how cues in these modalities are combined optimally. Model predictions are tested in four experiments where recognition is made under various sources of uncertainty. We found that participants use both auditory and visual cues to recognize novel words. When the signal is not distorted with environmental noise, participants weight the auditory and visual cues optimally, that is, according to the relative reliability of each modality. In contrast, when one modality has noise added to it, human perceivers systematically prefer the unperturbed modality to a greater extent than the optimal model does. This work extends the literature on perceptual cue combination to the case of word recognition in a referential context. In addition, this context offers a link to the study of multimodal information in word meaning learning.