Stimulus Space Research Articles

Derivatives and inverse of cascaded linear+nonlinear neural models.

In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina).

Seeing through transparent layers.

The human visual system is remarkably good at decomposing local and global deformations in the flow of visual information into different perceptual layers, a critical ability for daily tasks such as driving through rain or fog or catching that evasive trout. In these scenarios, changes in the visual information might be due to a deforming object or deformations due to a transparent medium, such as structured glass or water, or a combination of these. How does the visual system use image deformations to make sense of layering due to transparent materials? We used eidolons to investigate equivalence classes for perceptually similar transparent layers. We created a stimulus space for perceptual equivalents of a fiducial scene by systematically varying the local disarray parameters reach and grain. This disarray in eidolon space leads to distinct impressions of transparency, specifically, high reach and grain values vividly resemble water whereas smaller grain values appear diffuse like structured glass. We asked observers to adjust image deformations so that the objects in the scene looked like they were seen (a) under water, (b) behind haze, or (c) behind structured glass. Observers adjusted image deformation parameters by moving the mouse horizontally (grain) and vertically (reach). For two conditions, water and glass, we observed high intraobserver consistency: responses were not random. Responses yielded a concentrated equivalence class for water and structured glass.

Reconstructing Stimulus Space Geometry from Auditory Cortex Responses

Reconstructing Stimulus Space Geometry from Auditory Cortex Responses

Thalamocortical Circuits and Functional Architecture.

The thalamocortical pathway is the main route of communication between the eye and the cerebral cortex. During embryonic development, thalamocortical afferents travel to L4 and are sorted by receptive field position, eye of origin, and contrast polarity (i.e., preference for light or dark stimuli). In primates and carnivores, this sorting involves numerous afferents, most of which sample a limited region of the binocular field. Devoting abundant thalamocortical resources to process a limited visual field has a clear advantage: It allows many stimulus combinations to be sampled at each spatial location. Moreover, the sampling efficiency can be further enhanced by organizing the afferents in a cortical grid for eye input and contrast polarity. We argue that thalamocortical interactions within this eye-polarity grid can be used to represent multiple stimulus combinations found in nature and to build an accurate cortical map for multidimensional stimulus space.

Correcting "confusability regions" in face morphs.

The visual system represents summary statistical information from a set of similar items, a phenomenon known as ensemble perception. In exploring various ensemble domains (e.g., orientation, color, facial expression), researchers have often employed the method of continuous report, in which observers select their responses from a gradually changing morph sequence. However, given their current implementation, some face morphs unintentionally introduce noise into the ensemble measurement. Specifically, some facial expressions on the morph wheel appear perceptually similar even though they are far apart in stimulus space. For instance, in a morph wheel of happy-sad-angry-happy expressions, an expression between happy and sad may not be discriminable from an expression between sad and angry. Without accounting for this confusability, observer ability will be underestimated. In the present experiments we accounted for this by delineating the perceptual confusability of morphs of multiple expressions. In a two-alternative forced choice task, eight observers were asked to discriminate between anchor images (36 in total) and all 360 facial expressions on the morph wheel. The results were visualized on a "confusability matrix," depicting the morphs most likely to be confused for one another. The matrix revealed multiple confusable images between distant expressions on the morph wheel. By accounting for these "confusability regions," we demonstrated a significant improvement in performance estimation on a set of independent ensemble data, suggesting that high-level ensemble abilities may be better than has been previously thought. We also provide an alternative computational approach that may be used to determine potentially confusable stimuli in a given morph space.

Dynamic representation of 3D auditory space in the midbrain of the free-flying echolocating bat.

Essential to spatial orientation in the natural environment is a dynamic representation of direction and distance to objects. Despite the importance of 3D spatial localization to parse objects in the environment and to guide movement, most neurophysiological investigations of sensory mapping have been limited to studies of restrained subjects, tested with 2D, artificial stimuli. Here, we show for the first time that sensory neurons in the midbrain superior colliculus (SC) of the free-flying echolocating bat encode 3D egocentric space, and that the bat's inspection of objects in the physical environment sharpens tuning of single neurons, and shifts peak responses to represent closer distances. These findings emerged from wireless neural recordings in free-flying bats, in combination with an echo model that computes the animal's instantaneous stimulus space. Our research reveals dynamic 3D space coding in a freely moving mammal engaged in a real-world navigation task.

The relative importance of perceptual and memory sampling processes in determining the time course of absolute identification.

In absolute identification, the extended generalized context model (EGCM; Kent & Lamberts, 2005, 2016) proposes that perceptual processing determines systematic response time (RT) variability; all other models of RT emphasize response selection processes. In the EGCM-RT the bow effect in RTs (longer responses for stimuli in the middle of the range) occurs because these middle stimuli are less isolated, and as perceptual information is accumulated, the evidence supporting a correct response grows more slowly than for stimuli at the ends of the range. More perceptual information is therefore accumulated in order to increase certainty in response for middle stimuli, lengthening RT. According to the model reducing perceptual sampling time should reduce the size of the bow effect in RT. We tested this hypothesis in 2 pitch identification experiments. Experiment 1 found no effect of stimulus duration on the size of the RT bow. Experiment 2 used multiple short stimulus durations as well as manipulating set size and stimulus spacing. Contrary to EGCM-RT predictions, the bow effect on RTs was large for even very short durations. A new version of the EGCM-RT could only capture this, alongside the effect of stimulus duration on accuracy, by including both a perceptual and a memory sampling process. A modified version of the selective attention, mapping, and ballistic accumulator model (Brown, Marley, Donkin, & Heathcote, 2008) could also capture the data, by assuming psychophysical noise diminishes with increased exposure duration. This modeling suggests systematic variability in RT in absolute identification is largely determined by memory sampling and response selection processes. (PsycINFO Database Record

Alertness and cognitive control: Toward a spatial grouping hypothesis.

A puzzling interaction involving alertness and cognitive control is indicated by the finding of faster performance but larger congruency effects on alert trials (on which alerting cues are presented before the task stimuli) than on no-alert trials in selective attention tasks. In the present study, the author conducted four experiments to test hypotheses about the interaction. Manipulation of stimulus spacing revealed a difference in congruency effects between alert and no-alert trials for narrowly spaced stimuli but not for widely spaced stimuli, inconsistent with the hypothesis that increased alertness is associated with more diffuse attention. Manipulation of color grouping revealed similar differences in congruency effects between alert and no-alert trials for same-color and different-color groupings of targets and distractors, inconsistent with the general hypothesis that increased alertness is associated with more perceptual grouping. To explain the results, the author proposes that increased alertness is associated specifically with more spatial grouping of stimuli, possibly by modulating the threshold for parsing stimulus displays into distinct objects.

Stimulus space complexity determines the ratio of specialist and generalist neurons during pattern recognition

Neural processing layers built on divergent connectivity patterns display two types of stimulus-dependent responses: neurons that react to a few stimuli, specialists, and other ones that respond to a wide range of inputs, generalists. Specialists are essential for the discrimination of stimuli and generalists extract common and generic properties from them. This neural heterogeneity could have emerged because of animal adaptation to the environment. Thus, we suggest that there is a relationship between the percentage of specialists and generalists and the stimulus complexity. In order to study this possible relationship, we use patterns with different complexities in a bio-inspired neural network and calculate their classification errors for different ratios of these types of neurons. This study shows that, when the complexity of the stimuli is low, the minimum classification error is achieved with almost any specialist-generalist ratio. Thus, in this case, the role of these neurons during pattern recognition is unspecific. When this complexity is intermediate, both are needed to minimize the classification error, usually in a similar proportion. For increasing stimulus complexity, the importance of generalists decreases, until their relevance is fully nullified when the complexity is high. Therefore, if we adjust the specialist-generalist ratio to the complexity of patterns, we can build more effective neural networks for pattern recognition. Finally, we propose an estimation of stimulus complexity based on the proportion of these types of neurons observed by neural recordings. This offers the possibility to evaluate the stimulus complexity to which animals are adapted.

The transfer of category knowledge by macaques (Macaca mulatta) and humans (Homo sapiens).

Cognitive psychologists distinguish implicit, procedural category learning (stimulus-response associations learned outside declarative cognition) from explicit-declarative category learning (conscious category rules). These systems are dissociated by category learning tasks with either a multidimensional, information-integration (II) solution or a unidimensional, rule-based (RB) solution. In the present experiments, humans and two monkeys learned II and RB category tasks fostering implicit and explicit learning, respectively. Then they received occasional transfer trials-never directly reinforced-drawn from untrained regions of the stimulus space. We hypothesized that implicit-procedural category learning-allied to associative learning-would transfer weakly because it is yoked to the training stimuli. This result was confirmed for humans and monkeys. We hypothesized that explicit category learning-allied to abstract category rules-would transfer robustly. This result was confirmed only for humans. That is, humans displayed explicit category knowledge that transferred flawlessly. Monkeys did not. This result illuminates the distinctive abstractness, stimulus independence, and representational portability of humans' explicit category rules. (PsycINFO Database Record

Normalizing brain activity across individuals using functional reference mapping

Neural activity can be mapped across individuals using brain atlases, but when spatial relationships are not equal, these techniques collapse. We map activity across individuals using functional registration, based on physiological responses to predetermined reference stimuli. Data from several individuals are integrated into a common multidimensional stimulus space, where dimensionality and axes are defined by these reference stimuli. We used this technique to discriminate volatile compounds with a cohort of Drosophila flies, by recording odor responses in receptor neurons on the flies’ antennae. We propose this technique for the development of reliable biological sensors when activity raw data cannot be calibrated. In particular, this technique will be useful for evaluating physiological measurements in natural chemosensory systems, and therefore will allow to exploit the sensitivity and selectivity of olfactory receptors present in the animal kingdom for analytical purposes.

Neural ideals and stimulus space visualization

A neural code C is a collection of binary vectors of a given length n that record the co-firing patterns of a set of neurons. Our focus is on neural codes arising from place cells, neurons that respond to geographic stimulus. In this setting, the stimulus space can be visualized as subset of R2 covered by a collection U of convex sets such that the arrangement U forms an Euler diagram for C. There are some methods to determine whether such a convex realization U exists; however, these methods do not describe how to draw a realization. In this work, we look at the problem of algorithmically drawing Euler diagrams for neural codes using two polynomial ideals: the neural ideal, a pseudo-monomial ideal; and the neural toric ideal, a binomial ideal. In particular, we study how these objects are related to the theory of piercings in information visualization, and we show how minimal generating sets of the ideals reveal whether or not a code is 0, 1, or 2-inductively pierced.

Closed-Loop Estimation of Retinal Network Sensitivity by Local Empirical Linearization.

Understanding how sensory systems process information depends crucially on identifying which features of the stimulus drive the response of sensory neurons, and which ones leave their response invariant. This task is made difficult by the many nonlinearities that shape sensory processing. Here, we present a novel perturbative approach to understand information processing by sensory neurons, where we linearize their collective response locally in stimulus space. We added small perturbations to reference stimuli and tested if they triggered visible changes in the responses, adapting their amplitude according to the previous responses with closed-loop experiments. We developed a local linear model that accurately predicts the sensitivity of the neural responses to these perturbations. Applying this approach to the rat retina, we estimated the optimal performance of a neural decoder and showed that the nonlinear sensitivity of the retina is consistent with an efficient encoding of stimulus information. Our approach can be used to characterize experimentally the sensitivity of neural systems to external stimuli locally, quantify experimentally the capacity of neural networks to encode sensory information, and relate their activity to behavior.

Dynamic representation of 3D auditory space in the midbrain of the free-flying echolocating bat

Our natural world is three-dimensional. A fundamental requirement of spatial orientating behaviors in the natural environment is the representation of 3D sensory space. Despite the importance of 3D sensory coding of a natural scene to guide movement, most neurophysiological investigations of this problem have been limited to studies of restrained subjects, tested with 2D, artificial stimuli. Here we show for the first time that auditory neurons in the midbrain superior colliculus of the free-flying echolocating bat encode 3D egocentric sensory space, and that sonar-guided inspection of objects in the environment sharpens spatial tuning of single neurons. Combining wireless multichannel neural recordings from free-flying bats, synchronized with video and audio data, and an echo model that computes the flying animal’s instantaneous, stimulus space, we demonstrate 3D echo-evoked receptive fields of single auditory midbrain neurons in animals orienting in a complex environment. We discovered that the bat’s ac...

Bayesian population receptive field modelling

We introduce a probabilistic (Bayesian) framework and associated software toolbox for mapping population receptive fields (pRFs) based on fMRI data. This generic approach is intended to work with stimuli of any dimension and is demonstrated and validated in the context of 2D retinotopic mapping. The framework enables the experimenter to specify generative (encoding) models of fMRI timeseries, in which experimental stimuli enter a pRF model of neural activity, which in turns drives a nonlinear model of neurovascular coupling and Blood Oxygenation Level Dependent (BOLD) response. The neuronal and haemodynamic parameters are estimated together on a voxel-by-voxel or region-of-interest basis using a Bayesian estimation algorithm (variational Laplace). This offers several novel contributions to receptive field modelling. The variance/covariance of parameters are estimated, enabling receptive fields to be plotted while properly representing uncertainty about pRF size and location. Variability in the haemodynamic response across the brain is accounted for. Furthermore, the framework introduces formal hypothesis testing to pRF analysis, enabling competing models to be evaluated based on their log model evidence (approximated by the variational free energy), which represents the optimal tradeoff between accuracy and complexity. Using simulations and empirical data, we found that parameters typically used to represent pRF size and neuronal scaling are strongly correlated, which is taken into account by the Bayesian methods we describe when making inferences. We used the framework to compare the evidence for six variants of pRF model using 7 T functional MRI data and we found a circular Difference of Gaussians (DoG) model to be the best explanation for our data overall. We hope this framework will prove useful for mapping stimulus spaces with any number of dimensions onto the anatomy of the brain.

Synaptic Mechanisms of Feature Coding in the Visual Cortex of Awake Mice

The synaptic mechanisms of feature coding in the visual cortex are poorly understood, particularly in awake animals. The ratio between excitation (E) and inhibition (I) might be constant across stimulus space, controlling only the gain and timing of neuronal responses, or it might change, directly contributing to feature coding. Whole-cell recordings in L2/3 of awake mice revealed that the E/I ratio systematically declines with increasing stimulus contrast or size. Suppressing somatostatin (SOM) neurons enhanced the E and I underlying size tuning, explaining SOM neurons' role in surround suppression. These data imply that contrast and size tuning result from a combination of a changing E/I ratio and the tuning of total synaptic input. Furthermore, they provide experimental support in awake animals for the "Stabilized Supralinear Network," a model that explains diverse cortical phenomena, and suggest that a decreasing E/I ratio with increasing cortical drive could contribute to many different cortical computations.

Alchimia

This article provides insights on an experimental artwork using a non-touch interaction interface. It details the use of a generative figurative approach and describes the audio-visual psychological and relational impact on the target-audience, through direct trial, observation and adaptation, in a first-person account, since authorship is shared between article and artwork. Alchimia is an interactive installation in which a single viewer/interactor (i.e.: the one who interacts with the artefact) stands in front of a screen and a webcam, and by facial detection and simultaneous processing of pre-prepared images and sounds, a virtual mirror space of constant audio-visual stimuli is created. The artwork aims at questioning personal representation, focusing on the face, altering the expression and gender, assigning masks, making the facial traits diffuse, mixed, unexpected, in search of another self, while allowing for self-discovery and playfulness – or intimidation – by means of a simple, familiar and almost invisible interface: the webcam.

Representation of Multidimensional Stimuli: Quantifying the Most Informative Stimulus Dimension from Neural Responses.

A common way to assess the function of sensory neurons is to measure the number of spikes produced by individual neurons while systematically varying a given dimension of the stimulus. Such measured tuning curves can then be used to quantify the accuracy of the neural representation of the stimulus dimension under study, which can in turn be related to behavioral performance. However, tuning curves often change shape when other dimensions of the stimulus are varied, reflecting the simultaneous sensitivity of neurons to multiple stimulus features. Here we illustrate how one-dimensional information analyses are misleading in this context, and propose a framework derived from Fisher information that allows the quantification of information carried by neurons in multidimensional stimulus spaces. We use this method to probe the representation of sound localization in auditory neurons of chinchillas and guinea pigs of both sexes, and show how heterogeneous tuning properties contribute to a representation of sound source position that is robust to changes in sound level.SIGNIFICANCE STATEMENT Sensory neurons' responses are typically modulated simultaneously by numerous stimulus properties, which can result in an overestimation of neural acuity with existing one-dimensional neural information transmission measures. To overcome this limitation, we develop new, compact expressions of Fisher information-derived measures that bound the robust encoding of separate stimulus dimensions in the context of multidimensional stimuli. We apply this method to the problem of the representation of sound source location in the face of changes in sound source level by neurons of the auditory midbrain.

Modelling auditory attention.

Sounds in everyday life seldom appear in isolation. Both humans and machines are constantly flooded with a cacophony of sounds that need to be sorted through and scoured for relevant information—a phenomenon referred to as the ‘cocktail party problem’. A key component in parsing acoustic scenes is the role of attention, which mediates perception and behaviour by focusing both sensory and cognitive resources on pertinent information in the stimulus space. The current article provides a review of modelling studies of auditory attention. The review highlights how the term attention refers to a multitude of behavioural and cognitive processes that can shape sensory processing. Attention can be modulated by ‘bottom-up’ sensory-driven factors, as well as ‘top-down’ task-specific goals, expectations and learned schemas. Essentially, it acts as a selection process or processes that focus both sensory and cognitive resources on the most relevant events in the soundscape; with relevance being dictated by the stimulus itself (e.g. a loud explosion) or by a task at hand (e.g. listen to announcements in a busy airport). Recent computational models of auditory attention provide key insights into its role in facilitating perception in cluttered auditory scenes.This article is part of the themed issue ‘Auditory and visual scene analysis’.

Optimal voltage stimulation parameters for network-mediated responses in wild type and rd10 mouse retinal ganglion cells

To further improve the quality of visual percepts elicited by microelectronic retinal prosthetics, substantial efforts have been made to understand how retinal neurons respond to electrical stimulation. It is generally assumed that a sufficiently strong stimulus will recruit most retinal neurons. However, recent evidence has shown that the responses of some retinal neurons decrease with excessively strong stimuli (a non-monotonic response function). Therefore, it is necessary to identify stimuli that can be used to activate the majority of retinal neurons even when such non-monotonic cells are part of the neuronal population. Taking these non-monotonic responses into consideration, we establish the optimal voltage stimulation parameters (amplitude, duration, and polarity) for epiretinal stimulation of network-mediated (indirect) ganglion cell responses.We recorded responses from 3958 mouse retinal ganglion cells (RGCs) in both healthy (wild type, WT) and a degenerating (rd10) mouse model of retinitis pigmentosa—using flat-mounted retina on a microelectrode array. Rectangular monophasic voltage-controlled pulses were presented with varying voltage, duration, and polarity.We found that in 4–5 weeks old rd10 mice the RGC thresholds were comparable to those of WT. There was a marked response variability among mouse RGCs. To account for this variability, we interpolated the percentage of RGCs activated at each point in the voltage-polarity-duration stimulus space, thus identifying the optimal voltage-controlled pulse (−2.4 V, 0.88 ms).The identified optimal voltage pulse can activate at least 65% of potentially responsive RGCs in both mouse strains. Furthermore, this pulse is well within the range of stimuli demonstrated to be safe and effective for retinal implant patients. Such optimized stimuli and the underlying method used to identify them support a high yield of responsive RGCs and will serve as an effective guideline for future in vitro investigations of retinal electrostimulation by establishing standard stimuli for each unique experimental condition.