Representation Of Stimuli Research Articles

Spontaneous repulsive adaptation in the absence of attractive serial dependence.

Despite noisy and discontinuous input, vision is remarkably stable and continuous. Recent work suggests that such a remarkable feat is enabled by an active stabilization process integrating information over time, resulting in attractive serial dependence. However, precise mechanisms underlying serial dependence are still unknown. Across three psychophysical experiments, we demonstrate that suppressing high-level modulatory signals on early cortical activity via visual backward masking completely abolishes the serial dependence effect, indicating the critical role of cortical feedback processing on serial dependence. Moreover, we show that the absence of modulatory feedback results in a robust repulsive aftereffect, as in perceptual adaptation, after only 50 ms of stimulation, indicating the presence of a local neurocomputational process for an automatic and spontaneous recalibration of the stimulus representation. These findings collectively illustrate the interplay between two contrasting cortical mechanisms at short timescales that serve as a basis for our perceptual experience.

Effects of arousal on biased competition in attention and short-term memory.

A recent theory proposes that arousal amplifies the competition between stimulus representations, strengthening already strong representations and weakening already weak representations in perception and memory. Here, we report a stringent test of this arousal-biased competition theory in the context of visual attention and short-term memory. We examined whether pre-trial arousal enhances the bottom-up attentional bias toward physically salient versus less salient stimuli in a multi-letter identification task. Arousal was manipulated by presenting an arousing versus a neutral picture (Experiment 1) or sound (Experiment 2) at the start of each trial. Bayesian statistics revealed strong evidence for the null hypothesis in both experiments: Arousal did not modulate the effects of physical salience on letter identification. The experiments were repeated with EEG measurements and subjective stimulus ratings, which confirmed that the stimuli successfully manipulated physiological and subjective arousal. These results pose a challenge for the arousal-biased competition theory.

Measuring the fidelity and connectivity of stimulus representations provides a richer neural characterization of attentional fluctuations

Measuring the fidelity and connectivity of stimulus representations provides a richer neural characterization of attentional fluctuations

Causal contributions of parietal cortex to perceptual decision-making during stimulus categorization.

The posterior parietal cortex (PPC) has been implicated in perceptual decision-making and categorization, but whether its activity plays a causal role remains controversial. Here we examined the population dynamics of PPC activity during an auditory-guided decision task in mice. We found that silencing of PPC activity impaired several aspects of decision-making. First, categorization of new, but not well-learned, stimuli was impaired. Second, re-categorization of previously experienced stimuli based on newly learned categories was also impaired. Third, the bias on behavioral choices created by preceding trials significantly increased. In vivo two-photon imaging of PPC activity during stimulus categorization revealed differential dynamics in representations of new stimuli and learned categories, consistent with rapid incorporation of new sensory information during categorization. At the circuit level, inactivation of PPC axonal projections to the auditory cortex also significantly reduced categorization performance. Thus, PPC circuits play a causal role in decision-making during stimulus categorization.

The relative contribution of color and material in object selection.

Object perception is inherently multidimensional: information about color, material, texture and shape all guide how we interact with objects. We developed a paradigm that quantifies how two object properties (color and material) combine in object selection. On each experimental trial, observers viewed three blob-shaped objects—the target and two tests—and selected the test that was more similar to the target. Across trials, the target object was fixed, while the tests varied in color (across 7 levels) and material (also 7 levels, yielding 49 possible stimuli). We used an adaptive trial selection procedure (Quest+) to present, on each trial, the stimulus test pair that is most informative of underlying processes that drive selection. We present a novel computational model that allows us to describe observers’ selection data in terms of (1) the underlying perceptual stimulus representation and (2) a color-material weight, which quantifies the relative importance of color vs. material in selection. We document large individual differences in the color-material weight across the 12 observers we tested. Furthermore, our analyses reveal limits on how precisely selection data simultaneously constrain perceptual representations and the color-material weight. These limits should guide future efforts towards understanding the multidimensional nature of object perception.

Perceptual Pluralism

AbstractPerceptual systems respond to proximal stimuli by forming mental representations of distal stimuli. A central goal for the philosophy of perception is to characterize the representations delivered by perceptual systems. It may be that all perceptual representations are in some way proprietarily perceptual and differ from the representational format of thought (Dretske 1981; Carey 2009; Burge 2010; Block ms.). Or it may instead be that perception and cognition always trade in the same code (Prinz 2002; Pylyshyn 2003). This paper rejects both approaches in favor ofperceptual pluralism, the thesis that perception delivers a multiplicity of representational formats, some proprietary and some shared with cognition. The argument for perceptual pluralism marshals a wide array of empirical evidence in favor of iconic (i.e., image‐like, analog) representations in perception as well as discursive (i.e., language‐like, digital) perceptual object representations.

A computational-observer model of spatial contrast sensitivity: Effects of wave-front-based optics, cone-mosaic structure, and inference engine.

We present a computational-observer model of the human spatial contrast-sensitivity function based on the Image Systems Engineering Toolbox for Biology (ISETBio) simulation framework. We demonstrate that ISETBio-derived contrast-sensitivity functions agree well with ones derived using traditional ideal-observer approaches, when the mosaic, optics, and inference engine are matched. Further simulations extend earlier work by considering more realistic cone mosaics, more recent measurements of human physiological optics, and the effect of varying the inference engine used to link visual representations to psychophysical performance. Relative to earlier calculations, our simulations show that the spatial structure of realistic cone mosaics reduces the upper bounds on performance at low spatial frequencies, whereas realistic optics derived from modern wave-front measurements lead to increased upper bounds at high spatial frequencies. Finally, we demonstrate that the type of inference engine used has a substantial effect on the absolute level of predicted performance. Indeed, the performance gap between an ideal observer with exact knowledge of the relevant signals and human observers is greatly reduced when the inference engine has to learn aspects of the visual task. ISETBio-derived estimates of stimulus representations at various stages along the visual pathway provide a powerful tool for computing the limits of human performance.

A Temporal Neural Trace of Wavelet Coefficients in Human Object Vision: An MEG Study.

Wavelet transform has been widely used in image and signal processing applications such as denoising and compression. In this study, we explore the relation of the wavelet representation of stimuli with MEG signals acquired from a human object recognition experiment. To investigate the signature of wavelet descriptors in the visual system, we apply five levels of multi-resolution wavelet decomposition to the stimuli presented to participants during MEG recording and extract the approximation and detail sub-bands (horizontal, vertical, diagonal) coefficients in each level of decomposition. Apart from, employing multivariate pattern analysis (MVPA), a linear support vector classifier (SVM) is trained and tested over the time on MEG pattern vectors to decode neural information. Then, we calculate the representational dissimilarity matrix (RDM) on each time point of the MEG data and also on wavelet descriptors using classifier accuracy and one minus Pearson correlation coefficient, respectively. Given the time-courses calculated from performing the Pearson correlation between the wavelet descriptors RDMs and MEG decoding accuracy in each time point, our result shows that the peak latency of the wavelet approximation time courses occurs later for higher level coefficients. Furthermore, studying the neural trace of detail sub-bands indicates that the overall number of statistically significant time points for the horizontal and vertical detail coefficients is noticeably higher than diagonal detail coefficients, confirming the evidence of the oblique effect that the horizontal and vertical lines are more decodable in the human brain.

Effect of ripple repetition rate on discrimination of ripple glide direction and the detection of brief tones in spectro-temporal ripple noise.

The effect of temporal repetition rate R on the discrimination and internal representation of stimuli with spectro-temporal ripples was examined. Experiment 1 measured the highest ripple density D at which upward- and downward-gliding ripples could be discriminated. Thresholds varied only slightly for R from 2 to 8 Hz, with a median threshold just above 5 ripples/oct. The threshold decreased (worsened) when R was increased to 16 and 32 Hz, suggesting that the limited temporal resolution of the auditory system plays a role for these higher values of R. Experiment 2 explored the internal representation of stimuli with static and downward-gliding spectral ripples by measuring the detection threshold for a brief tone presented at a peak or a valley in the stimulus spectrum. Thresholds were generally higher when the signal was at a peak than when it was at a valley. The peak-valley difference tended to decrease with increasing D, and the variation of thresholds with D was greater for low R than for high R. The results suggest that the discrimination of spectro-temporal ripples is limited mainly by frequency resolution for lower ripple rates (up to 4-8 Hz) but temporal resolution plays a major role for higher rates.

Weighting neurons by selectivity produces near-optimal population codes

Perception is produced by "reading out" the representation of a sensory stimulus contained in the activity of a population of neurons. To examine experimentally how populations code information, a common approach is to decode a linearly weighted sum of the neurons' spike counts. This approach is popular because of the biological plausibility of weighted, nonlinear integration. For neurons recorded in vivo, weights are highly variable when derived through optimization methods, but it is unclear how the variability affects decoding performance in practice. To address this, we recorded from neurons in the middle temporal area (MT) of anesthetized marmosets (Callithrix jacchus) viewing stimuli comprising a sheet of dots that moved coherently in 1 of 12 different directions. We found that high peak response and direction selectivity both predicted that a neuron would be weighted more highly in an optimized decoding model. Although learned weights differed markedly from weights chosen according to a priori rules based on a neuron's tuning profile, decoding performance was only marginally better for the learned weights. In the models with a priori rules, selectivity is the best predictor of weighting, and defining weights according to a neuron's preferred direction and selectivity improves decoding performance to very near the maximum level possible, as defined by the learned weights. NEW & NOTEWORTHY We examined which aspects of a neuron's tuning account for its contribution to sensory coding. Strongly direction-selective neurons are weighted most highly by optimal decoders trained to discriminate motion direction. Models with predefined decoding weights demonstrate that this weighting scheme causally improved direction representation by a neuronal population. Optimizing decoders (using a generalized linear model or Fisher's linear discriminant) led to only marginally better performance than decoders based purely on a neuron's preferred direction and selectivity.

Inhibition increases response variability and reduces stimulus discrimination in random networks of cortical neurons

Much of what is known about the contribution of inhibition to stimulus discrimination is due to extensively studied sensory systems, which are highly structured neural circuits. The effect of inhibition on stimulus representation in less structured networks is not as clear. Here we exercise a biosynthetic approach in order to study the impacts of inhibition on stimulus representation in non-specialized network anatomy. Combining pharmacological manipulation, multisite electrical stimulation and recording from ex-vivo randomly rewired networks of cortical neurons, we quantified the effects of inhibition on response variability and stimulus discrimination at the population and single unit levels. We find that blocking inhibition quenches variability of responses evoked by repeated stimuli and enhances discrimination between stimuli that invade the network from different spatial loci. Enhanced stimulus discrimination is reserved for representation schemes that are based on temporal relation between spikes emitted in groups of neurons. Our data indicate that – under intact inhibition – the response to a given stimulus is a noisy version of the response evoked in the absence of inhibition. Spatial analysis suggests that the dispersion effect of inhibition is due to disruption of an otherwise coherent, wave-like propagation of activity.

Multiple Timescale Online Learning Rules for Information Maximization with Energetic Constraints

A key aspect of the neural coding problem is understanding how representations of afferent stimuli are built through the dynamics of learning and adaptation within neural networks. The infomax paradigm is built on the premise that such learning attempts to maximize the mutual information between input stimuli and neural activities. In this letter, we tackle the problem of such information-based neural coding with an eye toward two conceptual hurdles. Specifically, we examine and then show how this form of coding can be achieved with online input processing. Our framework thus obviates the biological incompatibility of optimization methods that rely on global network awareness and batch processing of sensory signals. Central to our result is the use of variational bounds as a surrogate objective function, an established technique that has not previously been shown to yield online policies. We obtain learning dynamics for both linear-continuous and discrete spiking neural encoding models under the umbrella of linear gaussian decoders. This result is enabled by approximating certain information quantities in terms of neuronal activity via pairwise feedback mechanisms. Furthermore, we tackle the problem of how such learning dynamics can be realized with strict energetic constraints. We show that endowing networks with auxiliary variables that evolve on a slower timescale can allow for the realization of saddle-point optimization within the neural dynamics, leading to neural codes with favorable properties in terms of both information and energy.

Overlapping and distinct contributions of stimulus location and of spatial context to nonspatial visual short-term memory.

Stimulus location is not always informative during visual short-term memory (VSTM) for nonspatial features. Nevertheless, there is considerable evidence for the automatic encoding and retention of location information, regardless of its task relevance. To explore the functional and neural bases of the representation of spatial context in VSTM for nonspatial information, functional magnetic resonance imaging was performed while subjects performed delayed recall for the orientation of individual stimuli. Stimulus location varied across trials, and although this information was irrelevant for task performance, multivariate pattern analysis decoding of stimulus location sustained across trials, and also the decoding strength, predicted the precision of the recall of orientation. The influence of spatial context on the representation of orientation was operationalized by comparing the orientation reconstructions with multivariate inverted encoding models (IEM) trained in location context-dependent vs. -independent data. Although orientation reconstructions were robust for both location-dependent and location-independent IEMs, they were markedly stronger for the former. Furthermore, the functional relevance of location context was demonstrated by the fact that only the location-dependent neural representations of stimulus orientation predicted recall precision. NEW & NOTEWORTHY Neural representation strength of stimulus location predicts the precision of visual short-term memory (VSTM) recall of nonspatial stimulus, even when this information is task irrelevant. Neural representations of nonspatial stimuli that incorporate location context are stronger than those that do not, and only the former representations are strongly linked to behavior. The contributions to nonspatial VSTM performance of the representation of location context are at least partly distinct from those of the representation of stimulus content.

Bidirectional decay of auditory memory traces for pitch in speech sounds

Previous studies have shown human listeners have greater detection sensitivity to pitch increment than decrement in successive sounds. Assuming deviance detection is based on the comparison between the memory trace of a recent stimulus and the neural representation of a new stimulus, one hypothesis is that this differential sensitivity between increment and decrement is caused by the downward decay of pitch’s memory trace. Under the same assumption, the present study found bidirectional—as opposed to the predicted unidirectional—decay of memory traces for pitch in speech sounds by measuring listeners' sensitivity to pitch change over varying time intervals with an AX discrimination task. Three properties in the AX task were randomly sampled from preset ranges: (1) f0 of the A token (163:320 Hz), (2) difference between the f0s of A and X (-30:30Hz), and (3) the inter-stimulus interval (ISI; 0:3 s). We found when the stimuli were in the lower portion of the speaker’s pitch range, listeners were less sensitive to pitch increments at larger ISIs. Meanwhile, when the stimuli were in the higher pitch range, listeners were less sensitive to pitch decrements at larger ISIs. These results suggest memory traces for pitch in speech sounds decay toward a center pitch.

Critical examination of critical bands

The concept of the critical band has driven the design and interpretation of psychophysical experiments for decades. The critical band was initially proposed by Fletcher [Rev. Mod. Phys. 12, 47 (1940)] to describe results of masking experiments, and it is generally interpreted as evidence for a filter that limits the signal-to-noise ratio in stimulus representations that are provided to the central nervous system. This filter-based interpretation of masked-detection thresholds has been extended and applied to numerous tasks, including many without masking. The concept of a bank of critical-band filters also forms the basis for signal-processing strategies used for hearing aids and cochlear implants. This talk will review both psychophysical and physiological results that challenge the concept of the critical-band filter. For example, the minimal effect of roving-level paradigms on masked-detection thresholds directly refutes the critical-band based power-spectrum model. Furthermore, most auditory neurons have receptive fields that are much broader than critical bands at the moderate to high sound levels used in most psychophysical tasks, with or without masking. Alternative concepts that are robust across a wide range of levels and in roving-level paradigms, such as fluctuation profiles, explain the original masked-detection results and related experiments, such as psychophysical tuning curves.

Consequences of auditory experience and cochlear implant stimulation on tuning and other measures obtained in pre-lingually deaf children and postlingually deaf adults

Children and adults with moderate to severe hearing loss may obtain cochlear implants (CIs) to restore auditory perception, but auditory development differs among them. We have compared various peripheral measures across CI listeners differing in etiology, age of onset and duration of hearing loss, and duration of CI use. Several measures assess the efficacy with which CI electrodes activate their target auditory neurons in individuals with varying hearing demographics. In addition to peripheral contributions to auditory perception, central reorganization might occur when the auditory nerve is stimulated with coarse, electrical input from CIs. Evidence from neurophysiological studies in cats suggest that the central representation of spectral/spatial resolution is altered by chronic CI stimulation. In humans, psychophysical tuning curves might reflect both the spread of electrical current in the cochlea and the central representation of electrical stimuli. Understanding how chronic, electrical stimulation during auditory system development affects spectral resolution may be useful for optimizing CI programming in children and adults.

Inverted Encoding Models Reconstruct an Arbitrary Model Response, Not the Stimulus.

Probing how large populations of neurons represent stimuli is key to understanding sensory representations as many stimulus characteristics can only be discerned from population activity and not from individual single-units. Recently, inverted encoding models have been used to produce channel response functions from large spatial-scale measurements of human brain activity that are reminiscent of single-unit tuning functions and have been proposed to assay “population-level stimulus representations” (Sprague et al., 2018a). However, these channel response functions do not assay population tuning. We show by derivation that the channel response function is only determined up to an invertible linear transform. Thus, these channel response functions are arbitrary, one of an infinite family and therefore not a unique description of population representation. Indeed, simulations demonstrate that bimodal, even random, channel basis functions can account perfectly well for population responses without any underlying neural response units that are so tuned. However, the approach can be salvaged by extending it to reconstruct the stimulus, not the assumed model. We show that when this is done, even using bimodal and random channel basis functions, a unimodal function peaking at the appropriate value of the stimulus is recovered which can be interpreted as a measure of population selectivity. More precisely, the recovered function signifies how likely any value of the stimulus is, given the observed population response. Whether an analysis is recovering the hypothetical responses of an arbitrary model rather than assessing the selectivity of population representations is not an issue unique to the inverted encoding model and human neuroscience, but a general problem that must be confronted as more complex analyses intervene between measurement of population activity and presentation of data.

The Graded Fate of Unattended Stimulus Representations in Visuospatial Working Memory.

As in visual perception, information can be selected for prioritized processing at the expense of unattended representations in visual working memory (VWM). However, what is not clear is whether and how this prioritization degrades the unattended representations. We addressed two hypotheses. First, the representational quality of unattended items could be degraded as a function of the spatial distance to attended information in VWM. Second, the strength with which an item is bound to its location is degraded as a function of the spatial distance to attended information in VWM. To disentangle these possibilities, we designed an experiment in which participants performed a continuous production task in which they memorized a visual array with colored discs, one of which was spatially retro-cued, informing the target location of an impending probe that was to be recalled (Experiment 1). We systematically varied the spatial distance between the cued and probed locations and obtained model-based estimates of the representational quality and binding strengths at varying cue-probe distances. Although the representational quality of the unattended representations remained unaffected by the cue-probe distance, spatially graded binding strengths were observed, as reflected in more spatial confusions at smaller cue-probe distances. These graded binding strengths were further replicated with a model-free approach in a categorical version of the production task in which stimuli and responses consisted of easily discriminable colors (Experiment 2). These results demonstrate that unattended representations are prone to spatial confusions due to spatial degradation of binding strengths in WM, even though they are stored with the same representational quality.

Effects of Stimulus Exploration Length and Time on the Integration of Information in Haptic Softness Discrimination.

In haptic perception, information is often sampled serially (e.g., a stimulus is repeatedly indented to estimate its softness), requiring that sensory information is retained and integrated over time. Hence, integration of sequential information is likely affected by memory. Particularly, when two sequentially explored stimuli are compared, integration of information on the second stimulus might be determined by the fading representation of the first stimulus. We investigated how the exploration length of the first stimulus and a temporal delay affect contributions of sequentially gathered estimates of the second stimulus in haptic softness discrimination. Participants subsequently explored two silicon rubber stimuli by indenting the first stimulus one or five times and the second stimulus always three times. In an additional experiment, we introduced a 5-s delay after the first stimulus was indented five times. We show that the longer the first stimulus is explored, the more estimates of the second stimulus' softness contribute to the discrimination of the two stimuli, independent of the delay. This suggests that the exploration length of the first stimulus influences the strength of its representation, persisting at least for 5s, and determines how much information about the second stimulus is exploited for the comparison.

The representation of symmetry in multi-voxel response patterns and functional connectivity throughout the ventral visual stream

Several computational models explain how symmetry might be detected and represented in the human brain. However, while there is an abundance of psychophysical studies on symmetry detection and several neural studies showing where and when symmetry is detected in the brain, important questions remain about how this detection happens and how symmetric patterns are represented. We studied the representation of (vertical) symmetry in regions of the ventral visual stream, using multi-voxel pattern analyses (MVPA) and functional connectivity analyses. Our results suggest that neural representations gradually change throughout the ventral visual stream, from very similar part-based representations for symmetrical and asymmetrical stimuli in V1 and V2, over increasingly different representations for symmetrical and asymmetrical stimuli which are nevertheless still part-based in both V3 and V4, to a more holistic representation for symmetrical compared to asymmetrical stimuli in high-level LOC. This change in representations is accompanied by increased communication between left and right retinotopic areas, evidenced by higher interhemispheric functional connectivity during symmetry perception in areas V2 and V4.