Reverse Correlation Analysis Research Articles

Neural Basis of Perceptual Surface Qualities and Materials: Evidence from EEG Decoding

Abstract The human visual system can easily recognize object material categories and estimate surface properties such as glossiness and smoothness. Recent psychophysical and computational studies suggest that the material perception depends on global feature statistics. To elucidate dynamic neural representations underlying surface property and material perception in humans, we measured visual evoked potentials (VEPs) for 191 natural images consisting of 20 categories of materials and then classified material categories and surface properties from the VEPs. As a result, we found that material categories were correctly classified by the VEPs even at latencies of 150 msec or less. The apparent surface properties were also significantly classified within 175 msec (lightness, colorfulness, and smoothness) and after 200 msec (glossiness, hardness, and heaviness). The subsequent reverse-correlation analysis revealed that the VEPs at these latencies are highly correlated with low- and high-level global feature statistics of the surface images, indicating that neural activities about such global features are reflected in the VEPs. Moreover, by using deep generative models (multimodal variational autoencoder models) to reconstruct surface images from the VEPs via style information, we demonstrated that the reconstructed surface images are judged by observers to have very similar material categories and surface properties as the original natural surfaces. These results support the notion that neural representations of statistical features in the early cortical response play a crucial role in the perception and recognition of surface materials in humans.

Identification of movie encoding neurons enables movie recognition AI

Natural visual scenes are dominated by spatiotemporal image dynamics, but how the visual system integrates "movie" information over time is unclear. We characterized optic tectal neuronal receptive fields using sparse noise stimuli and reverse correlation analysis. Neurons recognized movies of ~200-600 ms durations with defined start and stop stimuli. Movie durations from start to stop responses were tuned by sensory experience though a hierarchical algorithm. Neurons encoded families of image sequences following trigonometric functions. Spike sequence and information flow suggest that repetitive circuit motifs underlie movie detection. Principles of frog topographic retinotectal plasticity and cortical simple cells are employed in machine learning networks for static image recognition, suggesting that discoveries of principles of movie encoding in the brain, such as how image sequences and duration are encoded, may benefit movie recognition technology. We built and trained a machine learning network that mimicked neural principles of visual system movie encoders. The network, named MovieNet, outperformed current machine learning image recognition networks in classifying natural movie scenes, while reducing data size and steps to complete the classification task. This study reveals how movie sequences and time are encoded in the brain and demonstrates that brain-based movie processing principles enable efficient machine learning.

A simple psychophysical procedure separates representational and noise components in impairments of speech prosody perception after right-hemisphere stroke

After a right hemisphere stroke, more than half of the patients are impaired in their capacity to produce or comprehend speech prosody. Yet, and despite its social-cognitive consequences for patients, aprosodia following stroke has received scant attention. In this report, we introduce a novel, simple psychophysical procedure which, by combining systematic digital manipulations of speech stimuli and reverse-correlation analysis, allows estimating the internal sensory representations that subtend how individual patients perceive speech prosody, and the level of internal noise that govern behavioral variability in how patients apply these representations. Tested on a sample of N = 22 right-hemisphere stroke survivors and N = 21 age-matched controls, the representation + noise model provides a promising alternative to the clinical gold standard for evaluating aprosodia (MEC): both parameters strongly associate with receptive, and not expressive, aprosodia measured by MEC within the patient group; they have better sensitivity than MEC for separating high-functioning patients from controls; and have good specificity with respect to non-prosody-related impairments of auditory attention and processing. Taken together, individual differences in either internal representation, internal noise, or both, paint a potent portrait of the variety of sensory/cognitive mechanisms that can explain impairments of prosody processing after stroke.

Stimulus-dependent differences in cortical versus subcortical contributions to visual detection in mice

The primary visual cortex (V1) and the superior colliculus (SC) both occupy stations early in the processing of visual information. They have long been thought to perform distinct functions, with the V1 supporting the perception of visual features and the SC regulating orienting to visual inputs. However, growing evidence suggests that the SC supports the perception of many of the same visual features traditionally associated with the V1. To distinguish V1 and SC contributions to visual processing, it is critical to determine whether both areas causally contribute to the detection of specific visual stimuli. Here, mice reported changes in visual contrast or luminance near their perceptual threshold while white noise patterns of optogenetic stimulation were delivered to V1 or SC inhibitory neurons. We then performed a reverse correlation analysis on the optogenetic stimuli to estimate a neuronal-behavioral kernel (NBK), a moment-to-moment estimate of the impact of V1 or SC inhibition on stimulus detection. We show that the earliest moments of stimulus-evoked activity in the SC are critical for the detection of both luminance and contrast changes. Strikingly, there was a robust stimulus-aligned modulation in the V1 contrast-detection NBK but no sign of a comparable modulation for luminance detection. The data suggest that behavioral detection of visual contrast depends on both V1 and SC spiking, whereas mice preferentially use SC activity to detect changes in luminance. Electrophysiological recordings showed that neurons in both the SC and V1 responded strongly to both visual stimulus types, while the reverse correlation analysis reveals when these neuronal signals actually contribute to visually guided behaviors.

Perceptual confirmation bias and decision bias underlie adaptation to sequential regularities.

Our perception does not depend exclusively on the immediate sensory input. It is also influenced by our internal predictions derived from prior observations and the temporal regularities of the environment, which can result in choice history biases. However, it is unclear how this flexible use of prior information to predict the future influences perceptual decisions. Prior information may bias decisions independently of the current sensory input, or it may modulate the weight of current sensory input based on its consistency with the expectation. To address this question, we used a visual decision-making task and manipulated the transitional probabilities between successive noisy grating stimuli. Using a reverse correlation analysis, we evaluated the contribution of stimulus-independent decision bias and stimulus-dependent sensitivity modulations to choice history biases. We found that both effects coexist, whereby there was increased bias to respond in line with the predicted orientation alongside modulations in perceptual sensitivity to favor perceptual information consistent with the prediction, akin to selective attention. Furthermore, at the individual differences level, we investigated the relationship between autistic-like traits and the adaptation of choice history biases to the sequential statistics of the environment. Over two studies, we found no convincing evidence of reduced adaptation to sequential regularities in individuals with high autistic-like traits. In sum, we present robust evidence for both perceptual confirmation bias and decision bias supporting adaptation to sequential regularities in the environment.

The MADS-box gene EjAGL15 positively regulates lignin deposition in the flesh of loquat fruit during its storage.

Lignification of fruit flesh is a common physiological disorder that occurs during post-harvest storage, resulting in the deterioration of fruit quality. Lignin deposition in loquat fruit flesh occurs due to chilling injury or senescence, at temperatures around 0°C or 20°C, respectively. Despite extensive research on the molecular mechanisms underlying chilling-induced lignification, the key genes responsible for the lignification process during senescence in loquat fruit remain unknown. MADS-box genes, an evolutionarily conserved transcription factor family, have been suggested to play a role in regulating senescence. However, it is still unclear whether MADS-box genes can regulate the lignin deposition that arises from fruit senescence. Both senescence- and chilling-induced flesh lignification were simulated by applying temperature treatments on loquat fruits. The flesh lignin content during the storage was measured. Transcriptomic, quantitative reverse transcription PCR and correlation analysis were employed to identify key MADS-box genes that may be involved in flesh lignification. The Dual-luciferase assay was utilized to identify the potential interactions between MADS-box members and genes in phenylpropanoid pathway. The lignin content of the flesh samples treated at 20°C or 0°C increased during storage, but at different rates. Results from transcriptome analysis, quantitative reverse transcription PCR, and correlation analysis led us to identify a senescence-specific MADS-box gene, EjAGL15, which correlated positively with the variation in lignin content of loquat fruit. Luciferase assay results confirmed that EjAGL15 activated multiple lignin biosynthesis-related genes. Our findings suggest that EjAGL15 functions as a positive regulator of senescence-induced flesh lignification in loquat fruit.

Perceptual Weighting of V1 Spikes Revealed by Optogenetic White Noise Stimulation.

During visually guided behaviors, mere hundreds of milliseconds can elapse between a sensory input and its associated behavioral response. How spikes occurring at different times are integrated to drive perception and action remains poorly understood. We delivered random trains of optogenetic stimulation (white noise) to excite inhibitory interneurons in V1 of mice of both sexes while they performed a visual detection task. We then performed a reverse correlation analysis on the optogenetic stimuli to generate a neuronal-behavioral kernel, an unbiased, temporally precise estimate of how suppression of V1 spiking at different moments around the onset of a visual stimulus affects detection of that stimulus. Electrophysiological recordings enabled us to capture the effects of optogenetic stimuli on V1 responsivity and revealed that the earliest stimulus-evoked spikes are preferentially weighted for guiding behavior. These data demonstrate that white noise optogenetic stimulation is a powerful tool for understanding how patterns of spiking in neuronal populations are decoded in generating perception and action.SIGNIFICANCE STATEMENT During visually guided actions, continuous chains of neurons connect our retinas to our motoneurons. To unravel circuit contributions to behavior, it is crucial to establish the relative functional position(s) that different neural structures occupy in processing and relaying the signals that support rapid, precise responses. To address this question, we randomly inhibited activity in mouse V1 throughout the stimulus-response cycle while the animals did many repetitions of a visual task. The period that led to impaired performance corresponded to the earliest stimulus-driven response in V1, with no effect of inhibition immediately before or during late stages of the stimulus-driven response. This approach offers experimenters a powerful method for uncovering the temporal weighting of spikes from stimulus to response.

Probing temporal modulation detection in white noise using intrinsic envelope fluctuations: A reverse-correlation study.

Part of the detrimental effect caused by a stationary noise on sound perception results from the masking of relevant amplitude modulations (AM) in the signal by random intrinsic envelope fluctuations arising from the filtering of noise by cochlear channels. This study capitalizes on this phenomenon to probe AM detection strategies for human listeners using a reverse correlation analysis. Eight normal-hearing listeners were asked to detect the presence of a 4-Hz sinusoidal AM target applied to a 1-kHz tone carrier using a yes-no task with 3000 trials/participant. All stimuli were embedded in a white-noise masker. A reverse-correlation analysis was then carried on the data to compute "psychophysical kernels" showing which aspects of the stimulus' temporal envelope influenced the listener's responses. These results were compared to data simulated with different implementations of a modulation-filterbank model. Psychophysical kernels revealed that human listeners were able to track the position of AM peaks in the target, similar to the models. However, they also showed a marked temporal decay and a consistent phase shift compared to the ideal template. In light of the simulated data, this was interpreted as an evidence for the presence of phase uncertainty in the processing of intrinsic envelope fluctuations.

It’s not what you say, it’s how you say it: A retrospective study of the impact of prosody on own-name P300 in comatose patients

ObjectiveThe acoustic characteristics of stimuli influence the characteristics of the corresponding evoked potentials in healthy subjects. Own-name stimuli are used in clinical practice to assess the level of consciousness in intensive care units. The influence of the acoustic variability of these stimuli has never been evaluated. Here, we explored the influence of this variability on the characteristics of the subject’s own name (SON) P300. MethodsWe retrospectively analyzed 251 disorders of consciousness patients from Lyon and Paris Hospitals who underwent an “own-name protocol”. A reverse correlation analysis was performed to test for an association between acoustic properties of own-names stimuli used and the characteristics of the P300 wave observed. ResultsOwn-names pronounced with increasing pitch prosody showed P300 responses 66 ms earlier than own-names that had a decreasing prosody [IC95% = 6.36; 125.9 ms]. ConclusionsSpeech prosody of the stimuli in the “own name protocol” is associated with latencies differences of the P300 response among patients for whom these responses were observed. Further investigations are needed to confirm these results. SignificanceSpeech prosody of the stimuli in the “own name protocol” is a non-negligible parameter, associated with P300 latency differences. Speech prosody should be standardized in SON P300 studies.

Reverse-correlation reveals internal error-corrections during information-seeking

In the area of metacognition research, different methods have been used to study participants’ subjective sense of confidence in their choices. Among the most often used methods are explicit reports of subjective confidence, post-decision wagering and measuring additional info-seeking behavior. While all three methods are thought to measure confidence, they differ greatly in terms of practical execution and theoretical foundation. The method of reverse correlation has previously been used to determine which aspects of the stimulus influence decisions and confidence judgments. Here we compare the three methods of confidence assessment using reverse correlation analysis. Explicit reports and post-decision wagering revealed a positive association of stimulus information with choices and reduced decision weights for low-confidence trials. When confidence was assessed using the info-seeking method, low-confidence trials showed an inverted association with primary stimulus information. Using modelling of the behavioral data, we show how the reverse correlation results of all three methods can be explained by a simple model of confidence when internal error-corrections are allowed during seeking of additional information.

Transsaccadic integration of visual information is predictive, attention-based, and spatially precise.

Eye movements produce shifts in the positions of objects in the retinal image, but observers are able to integrate these shifting retinal images into a coherent representation of visual space. This ability is thought to be mediated by attention-dependent saccade-related neural activity that is used by the visual system to anticipate the retinal consequences of impending eye movements. Previous investigations of the perceptual consequences of this predictive activity typically infer attentional allocation using indirect measures such as accuracy or reaction time. Here, we investigated the perceptual consequences of saccades using an objective measure of attentional allocation, reverse correlation. Human observers executed a saccade while monitoring a flickering target object flanked by flickering distractors and reported whether the average luminance of the target was lighter or darker than the background. Successful task performance required subjects to integrate visual information across the saccade. A reverse correlation analysis yielded a spatiotemporal “psychophysical kernel” characterizing how different parts of the stimulus contributed to the luminance decision throughout each trial. Just before the saccade, observers integrated luminance information from a distractor located at the post-saccadic retinal position of the target, indicating a predictive perceptual updating of the target. Observers did not integrate information from distractors placed in alternative locations, even when they were nearer to the target object. We also observed simultaneous predictive perceptual updating for two spatially distinct targets. These findings suggest both that shifting neural representations mediate the coherent representation of visual space, and that these shifts have significant consequences for transsaccadic perception.

Analysis and Synthesis of Natural Texture Perception From Visual Evoked Potentials.

The primate visual system analyzes statistical information in natural images and uses it for the immediate perception of scenes, objects, and surface materials. To investigate the dynamical encoding of image statistics in the human brain, we measured visual evoked potentials (VEPs) for 166 natural textures and their synthetic versions, and performed a reverse-correlation analysis of the VEPs and representative texture statistics of the image. The analysis revealed occipital VEP components strongly correlated with particular texture statistics. VEPs correlated with low-level statistics, such as subband SDs, emerged rapidly from 100 to 250 ms in a spatial frequency dependent manner. VEPs correlated with higher-order statistics, such as subband kurtosis and cross-band correlations, were observed at slightly later times. Moreover, these robust correlations enabled us to inversely estimate texture statistics from VEP signals via linear regression and to reconstruct texture images that appear similar to those synthesized with the original statistics. Additionally, we found significant differences in VEPs at 200–300 ms between some natural textures and their Portilla–Simoncelli (PS) synthesized versions, even though they shared almost identical texture statistics. This differential VEP was related to the perceptual “unnaturalness” of PS-synthesized textures. These results suggest that the visual cortex rapidly encodes image statistics hidden in natural textures specifically enough to predict the visual appearance of a texture, while it also represents high-level information beyond image statistics, and that electroencephalography can be used to decode these cortical signals.

Neural mechanism of spatio-chromatic opponency in the Drosophila amacrine neurons

Visual animals detect spatial variations of light intensity and wavelength composition. Opponent coding is a common strategy for reducing information redundancy. Neurons equipped with both spatial and spectral opponency have been identified in vertebrates but not yet in insects. The Drosophila amacrine neuron Dm8 was recently reported to show color opponency. Here, we demonstrate Dm8 exhibits spatio-chromatic opponency. Antagonistic convergence of the direct input from the UV-sensing R7s and indirect input from the broadband receptors R1-R6 through Tm3 and Mi1 is sufficient to confer Dm8's UV/Vis (ultraviolet/visible light) opponency. Using high resolution monochromatic stimuli, we show the pale and yellow subtypes of Dm8s, inheriting retinal mosaic characteristics, have distinct spectral tuning properties. Using 2D white-noise stimulus and reverse correlation analysis, we found that the UV receptive field (RF) of Dm8 has a center-inhibition/surround-excitation structure. In the absence of UV-sensing R7 inputs, the polarity of the RF is inverted owing to the excitatory input from the broadband photoreceptors R1-R6. Using a new synGRASP method based on endogenous neurotransmitter receptors, we show that neighboring Dm8s form mutual inhibitory connections mediated by the glutamate-gated chloride channel GluClα, which is essential for both Dm8's spatial opponency and animals' phototactic behavior. Our study shows spatio-chromatic opponency could arise in the early visual stage, suggesting a common information processing strategy in both invertebrates and vertebrates.

Improving reverse correlation analysis of faces: Diagnostics of order effects, runs, rater agreement, and image pairs.

Examinations of the reliability and validity of classification images of faces using the reverse correlation approach remain rare. In the present paper, we focus on order effects of trials, compliance, and reliability effects, as well as the degree of contextual contrast of image pairs. We present different diagnostic methods to examine these three aspects using data from 12 reverse correlation studies conducted both in-lab and online with diverse samples (i.e., from Burkina Faso, China, the Netherlands, the U.S., and an international sample) using five different base faces (i.e., female black, female Asian, female and gender-neutral white, and black/white/female/male morphed composite). For each of the 12 studies, we compare the individual CIs of subgroups of likely non-complier respondents and trials with non-contrastful image pairs to individual CIs of likely compliers and contrastful image pairs. In an appendix, we also examine the effects of filtering out data from individual participants and trials on the signal-to-noise ratio of group CIs. R scripts are publicly available for easy implementation of our suggestions in related research.

Reverse correlation analysis of a reaction time task for stochastic stimuli

Reverse correlation analysis of a reaction time task for stochastic stimuli

Perception and decision mechanisms involved in average estimation of spatiotemporal ensembles

A number of studies on texture and ensemble perception have shown that humans can immediately estimate the average of spatially distributed visual information. The present study characterized mechanisms involved in estimating averages for information distributed over both space and time. Observers viewed a rapid sequence of texture patterns in which elements’ orientation were determined by dynamic Gaussian noise with variable spatial and temporal standard deviations (SDs). We found that discrimination thresholds increased beyond a certain spatial SD if temporal SD was small, but if temporal SD was large, thresholds remained nearly constant regardless of spatial SD. These data are at odds with predictions that threshold is uniquely determined by spatiotemporal SD. Moreover, a reverse correlation analysis revealed that observers judged the spatiotemporal average orientation largely depending on the spatial average orientation over the last few frames of the texture sequence – a recency effect widely observed in studies of perceptual decision making. Results are consistent with the notion that the visual system rapidly computes spatial ensembles and adaptively accumulates information over time to make a decision on spatiotemporal average. A simple computational model based on this notion successfully replicated observed data.

The monaural spectral cues identified by a reverse correlation analysis of free-field auditory localization data.

The outer-ear's location-dependent pattern of spectral filtering generates cues used to determine a sound source's elevation as well as front-back location. The authors aim to identify these features using a reverse correlation analysis (RCA), combining free-field localization behaviour with the associated head-related transfer functions' (HRTFs) magnitude spectrum from a sample of 73 participants. Localization responses were collected before and immediately after introducing a pair of outer-ear inserts which modified the listener's HRTFs to varying extent. The RCA identified several different features responsible for eliciting localization responses. The efficacy of these was examined using two models of monaural localization. In general, the predicted performance was closely aligned with the free-field localization error for the bare-ear condition; however, both models tended to grossly over-estimate the localization error based on HRTFs modified by the outer-ear inserts. The RCA's feature selection notably had the effect of better aligning the predicted performance of both models to the actual localization performance. This suggests that the RCA revealed sufficient detail for both models to correctly predict localization performance and also limited the influence of filtered-out elements in the distorted HRTFs that contributed to the degraded accuracy of both models.

Flexibility of Timescales of Evidence Evaluation for Decision Making

To understand the neural mechanisms that support decision making, it is critical to characterize the timescale of evidence evaluation. Recent work has shown that subjects can adaptively adjust the timescale of evidence evaluation across blocks of trials depending on context [1]. However, it's currently unknown if adjustmentsto evidence evaluation occur online duringdeliberations based on a single stream of evidence. To examine this question, we employed a change-detection task in which subjects report their level of confidence in judging whether there has been a change in a stochastic auditory stimulus. Usinga combination of psychophysical reverse correlation analyses and single-trial behavioral modeling, we compared the time period over which sensory information has leverage on detection report choices versus confidence. We demonstrate that the length of this period differs on separate sets of trials based on what's being reported. Surprisingly, confidence judgments on trials with no detection report are influenced by evidence occurring earlier than the time period of influence for detection reports. Our findings call into question models of decision formation involving static parameters that yield a singular timescale of evidence evaluation and instead suggest that the brain represents and utilizes multiple timescales of evidence evaluation during deliberation.

Transcriptomic study of the mechanism of anoikis resistance in head and neck squamous carcinoma.

BackgroundNormal epithelial cells rapidly undergo apoptosis as soon as they lose contact with the extracellular matrix (ECM), which is termed as anoikis. However, cancer cells tend to develop a resistance mechanism to anoikis. This acquired ability is termed as anoikis resistance. Cancer cells, with anoikis resistance, can spread to distant tissues or organs via the peripheral circulatory system and cause cancer metastasis. Thus, inhibition of anoikis resistance blocks the metastatic ability of cancer cells.MethodsAnoikis-resistant CAL27 (CAL27AR) cells were induced from CAL27 cells using the suspension culture approach. Transcriptome analysis was performed using RNA-Seq to study the differentially expressed genes (DEGs) between the CAL27ARcells and the parental CAL27 cells. Gene function annotation and Gene Ontology (GO) enrichment analysis were performed using DAVID database. Signaling pathways involved in DEGs were analyzed using Gene Set Enrichment Analysis (GSEA) software. Analysis results were confirmed by reverse transcription PCR (RT-PCR), western blotting, and gene correlation analysis based on the TCGA database.ResultsGO enrichment analysis indicated that the biological process (BP) of the DEGs was associated with epidermal development, DNA replication, and G1/S transition of the mitotic cell cycle. The analysis of cellular component (CC) showed that the most significant up-regulated genes were related to extracellular exosome. KEGG Pathway analysis revealed that 23 signaling pathways were activated (p-value ≤ 0.05, FDR q-value ≤ 0.05) and 22 signaling pathways were suppressed (p-value ≤ 0.05, FDR q-value ≤ 0.05). The results from the GSEA indicated that in contrast to the inhibition of EGFR signaling pathway, the VEGF signaling pathway was activated. The VEGF signaling pathway possibly activates STAT3 though induction of STAT3 phosphorylation. Gene correlation analysis revealed that the VEGFA- STAT3-KLF4-CDKN1A signal axis was not only present in head and neck squamous carcinoma (HNSCC) but also two other epithelial-derived carcinomas that highly express VEGFA, including kidney renal clear cell carcinoma (KIRC) and ovarian serous cystadenocarcinoma (OV).

Reverse-Correlation Analysis of the Mechanosensation Circuit and Behavior in C. elegans Reveals Temporal and Spatial Encoding

Animals must integrate the activity of multiple mechanoreceptors to navigate complex environments. In Caenorhabditis elegans, the general roles of the mechanosensory neurons have been defined, but most studies involve end-point or single-time-point measurements, and thus lack dynamic information. Here, we formulate a set of unbiased quantitative characterizations of the mechanosensory system by using reverse correlation analysis on behavior. We use a custom tracking, selective illumination, and optogenetics platform to compare two mechanosensory systems: the gentle-touch (TRNs) and harsh-touch (PVD) circuits. This method yields characteristic linear filters that allow for the prediction of behavioral responses. The resulting filters are consistent with previous findings and further provide new insights on the dynamics and spatial encoding of the systems. Our results suggest that the tiled network of the gentle-touch neurons has better resolution for spatial encoding than the harsh-touch neurons. Additionally, linear-nonlinear models can predict behavioral responses based only on sensory neuron activity. Our results capture the overall dynamics of behavior induced by the activation of sensory neurons, providing simple transformations that quantitatively characterize these systems. Furthermore, this platform can be extended to capture the behavioral dynamics induced by any neuron or other excitable cells in the animal.