Features Of Visual Stimuli Research Articles

Assessing the Robustness of Cluster Solutions in Emotionally-Annotated Pictures Using Monte-Carlo Simulation Stabilized K-Means Algorithm

Clustering is a very popular machine-learning technique that is often used in data exploration of continuous variables. In general, there are two problems commonly encountered in clustering: (1) the selection of the optimal number of clusters, and (2) the undecidability of the affiliation of border data points to neighboring clusters. We address both problems and describe how to solve them in application to affective multimedia databases. In the experiment, we used the unsupervised learning algorithm k-means and the Nencki Affective Picture System (NAPS) dataset, which contains 1356 semantically and emotionally annotated pictures. The optimal number of centroids was estimated, using the empirical elbow and silhouette rules, and validated using the Monte-Carlo simulation approach. Clustering with k = 1–50 centroids is reported, along with dominant picture keywords and descriptive statistical parameters. Affective multimedia databases, such as the NAPS, have been specifically designed for emotion and attention experiments. By estimating the optimal cluster solutions, it was possible to gain deeper insight into affective features of visual stimuli. Finally, a custom software application was developed for study in the Python programming language. The tool uses the scikit-learn library for the implementation of machine-learning algorithms, data exploration and visualization. The tool is freely available for scientific and non-commercial purposes.

Decoding Concurrent Representations of Pitch and Location in Auditory Working Memory.

Multivariate analyses of hemodynamic signals serve to identify the storage of specific stimulus contents in working memory (WM). Representations of visual stimuli have been demonstrated both in sensory regions and in higher cortical areas. While previous research has typically focused on the WM maintenance of a single content feature, it remains unclear whether two separate features of a single object can be decoded concurrently. Also, much less evidence exists for representations of auditory compared with visual stimulus features. To address these issues, human participants had to memorize both pitch and perceived location of one of two sample sounds. After a delay phase, they were asked to reproduce either pitch or location. At recall, both features showed comparable levels of discriminability. Region of interest (ROI)-based decoding of functional magnetic resonance imaging (fMRI) data during the delay phase revealed feature-selective activity for both pitch and location of a memorized sound in auditory cortex and superior parietal lobule. The latter region showed higher decoding accuracy for location than pitch. In addition, location could be decoded from angular and supramarginal gyrus and both superior and inferior frontal gyrus. The latter region also showed a trend for decoding of pitch. We found no region exclusively coding pitch memory information. In summary, the present study yielded evidence for concurrent representations of pitch and location of a single object both in sensory cortex and in hierarchically higher regions, pointing toward representation formats that enable feature integration within the same anatomic brain regions.SIGNIFICANCE STATEMENT Decoding of hemodynamic signals serves to identify brain regions involved in the storage of stimulus-specific information in working memory (WM). While to-be-remembered information typically consists of several features, most previous investigations have focused on the maintenance of one memorized feature belonging to one visual object. The present study assessed the concurrent storage of two features of the same object in auditory WM. We found that both pitch and location of memorized sounds were decodable both in early sensory areas, in higher-level superior parietal cortex and, to a lesser extent, in inferior frontal cortex. While auditory cortex is known to process different features in parallel, their concurrent representation in parietal regions may support the integration of object features in WM.

Audiovisual saliency prediction via deep learning

Neuroscience study verifies that synchronized audiovisual stimuli would make a stronger response of visual perception than an independent stimulus. Many researches show that audio signals would affect human gaze behavior in the viewing of natural video scenes. Thus in this paper, we propose a multi-sensory framework of audio and visual signals for video saliency prediction. It mainly includes four modules: auditory feature extraction, visual feature extraction, semantic interaction between auditory feature and visual feature, and feature fusion. With the inputs of audio and visual signals, we present a network architecture of deep learning to undertake the tasks of these four modules. It is an end-to-end architecture that could interact the semantics from its learned features of audio and visual stimuli. The numerical and visual results show our method achieves a significant improvement over eleven recent saliency models that are regardless of the audio stimuli, even some of them are state-of-the-art deep learning models.

The Effect of Visual Features of Search Stimuli on Learning of Spatial Contextual Cueing

The Effect of Visual Features of Search Stimuli on Learning of Spatial Contextual Cueing

Curvature processing in human visual cortical areas

Curvature is one of many visual features shown to be important for visual perception. We recently showed that curvilinear features provide sufficient information for categorizing animate vs. inanimate objects, while rectilinear features do not (Zachariou et al., 2018). Results from our fMRI study in rhesus monkeys (Yue et al., 2014) have shed light on some of the neural substrates underlying curvature processing by revealing a network of visual cortical patches with a curvature response preference. However, it is unknown whether a similar network exists in human visual cortex. Thus, the current study was designed to investigate cortical areas with a preference for curvature in the human brain using fMRI at 7T. Consistent with our monkey fMRI results, we found a network of curvature preferring cortical patches—some of which overlapped well-known face-selective areas. Moreover, principal component analysis (PCA) using all visually-responsive voxels indicated that curvilinear features of visual stimuli were associated with specific retinotopic regions in visual cortex. Regions associated with positive curvilinear PC values encompassed the central visual field representation of early visual areas and the lateral surface of temporal cortex, while those associated with negative curvilinear PC values encompassed the peripheral visual field representation of early visual areas and the medial surface of temporal cortex. Thus, we found that broad areas of curvature preference, which encompassed face-selective areas, were bound by central visual field representations. Our results support the hypothesis that curvilinearity preference interacts with central-peripheral processing biases as primary features underlying the organization of temporal cortex topography in the adult human brain.

Направленность внимания у детей с РАС

The special way of understanding the world and the brain mechanisms of cognitive impairment in children with autism spectrum disorders (ASD) are becoming the core topics in the autism research. The article presents a study of the general characteristics of attention in children of primary school age with ASD using instruments aimed at detecting impairments of attention parameters: Schulte Tables, Go/No Go Task, Kraepelin’s Arithmetic Test, Stroop Test used to study the focus of attention in the context of the interference of verbal and object stimuli. Results of study of attention in children with ASD seemed to be ambivalent. Children with ASD are successful in performing the Schulte Tables; on the contrary, the Go/No Go Task shows the difficulties of performing the tasks and inhibition of dominant response. Different degrees of unstable focus of attention in tasks on visual perception also shown. The features of focusing attention in conditions of interference require further research. Preliminary results showed that the focus of attention in younger schoolchildren with ASD is associated with a preference for verbal rather than object characteristics of visual stimuli.

Evidence for intact stimulus-specific neural adaptation for visual objects in schizophrenia and bipolar disorder: An ERP study.

People with schizophrenia (SZ) or bipolar disorder (BD) experience dysfunction in visual processing. Dysfunctional neural tuning, in which neurons and neuronal populations are selectively activated by specific features of visual stimuli, may contribute to these deficits. Few studies have examined this possibility and there are inconsistent findings of tuning deficits in the literature. We utilized an event-related potential (ERP) paradigm to examine neural adaptation for visual objects, a measure of neural tuning whereby neurons respond less strongly to the repeated presentation of the same stimulus. Seventy-seven SZ, 53 BD, and 49 healthy comparison participants (HC) were examined. In three separate conditions, pictures of objects were presented repeatedly: the same object (SS), different objects from the same category (e.g., two different vases; SD), or different objects from different categories (e.g., a barrel and a clock, DD). Mass-univariate cluster-based permutation analyses identified electrodes and time-windows in which there were significant differences between the SS vs. DD and the SD vs. DD conditions. Mean ERP amplitudes were extracted from these clusters and analyzed for group differences. Results revealed a significant condition difference over parieto-occipital electrodes for the SS-DD comparison between 109–164 ms and for the SD-DD comparison between 78–203 ms, with larger amplitudes in the DD compared to either SS or SD condition. However, there were no significant differences in the pattern of results between groups. Thus, while we found neural adaptation effects using this ERP paradigm, we did not find evidence of group differences. Our results suggest that people with SZ or BD may not exhibit deficits in neural tuning for processing of visual objects using this EEG task with rapidly presented stimuli. However, the results are inconsistent with other studies using different methodologies (e.g., fMRI, behavioral tasks) that have found tuning deficits in people with schizophrenia.

Anticipatory effect of execution on observation: an approach using ExoPinch finger robot

Background/aimThis study aims to explore the mirror neuron system (MNS) involvement using mu (8–12 Hz)/beta (15–25 Hz) band suppression in an action observation-execution paradigm.Materials and methodsElectrophysiological (EEG) data from 16 electrodes were recorded while 8 participants observed video clips of a hand squeezing a spring. Specifically, the effect of anticipated execution on observation was studied. For this purpose, a fully actuated finger exoskeleton robot was utilized to synchronize observation and execution and to control the execution condition for the participants. Anticipatory effect was created with a randomized robot accompany session.ResultsThe results showed that the observational condition (with or without anticipation) interacted with hemisphere at central channels near somatosensory cortex. Additionally, we explored the response of MNS on the kinetics features of visual stimuli (hard or soft spring).ConclusionThe results showed an interaction effect of kinetics features and hemisphere at frontal channels corresponding nearly to the ventral premotor cortex area of the brain. The activation of mirror neurons in this area plays a crucial role in observational learning. Based on our results, we propose that specific type of visual stimuli can be combined with the functional abilities of the MNS in the action observation based treatment of hand motor dysfunction of stroke patients to have a positive additional impact.

Evaluating the Relationship Between Multimedia Viewing, Sedentary Behavior, and Executive Function: a Systematic Review

Research and discussions over the relationship between executive function, multimedia viewing, and sedentary human behavior are becoming increasingly popular throughout academia and mainstream media. While some studies report screen-viewing behavior as being associated with reduced levels of physical activity, other studies have also reported associations with higher levels of cognitive activity. The aim of this paper is to establish the relationship between multimedia viewing, sedentary behavior, and executive function. In achieving this aim, the main objectives are to (1) identify a framework to specify core characteristics of visual stimuli that may influence behavior and (2) identify the ways executive functions may mediate the interpretation of visual media. Literature searches were conducted in the Cochrane Library, PsychInfo and PsychArticles, PubMed, Science Direct, Scopus, and Web of Science. Methods of assessment reported in the literature were predominantly reliant on experimenter observation and participant self-report, without psychometric, empirical, or objective measurement of the core variables under study. Despite a sizable proportion of research conducting large-scale experiments and/or secondary analysis of data from longitudinal databases, there were no indications of the characteristics or styles of visual stimuli used. The myriad of published works range in methodological design, resource, and stimuli use or sample population. Raising a question of comparability and replicability, recommendations are made for researchers in reporting multimedia stimuli and importance in addressing the role of cognitive engagement and social interaction within the methodological design and reporting.

Effects of visual stimulus characteristics and individual differences in heading estimation.

Visual heading estimation is subject to periodic patterns of constant (bias) and variable (noise) error. The nature of the errors, however, appears to differ between studies, showing underestimation in some, but overestimation in others. We investigated whether field of view (FOV), the availability of binocular disparity cues, motion profile, and visual scene layout can account for error characteristics, with a potential mediating effect of vection. Twenty participants (12 females) reported heading and rated vection for visual horizontal motion stimuli with headings ranging the full circle, while we systematically varied the above factors. Overall, the results show constant errors away from the fore-aft axis. Error magnitude was affected by FOV, disparity, and scene layout. Variable errors varied with heading angle, and depended on scene layout. Higher vection ratings were associated with smaller variable errors. Vection ratings depended on FOV, motion profile, and scene layout, with the highest ratings for a large FOV, cosine-bell velocity profile, and a ground plane scene rather than a dot cloud scene. Although the factors did affect error magnitude, differences in its direction were observed only between participants. We show that the observations are consistent with prior beliefs that headings align with the cardinal axes, where the attraction of each axis is an idiosyncratic property.

Distinct cell types in the superficial superior colliculus project to the dorsal lateral geniculate and lateral posterior thalamic nuclei

The superficial layers of the superior colliculus (sSC) receive retinal input and project to thalamic regions, the dorsal lateral geniculate (dLGN) and lateral posterior (LP; or pulvinar) nuclei, that convey visual information to cortex. A critical step toward understanding the functional impact of sSC neurons on these parallel thalamo-cortical pathways is determining whether different classes of sSC neurons, which are known to respond to different features of visual stimuli, innervate overlapping or distinct thalamic targets. Here, we identified a transgenic mouse line that labels sSC neurons that project to dLGN but not LP. We utilized selective expression of fluorophores and channelrhodopsin in this and previously characterized mouse lines to demonstrate that distinct cell types give rise to sSC projections to dLGN and LP. We further show that the glutamatergic sSC cell type that projects to dLGN also provides input to the sSC cell type that projects to LP. These results clarify the cellular origin of parallel sSC-thalamo-cortical pathways and reveal an interaction between these pathways via local connections within the sSC.NEW & NOTEWORTHY The superficial layers of the superior colliculus (sSC) project to two visual thalamic targets: the dorsal lateral geniculate (dLGN) and lateral posterior (LP) nuclei. We show that distinct excitatory sSC cell types give rise to these projections; stellate cells project to dLGN and wide-field (WF) cells project to LP. Moreover, these pathways interact via a connection within the sSC from stellate to WF cells.

Decoding natural images from evoked brain activities using encoding models with invertible mapping

Recent studies have built encoding models in the early visual cortex, and reliable mappings have been made between the low-level visual features of stimuli and brain activities. However, these mappings are irreversible, so that the features cannot be directly decoded. To solve this problem, we designed a sparse framework-based encoding model that predicted brain activities from a complete feature representation. Moreover, according to the distribution and activation rules of neurons in the primary visual cortex (V1), three key transformations were introduced into the basic feature to improve the model performance. In this setting, the mapping was simple enough that it could be inverted using a closed-form formula. Using this mapping, we designed a hybrid identification method based on the support vector machine (SVM), and tested it on a published functional magnetic resonance imaging (fMRI) dataset. The experiments confirmed the rationality of our encoding model, and the identification accuracies for 2 subjects increased from 92% and 72% to 98% and 92% with the chance level only 0.8%.

Encoding of Spatial Attention by Primate Prefrontal Cortex Neuronal Ensembles.

Single neurons in the primate lateral prefrontal cortex (LPFC) encode information about the allocation of visual attention and the features of visual stimuli. However, how this compares to the performance of neuronal ensembles at encoding the same information is poorly understood. Here, we recorded the responses of neuronal ensembles in the LPFC of two macaque monkeys while they performed a task that required attending to one of two moving random dot patterns positioned in different hemifields and ignoring the other pattern. We found single units selective for the location of the attended stimulus as well as for its motion direction. To determine the coding of both variables in the population of recorded units, we used a linear classifier and progressively built neuronal ensembles by iteratively adding units according to their individual performance (best single units), or by iteratively adding units based on their contribution to the ensemble performance (best ensemble). For both methods, ensembles of relatively small sizes (n < 60) yielded substantially higher decoding performance relative to individual single units. However, the decoder reached similar performance using fewer neurons with the best ensemble building method compared with the best single units method. Our results indicate that neuronal ensembles within the LPFC encode more information about the attended spatial and nonspatial features of visual stimuli than individual neurons. They further suggest that efficient coding of attention can be achieved by relatively small neuronal ensembles characterized by a certain relationship between signal and noise correlation structures.

Audiovisual Modulation in Mouse Primary Visual Cortex Depends on Cross-Modal Stimulus Configuration and Congruency.

The sensory neocortex is a highly connected associative network that integrates information from multiple senses, even at the level of the primary sensory areas. Although a growing body of empirical evidence supports this view, the neural mechanisms of cross-modal integration in primary sensory areas, such as the primary visual cortex (V1), are still largely unknown. Using two-photon calcium imaging in awake mice, we show that the encoding of audiovisual stimuli in V1 neuronal populations is highly dependent on the features of the stimulus constituents. When the visual and auditory stimulus features were modulated at the same rate (i.e., temporally congruent), neurons responded with either an enhancement or suppression compared with unisensory visual stimuli, and their prevalence was balanced. Temporally incongruent tones or white-noise bursts included in audiovisual stimulus pairs resulted in predominant response suppression across the neuronal population. Visual contrast did not influence multisensory processing when the audiovisual stimulus pairs were congruent; however, when white-noise bursts were used, neurons generally showed response suppression when the visual stimulus contrast was high whereas this effect was absent when the visual contrast was low. Furthermore, a small fraction of V1 neurons, predominantly those located near the lateral border of V1, responded to sound alone. These results show that V1 is involved in the encoding of cross-modal interactions in a more versatile way than previously thought.SIGNIFICANCE STATEMENT The neural substrate of cross-modal integration is not limited to specialized cortical association areas but extends to primary sensory areas. Using two-photon imaging of large groups of neurons, we show that multisensory modulation of V1 populations is strongly determined by the individual and shared features of cross-modal stimulus constituents, such as contrast, frequency, congruency, and temporal structure. Congruent audiovisual stimulation resulted in a balanced pattern of response enhancement and suppression compared with unisensory visual stimuli, whereas incongruent or dissimilar stimuli at full contrast gave rise to a population dominated by response-suppressing neurons. Our results indicate that V1 dynamically integrates nonvisual sources of information while still attributing most of its resources to coding visual information.

Microsaccades are sensitive to word structure: A novel approach to study language processing

Microsaccades are miniature eye movements that occur involuntarily during fixation. They are typically inhibited following stimulus onset and are released from inhibition about 300 ms post-stimulus. Microsaccade-inhibition is modulated by low level features of visual stimuli, but it is currently unknown whether they are sensitive to higher level, abstract linguistic properties. To address this question, we measured the timing of microsaccades while subjects were presented with written Hebrew words and pronounceable nonwords (pseudowords). We manipulated the underlying structure of pseudowords such that half of them contained real roots while the other half contained invented roots. Importantly, orthographic similarity to real words was equated between the two conditions. Microsaccade onset was significantly slower following real-root compared to invented-root stimuli. Similar results were obtained when considering post-stimulus delay of eye blinks. Moreover, microsaccade-delay was positively and significantly correlated with measures of real-word similarity. These findings demonstrate, for the first time, sensitivity of microsaccades to linguistic structure. Because microsaccades are involuntary and can be measured in the absence of overt response, our results provide initial evidence that they can be used as a novel physiological measure in the study of language processes in healthy and clinical populations.

On the partnership between neural representations of object categories and visual features in the ventral visual pathway

A dominant view in the cognitive neuroscience of object vision is that regions of the ventral visual pathway exhibit some degree of category selectivity. However, recent findings obtained with multivariate pattern analyses (MVPA) suggest that apparent category selectivity in these regions is dependent on more basic visual features of stimuli. In which case a rethinking of the function and organization of the ventral pathway may be in order. We suggest that addressing this issue of functional specificity requires clear coding hypotheses, about object category and visual features, which make contrasting predictions about neuroimaging results in ventral pathway regions. One way to differentiate between categorical and featural coding hypotheses is to test for residual categorical effects: effects of category selectivity that cannot be accounted for by visual features of stimuli. A strong method for testing these effects, we argue, is to make object category and target visual features orthogonal in stimulus design. Recent studies that adopt this approach support a feature-based categorical coding hypothesis according to which regions of the ventral stream do indeed code for object category, but in a format at least partially based on the visual features of stimuli.

Neuronal population coding of perceived and memorized visual features in the lateral prefrontal cortex

The primate lateral prefrontal cortex (LPFC) encodes visual stimulus features while they are perceived and while they are maintained in working memory. However, it remains unclear whether perceived and memorized features are encoded by the same or different neurons and population activity patterns. Here we record LPFC neuronal activity while monkeys perceive the motion direction of a stimulus that remains visually available, or memorize the direction if the stimulus disappears. We find neurons with a wide variety of combinations of coding strength for perceived and memorized directions: some neurons encode both to similar degrees while others preferentially or exclusively encode either one. Reading out the combined activity of all neurons, a machine-learning algorithm reliably decode the motion direction and determine whether it is perceived or memorized. Our results indicate that a functionally diverse population of LPFC neurons provides a substrate for discriminating between perceptual and mnemonic representations of visual features.

More Than the Verbal Stimulus Matters: Visual Attention in Language Assessment for People With Aphasia Using Multiple-Choice Image Displays.

Language comprehension in people with aphasia (PWA) is frequently evaluated using multiple-choice displays: PWA are asked to choose the image that best corresponds to the verbal stimulus in a display. When a nontarget image is selected, comprehension failure is assumed. However, stimulus-driven factors unrelated to linguistic comprehension may influence performance. In this study we explore the influence of physical image characteristics of multiple-choice image displays on visual attention allocation by PWA. Eye fixations of 41 PWA were recorded while they viewed 40 multiple-choice image sets presented with and without verbal stimuli. Within each display, 3 images (majority images) were the same and 1 (singleton image) differed in terms of 1 image characteristic. The mean proportion of fixation duration (PFD) allocated across majority images was compared against the PFD allocated to singleton images. PWA allocated significantly greater PFD to the singleton than to the majority images in both nonverbal and verbal conditions. Those with greater severity of comprehension deficits allocated greater PFD to nontarget singleton images in the verbal condition. When using tasks that rely on multiple-choice displays and verbal stimuli, one cannot assume that verbal stimuli will override the effect of visual-stimulus characteristics.

Binding time: Evidence for integration of temporal stimulus features.

Several lines of evidence suggest that during processing of events, the features of these events become connected via episodic bindings. Such bindings have been demonstrated for a large number of visual and auditory stimulus features, like color and orientation, or pitch and loudness. Importantly, most visual and auditory events typically also involve temporal features, like onset time or duration. So far, however, whether temporal stimulus features are also bound into event representations has never been tested directly. The aim of the present study was to investigate possible binding between stimulus duration and other features of auditory events. In Experiment 1, participants had to respond with two keys to a low or high pitch sinus tone. Critically, the tones were presented with two different presentation durations. Sequential analysis of RT data indicated binding of stimulus duration into the event representation: at pitch repetitions, performance was better when both pitch and duration repeated, relative to when only pitch repeated and duration switched. This finding was replicated with loudness as relevant stimulus feature in Experiment 2. In sum, the results demonstrate that temporal features are bound into auditory event representations. This finding is an important advancement for binding theory in general, and raises several new questions for future research.

Neural mechanisms for integrating consecutive and interleaved natural events.

To understand temporally extended events, the human brain needs to accumulate information continuously across time. Interruptions that require switching of attention to other event sequences disrupt this process. To reveal neural mechanisms supporting integration of event information, we measured brain activity with functional magnetic resonance imaging (fMRI) from 18 participants while they viewed 6.5-minute excerpts from three movies (i) consecutively and (ii) as interleaved segments of approximately 50-s in duration. We measured inter-subject reliability of brain activity by calculating inter-subject correlations (ISC) of fMRI signals and analyzed activation timecourses with a general linear model (GLM). Interleaving decreased the ISC in posterior temporal lobes, medial prefrontal cortex, superior precuneus, medial occipital cortex, and cerebellum. In the GLM analyses, posterior temporal lobes were activated more consistently by instances of speech when the movies were viewed consecutively than as interleaved segments. By contrast, low-level auditory and visual stimulus features and editing boundaries caused similar activity patterns in both conditions. In the medial occipital cortex, decreases in ISC were seen in short bursts throughout the movie clips. By contrast, the other areas showed longer-lasting differences in ISC during isolated scenes depicting socially-relevant and suspenseful content, such as deception or inter-subject conflict. The areas in the posterior temporal lobes also showed sustained activity during continuous actions and were deactivated when actions ended at scene boundaries. Our results suggest that the posterior temporal and dorsomedial prefrontal cortices, as well as the cerebellum and dorsal precuneus, support integration of events into coherent event sequences. Hum Brain Mapp 38:3360-3376, 2017. © 2017 Wiley Periodicals, Inc.