Spatial Frequency Channels Research Articles

Influence of background on the spatial-frequency channel for object recognition

Influence of background on the spatial-frequency channel for object recognition

Unsupervised learning as a computational principle works in visual learning of natural scenes, but not of artificial stimuli.

The question of whether we learn exposed visual features remains a subject of controversy. A prevalent computational model suggests that visual features frequently exposed to observers in natural environments are likely to be learned. However, this unsupervised learning model appears to be contradicted by the significant body of experimental results with human participants that indicates visual perceptual learning (VPL) of visible task-irrelevant features does not occur with frequent exposure. Here, we demonstrate a resolution to this controversy with a new finding: Exposure to a dominant global orientation as task-irrelevant leads to VPL of the orientation, particularly when the orientation is derived from natural scene images, whereas VPL did not occur with artificial images even with matched distributions of local orientations and spatial frequencies to natural scene images. Further investigation revealed that this disparity arises from the presence of higher-order statistics derived from natural scene images-global structures such as correlations between different local orientation and spatial frequency channels. Moreover, behavioral and neuroimaging results indicate that the dominant orientation from these higher-order statistics undergoes less attentional suppression than that from artificial images, which may facilitate VPL. Our results contribute to resolving the controversy by affirming the validity of unsupervised learning models for natural scenes but not for artificial stimuli. They challenge the assumption that VPL occurring in everyday life can be predicted by laws governing VPL for conventionally used artificial stimuli.

Applying Resampling and Visualization Methods in Factor Analysis to Model Human Spatial Vision.

Studies have reported different numbers of spatial frequency channels for chromatic and achromatic vision. To resolve the difference, we performed factor analysis, a multivariate modeling technique, on population data of achromatic and chromatic sensitivity. In addition, we included resampling and visualization methods to evaluate models from factor analysis. These routines are complex but widely useful. Therefore we have archived our analysis routines by building smCSF, an open-source software package in R (https://smin95.github.io/dataviz/). Data of 103 normally-sighted adults were analyzed. They included blue-yellow, red-green, and achromatic contrast sensitivity. To obtain the confidence interval of relevant statistical parameters, factor analysis was performed using a resampling method. Then exploratory models were developed. We then performed model selections by fitting them against the empirical data and quantifying the quality of the fits. During the exploratory factor analysis, different statistical tests supported different factor models. These could partially be reasons for why there have been conflicting reports. However, after the confirmatory analysis, we found that a model that included two spatial channels was adequate to approximate the chromatic sensitivity data, whereas that with three channels was so for the achromatic sensitivity data. Our findings provide novel insights about the spatial channels for chromatic and achromatic contrast sensitivity from population data. Also, the analysis and visualization routines have been archived in a computational package to boost the transparency and replicability of science.

Spatial frequency channels depend on stimulus bandwidth in normal and amblyopic vision: an exploratory factor analysis.

The Contrast Sensitivity Function (CSF) is the measure of an observer's contrast sensitivity as a function of spatial frequency. It is a sensitive measure to assess visual function in fundamental and clinical settings. Human contrast sensitivity is subserved by different spatial frequency channels. Also, it is known that amblyopes have deficits in contrast sensitivity, particularly at high spatial frequencies. Therefore, the aim of this study was to assess whether the contrast sensitivity function is subtended by the same spatial frequency channels in control and amblyopic populations. To determine these spatial frequency channels, we performed an exploratory factor analysis on five datasets of contrasts sensitivity functions of amblyopic and control participants measured using either gratings or noise patches, taken from our previous studies. In the range of 0.25-10 c/d, we identified two spatial frequency channels. When the CSF was measured with noise patches, the spatial frequency channels presented very similar tuning in the amblyopic eye and the fellow eye and were also similar to what was observed in controls. The only major difference was that the weight attributed to the high frequency channel was reduced by approximately 50% in the amblyopic eye. However, when the CSF was measured using gratings, the spatial frequency channels of the amblyopic eye were tuned toward lower spatial frequencies. These findings suggest that there is no mechanistic deficit for contrast sensitivity in amblyopia and that amblyopic vision may just be subjected to excessive internal noise and attenuation at higher spatial frequencies, thereby supporting the use of therapeutic strategies that involve rebalancing contrast.

Spatial-frequency channels for object recognition by neural networks are twice as wide as those of humans

Spatial-frequency channels for object recognition by neural networks are twice as wide as those of humans

Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition

Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.

A quasi-comprehensive exploration of the mechanisms of spatial working memory.

Why are some spatial patterns remembered more easily than others? There are many possible mechanisms underlying spatial working memory function. Here, the author explores different mechanisms simultaneously in a single conceptual model. He conducts a large-scale experiment (35.4 million responses used to measure human observers' spatial working memory across 80,000 patterns) and builds a convolutional neural network as a benchmark for what is expected to be explainable. The author then creates a quasi-comprehensive exploration model of spatial working memory based on classic concepts, as well as new notions, including spatial uncertainty, Bayesian integration, out-of-range responses, averaging, grouping, categorical memory, line detection, gap detection, blurring, lateral inhibition, chunking, multiple spatial-frequency channels, redundancy, response bias and random guess. This model provides a tentative overarching framework for the mechanisms of spatial working memory.

Sensitivity to naturalistic texture relies primarily on high spatial frequencies.

Natural images contain information at multiple spatial scales. Though we understand how early visual mechanisms split multiscale images into distinct spatial frequency channels, we do not know how the outputs of these channels are processed further by mid-level visual mechanisms. We have recently developed a texture discrimination task that uses synthetic, multi-scale, "naturalistic" textures to isolate these mid-level mechanisms. Here, we use three experimental manipulations (image blur, image rescaling, and eccentric viewing) to show that perceptual sensitivity to naturalistic structure is strongly dependent on features at high object spatial frequencies (measured in cycles/image). As a result, sensitivity depends on a texture acuity limit, a property of the visual system that sets the highest retinal spatial frequency (measured in cycles/degree) at which observers can detect naturalistic features. Analysis of the texture images using a model observer analysis shows that naturalistic image features at high object spatial frequencies carry more task-relevant information than those at low object spatial frequencies. That is, the dependence of sensitivity on high object spatial frequencies is a property of the texture images, rather than a property of the visual system. Accordingly, we find human observers' ability to extract naturalistic information (their efficiency) is similar for all object spatial frequencies. We conclude that the mid-level mechanisms that underlie perceptual sensitivity effectively extract information from all image features below the texture acuity limit, regardless of their retinal and object spatial frequency.

Statistical Characterization and Modeling of Radio Frequency Signal Propagation in Mobile Broadband Cellular Next Generation Wireless Networks

An accurate assessment of the spatial and temporal radio frequency channel characteristics is essential for complex signal processing and cellular network optimization. Current research has employed numerous models to figure out how much signal propagation loss occurs along the propagation paths. However, there are issues in finding the right model for a particular terrain because these models are not universally applicable. By employing the lognormal function and the Maximum Likelihood model, a hybrid probabilistic statistical distribution model was evolved. Three LTE cell site locations in Port Harcourt, Nigeria, were used to create a hybrid model that describes the functional stochastic signal propagation loss in the area. The evaluated Maximum Likelihood model accurately estimates the relevant wireless channel properties based on observed field data. The minor square regression approach and the proposed hybrid parameter estimation methodology are compared. When it comes to estimating standard deviation errors as well as the root mean square errors, the ML‐based approach consistently outperforms the least square regression model. Finally, the proposed hybrid probabilistic statistical distribution model would be useful for mobile broadband network planning in related wireless propagation conditions.

Biases in the spectral amplitude distribution of a natural scene modulate horizontal size perception.

Visual perception is a complex process that involves the analysis of different spatial and temporal features of the visual environment. One critical aspect of this process is adaptation, which allows the visual system to adjust its sensitivity to specific features based on the context of the environment. Numerous theories highlight the significance of the visual scene and its spectral properties in perceptual and adaptation mechanisms. For example, size perception is known to be influenced by the spatial frequency content of the visual scene. Nonetheless, several inquiries still exist, including how specific spectral properties of the scene play a role in size perception and adaptation mechanisms. In this study, we explore aftereffects on size perception following adaptation to a natural scene with a biased spectral amplitude distribution. Twenty participants had to manually estimate the horizontal size of a projected rectangle after adaptation to three visually biased conditions: vertical-biased, non-biased, and horizontal-biased. Size adaptation aftereffects were quantified by comparing the perceptual responses from the non-biased condition with the vertical- and horizontal-biased conditions. We found size perception shifts which were contingent upon the specific orientation and spatial frequency distribution inherent in the amplitude spectra of the adaptation stimuli. Particularly, adaptation to vertical-biased produced a horizontal enlargement, while adaptation to horizontal-biased generated a decrease in the horizontal size perception of the rectangle. On average, size perception was modulated by 5-6%. These findings provide supporting evidence for the hypothesis that the neural mechanisms responsible for processing spatial frequency channels are involved in the encoding and perception of size information. The implications for neural mechanisms underlying spatial frequency and size information encoding are discussed.

FCMNet: Frequency-aware cross-modality attention networks for RGB-D salient object detection

RGB-D saliency detection aims to comprehensively use RGB images and depth maps to detect object saliency. This field still faces two challenges: 1) how to extract representative multimodal features and 2) how to effectively fuse them. Most of the previous methods in this field equally treat RGB and depth information as two modalities, while not considering the difference in the frequency domain of the two modalities, and may lose some complementary information. In this paper, we introduce the frequency channel attention mechanism into the fusion process. First, we design a frequency-aware cross-modality attention (FACMA) module to interweave adequate channel features and select representative features. In the FACMA module, we also propose a spatial frequency channel attention (SFCA) module to introduce more complementary information in different channels. Second, we develop a weighted cross-modality fusion (WCMF) module to adaptively fuse multimodality features by learning the content-dependent weight maps. Comprehensive experiments on several benchmark datasets demonstrate that the proposed framework outperforms seventeen state-of-the-art methods.

Model-Driven Deep Learning Based Channel Estimation and Feedback for Millimeter-Wave Massive Hybrid MIMO Systems

This paper proposes a model-driven deep learning (MDDL)-based channel estimation and feedback scheme for wideband millimeter-wave (mmWave) massive hybrid multiple-input multiple-output (MIMO) systems, where the angle-delay domain channels' sparsity is exploited for reducing the overhead. First, we consider the uplink channel estimation for time-division duplexing systems. To reduce the uplink pilot overhead for estimating high-dimensional channels from a limited number of radio frequency (RF) chains at the base station (BS), we propose to jointly train the phase shift network and the channel estimator as an auto-encoder. Particularly, by exploiting the channels' structured sparsity from an a priori model and learning the integrated trainable parameters from the data samples, the proposed multiple-measurement-vectors learned approximate message passing (MMV-LAMP) network with the devised redundant dictionary can jointly recover multiple subcarriers' channels with significantly enhanced performance. Moreover, we consider the downlink channel estimation and feedback for frequency-division duplexing systems. Similarly, the pilots at the BS and channel estimator at the users can be jointly trained as an encoder and a decoder, respectively. Besides, to further reduce the channel feedback overhead, only the received pilots on part of the subcarriers are fed back to the BS, which can exploit the MMV-LAMP network to reconstruct the spatial-frequency channel matrix. Numerical results show that the proposed MDDL-based channel estimation and feedback scheme outperforms state-of-the-art approaches.

Salience of spatiochromatic patterns.

We investigated the dependence of perceived contrast on cone-opponent stimulus content and its spatial distribution. Participants matched a comparison patch to a light gray standard of fixed contrast. The first experiment determined the point of iso-salience for gratings, Gabors and Gaussians along cardinal directions in cone-opponent color space for two-alternative forced choice (2AFC) and adjustment tasks. No difference was found between adjustment and 2AFC tasks, meaning that adjustment tasks provide a quick and robust way to measure perceived contrast, at least for relatively large suprathreshold stimuli. In line with the differences in contrast energy between Gaussians, Gabors, and gratings, Gaussians required less contrast to achieve equal perceived salience with a standard irrespective of color. More surprisingly, bluish Gaussians were found to have higher salience than yellowish Gaussians at equal levels of contrast. Although perceived contrast of grating and Gabor patterns likely depends on spatial frequency channels that at 1 cycle-per-degree are not too dissimilarly tuned for color and luminance, for Gaussians the contribution of single-opponent neurons would be greater for color than for luminance. In a follow-up experiment, we found that the bluish/yellowish asymmetry decreased as we reduced the proportion of the lowpass non-flat contrast distribution in the stimulus, with minimal asymmetry for the stimulus with a flat contrast distribution (i.e., uniform patch). Combined, this means that differential engagement of spatial frequency channels, single-opponent and double-opponent neurons impacts on perceived contrast of chromatic suprathreshold stimuli. Perceived contrast thus provides a window into neural computations enacted by low-level cone-opponent mechanisms.

A unified model for binocular fusion and depth perception

We describe a new unified model to explain both binocular fusion and depth perception, over a broad range of depths. At each location, the model consists of an array of paired spatial frequency filters, with different relative horizontal shifts (position disparity) and interocular phase disparities of 0, 90, ±180, or −90°. The paired filters with different spatial profiles (non-zero phase disparity) compute interocular misalignment and provide phase-disparity energy (binocular fusion energy) to drive selection of the appropriate filters along the position disparity space until the misalignment is eliminated and sensory fusion is achieved locally. The paired filters with identical spatial profiles (0 phase disparity) compute the position-disparity energy. After sensory fusion, the combination of position and possible residual phase disparity energies is calculated for binocular depth perception. Binocular fusion occurs at multiple scales following a coarse-to-fine process. At a given location, the apparent depth is the weighted sum of fusion shifts combined with residual phase disparity in all spatial-frequency channels, and the weights depend on stimulus spatial frequency and stimulus contrast. To test the theory, we measured disparity minimum and maximum thresholds (Dmin and Dmax) at three spatial frequencies and with different intraocular contrast levels. The stimuli were Random-Gabor-Patch (RGP) stereograms consisting of Gabor patches with random positions and phases, but with a fixed spatial frequency. The two eyes viewed identical arrays of patches except that one eye’s array could be shifted horizontally and could differ in contrast. Our experiments and modeling reveal two contrast normalization mechanisms: (1) Energy Normalization (EN): Binocular energy is normalized with monocular energy after the site of binocular combination. This predicts constant Dmin thresholds when varying stimulus contrast in the two eyes; (2) DSKL model Interocular interactions: Monocular contrasts are normalized before the binocular combination site through interocular contrast gain-control and gain-enhancement mechanisms. This predicts contrast dependent Dmax thresholds. We tested a range of models and found that a model consisting of a second-order pathway with DSKL interocular interactions and a first-order pathway with EN at each spatial-frequency band can account for both the Dmin and Dmax data very well. Simulations show that the model makes reasonable predictions of suprathreshold depth perception.

Mindfulness and De-automatization: Effect of Mindfulness-Based Interventions on Emotional Facial Expressions Processing

Past research has suggested that mindfulness training reduces automaticity while processing socio-emotional stimuli. The present study aimed to analyze how mindfulness practice may reduce the use of prior knowledge during the recognition of emotional facial expressions. Based on a predictive brain model, we hypothesized that mindfulness practice would reduce the top-down processing of low spatial frequency information. This experiment compared the performance of a mindfulness group (n = 32) and a waitlist control group (n = 30) in an emotional Stroop task before and after an 8-week training course. The emotional Stroop task comprised two emotional facial expressions (joy or anger) topped with a congruent or incongruent word, and was primed by facial expressions filtered in two spatial frequency bands: high spatial frequency (HSF) or low spatial frequency (LSF). Having measured the reaction time, the results showed a significant interaction between group (mindfulness vs. control) and session (before vs. after training; p = 0.04; R2 = 0.001), irrespective of spatial frequency channels. Breaking down the interaction showed that mindfulness-trained participants responded significantly faster than the controls to any type of information. The interaction Group by Session by Priming was not significant. These results are in line with research underlining the effects of mindfulness-based interventions on global attentional control. More precisely, the global reduced reaction time did not support lower top-down predictive coding abilities specifically driven by low spatial frequency channels, but indicated a better general sensitivity to the perceptual environment.

Tensor-Based Channel Estimation for Millimeter Wave MIMO-OFDM With Dual-Wideband Effects

We consider the channel estimation problem in millimeter wave (mmWave) multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) systems with hybrid analog-digital architectures. Leveraging the spatial- and frequency-wideband (dual-wideband) effects in massive MIMO scenarios, we derive a spatial-frequency channel model with dual-wideband effects that incorporates the multipath parameters, i.e., time delay, complex gain, angle of departure/arrival. We adopt a successive beam training scheme and formulate the training OFDM signal as a third-order low-rank tensor fitting a canonical polyadic (CP) model with factor matrices containing the channel parameters. Exploiting the Vandermonde nature of factor matrices, we propose a structured CP decomposition-based channel estimation strategy aided by the spatial smoothing method, where two dedicated algorithms with particular tensor modeling and parameter recovery operations are developed. The proposed scheme leverages standard linear algebra, and, hence, avoids the random initialization problem and iterative procedure. An analysis of the uniqueness condition of CP decomposition is also pursued. Simulation results indicate that the proposed strategy achieves enhanced estimation performance, which outperforms the traditional approaches in terms of accuracy, robustness and complexity.

Short-latency ocular-following responses: Weighted nonlinear summation predicts the outcome of a competition between two sine wave gratings moving in opposite directions.

We recorded horizontal ocular-following responses to pairs of superimposed vertical sine wave gratings moving in opposite directions in human subjects. This configuration elicits a nonlinear interaction: when the relative contrast of the gratings is changed, the response transitions abruptly between the responses elicited by either grating alone. We explore this interaction in pairs of gratings that differ in spatial and temporal frequency and show that all cases can be described as a weighted sum of the responses to each grating presented alone, where the weights are a nonlinear function of stimulus contrast: a nonlinear weighed summation model. The weights depended on the spatial and temporal frequency of the component grating. In many cases the dominant component was not the one that produced the strongest response when presented alone, implying that the neuronal circuits assigning weights precede the stages at which motor responses to visual motion are generated. When the stimulus area was reduced, the relationship between spatial frequency and weight shifted to higher frequencies. This finding may reflect a contribution from surround suppression. The nonlinear interaction is strongest when the two components have similar spatial frequencies, suggesting that the nonlinearity may reflect interactions within single spatial frequency channels. This framework can be extended to stimuli composed of more than two components: our model was able to predict the responses to stimuli composed of three gratings. That this relatively simple model successfully captures the ocular-following responses over a wide range of spatial/temporal frequency and contrast parameters suggests that these interactions reflect a simple mechanism.

The profile of attentional modulation to visual features.

Although it is well established that feature-based attention (FBA) can enhance an attended feature, how it modulates unattended features remains less clear. Previous studies have generally supported either a graded profile as predicted by the feature-similarity gain model or a nonmonotonic profile predicted by the surround suppression model. To reconcile these different views, we systematically measured the attentional profile in three basic feature dimensions—orientation, motion direction, and spatial frequency. In three experiments, we instructed participants to detect a coherent feature signal against noise under attentional or neutral condition. Our results support a nonmonotonic hybrid model of attentional modulation consisting of feature-similarity gain and surround suppression for orientation and motion direction. For spatial frequency, we also found a similar nonmonotonic profile for higher frequencies than the attended frequency, but a lack of attentional modulation for lower frequencies than the attended frequency. The current findings can reconcile the discrepancies in the literature and suggest the hybrid model as a new framework for attentional modulation in feature space. In addition, a computational model incorporating known properties of spatial frequency channels and attentional modulations at the neural level reproduced the asymmetric attentional modulation, thus revealing a connection between surround suppression and the basic neural architecture of an early visual system.

ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features

With the rapid development of spaceborne imaging techniques, object detection in optical remote sensing imagery has drawn much attention in recent decades. While many advanced works have been developed with powerful learning algorithms, the incomplete feature representation still cannot meet the demand for effectively and efficiently handling image deformations, particularly objective scaling and rotation. To this end, we propose a novel object detection framework, called optical remote sensing imagery detector (ORSIm detector), integrating diverse channel features extraction, feature learning, fast image pyramid matching, and boosting strategy. ORSIm detector adopts a novel spatial-frequency channel feature (SFCF) by jointly considering the rotation-invariant channel features constructed in frequency domain and the original spatial channel features (e.g., color channel, gradient magnitude). Subsequently, we refine SFCF using learning-based strategy in order to obtain the high-level or semantically meaningful features. In the test phase, we achieve a fast and coarsely-scaled channel computation by mathematically estimating a scaling factor in the image domain. Extensive experimental results conducted on the two different airborne datasets are performed to demonstrate the superiority and effectiveness in comparison with previous state-of-the-art methods.

Spatial summation of broadband contrast.

Spatial summation of luminance contrast signals has historically been psychophysically measured with stimuli isolated in spatial frequency (i.e., narrowband). Here, we revisit the study of spatial summation with noise patterns that contain the naturalistic 1/fα distribution of contrast across spatial frequency. We measured amplitude spectrum slope (α) discrimination thresholds and verified if sensitivity to α improved according to stimulus size. Discrimination thresholds did decrease with an increase in stimulus size. These data were modeled with a summation model originally designed for narrowband stimuli (i.e., single detecting channel; Baker & Meese, 2011; Meese & Baker, 2011) that we modified to include summation across multiple-differently tuned-spatial frequency channels. To fit our data, contrast gain control weights had to be inversely related to spatial frequency (1/f); thus low spatial frequencies received significantly more divisive inhibition than higher spatial frequencies, which is a similar finding to previous models of broadband contrast perception (Haun & Essock, 2010; Haun & Peli, 2013). We found summation across spatial frequency channels to occur prior to summation across space, channel summation was near linear and summation across space was nonlinear. Our analysis demonstrates that classical psychophysical models can be adapted to computationally define visual mechanisms under broadband visual input, with the adapted models offering novel insight on the integration of signals across channels and space.