Receptive Field Size Research Articles

Binocular receptive-field construction in the primary visual cortex

ON and OFF thalamic afferents from the two eyes converge in the primary visual cortex to form binocular receptive fields. The receptive fields need to be diverse to sample our visual world but also similar across eyes to achieve binocular fusion. It is currently unknown how the cortex balances these competing needs between receptive-field diversity and similarity. Our results demonstrate that receptive fields in the cat visual cortex are binocularly matched with exquisite precision for retinotopy, orientation/direction preference, orientation/direction selectivity, response latency, and ON-OFF polarity/structure. Specifically, the average binocular mismatches in retinotopy and ON-OFF structure are tightly restricted to 1/20 and 1/5 of the average receptive-field size but are still large enough to generate all types of binocular disparity tuning. Based on these results, we conclude that cortical receptive fields are binocularly matched with the high precision needed to facilitate binocular fusion while allowing restricted mismatches to process visual depth.

Convolutional neural networks develop major organizational principles of early visual cortex when enhanced with retinal sampling.

Primate visual cortex exhibits key organizational principles: cortical magnification, eccentricity-dependent receptive field size and spatial frequency tuning as well as radial bias. We provide compelling evidence that these principles arise from the interplay of the non-uniform distribution of retinal ganglion cells, and a quasi-uniform convergence rate from the retina to the cortex. We show that convolutional neural networks outfitted with a retinal sampling layer, which resamples images according to retinal ganglion cell density, develop these organizational principles. Surprisingly, our results indicate that radial bias is spatial-frequency dependent and only manifests for high spatial frequencies. For low spatial frequencies, the bias shifts towards orthogonal orientations. These findings introduce a novel hypothesis about the origin of radial bias. Quasi-uniform convergence limits the range of spatial frequencies (in retinal space) that can be resolved, while retinal sampling determines the spatial frequency content throughout the retina.

BEV-MAE: Bird’s Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios

Existing LiDAR-based 3D object detection methods for autonomous driving scenarios mainly adopt the training-from-scratch paradigm. Unfortunately, this paradigm heavily relies on large-scale labeled data, whose collection can be expensive and time-consuming. Self-supervised pre-training is an effective and desirable way to alleviate this dependence on extensive annotated data. In this work, we present BEV-MAE, an efficient masked autoencoder pre-training framework for LiDAR-based 3D object detection in autonomous driving. Specifically, we propose a bird's eye view (BEV) guided masking strategy to guide the 3D encoder learning feature representation in a BEV perspective and avoid complex decoder design during pre-training. Furthermore, we introduce a learnable point token to maintain a consistent receptive field size of the 3D encoder with fine-tuning for masked point cloud inputs. Based on the property of outdoor point clouds in autonomous driving scenarios, i.e., the point clouds of distant objects are more sparse, we propose point density prediction to enable the 3D encoder to learn location information, which is essential for object detection. Experimental results show that BEV-MAE surpasses prior state-of-the-art self-supervised methods and achieves a favorably pre-training efficiency. Furthermore, based on TransFusion-L, BEV-MAE achieves new state-of-the-art LiDAR-based 3D object detection results, with 73.6 NDS and 69.6 mAP on the nuScenes benchmark. The source code will be released at https://github.com/VDIGPKU/BEV-MAE.

Efficient Look-Up Table from Expanded Convolutional Network for Accelerating Image Super-resolution

The look-up table (LUT) has recently shown its practicability and effectiveness in super-resolution (SR) tasks due to its low computational cost and hardware independence. However, most existing methods focus on improving the performance of SR, neglecting the demand for high-speed SR on low-computational edge devices. In this paper, we propose an efficient expanded convolution (EC) layer, which expands the output size of regular convolution to enlarge the receptive field (RF) indirectly. It can increase the size of the LUT corresponding to the network linearly with the increase of RF. Additionally, after introducing the EC, multiple LUTs are merged into one LUT, achieving faster running speed while maintaining SR performance. More specifically, we expand the coverage of the convolutional output so that the output at the current position covers the target position and its surroundings, forming an overlapping sliding window at the output end. We sum up the overlapping parts of the sliding window as the output, thereby achieving the effect of enlarging the RF size. Moreover, by expanding the numerical range of the accumulated results and rescaling them to [0,255], the method can mitigate the error caused by quantization output. Experiments indicate that the proposed method performs better than the baseline method and is faster than other LUT-based SR methods.

Multi-Level Attention Split Network: A Novel Malaria Cell Detection Algorithm

Malaria is one of the major global health threats. Microscopic examination has been designated as the “gold standard” for malaria detection by the World Health Organization. However, it heavily relies on the experience of doctors, resulting in long diagnosis time, low efficiency, and a high risk of missed or misdiagnosed cases. To alleviate the pressure on healthcare workers and achieve automated malaria detection, numerous target detection models have been applied to the blood smear examination for malaria cells. This paper introduces the multi-level attention split network (MAS-Net) that improves the overall detection performance by addressing the issues of information loss for small targets and mismatch between the detection receptive field and target size. Therefore, we propose the split contextual attention structure (SPCot), which fully utilizes contextual information and avoids excessive channel compression operations, reducing information loss and improving the overall detection performance of malaria cells. In the shallow detection layer, we introduce the multi-scale receptive field detection head (MRFH), which better matches targets of different scales and provides a better detection receptive field, thus enhancing the performance of malaria cell detection. On the NLM—Malaria Dataset provided by the National Institutes of Health, the improved model achieves an average accuracy of 75.9% in the public dataset of Plasmodium vivax (malaria)-infected human blood smear. Considering the practical application of the model, we introduce the Performance-aware Approximation of Global Channel Pruning (PAGCP) to compress the model size while sacrificing a small amount of accuracy. Compared to other state-of-the-art (SOTA) methods, the proposed MAS-Net achieves competitive results.

Lightweight camouflaged object detection model based on multilevel feature fusion

The intrinsic similarity between camouflaged objects and background environment impedes the automatic detection/segmentation of camouflaged objects, and novel network architectures for deep learning are promising to overcome this challenge and improve detection accuracy. However, these existing network architectures for distinguishing between camouflaged objects and their backgrounds do not account for the constraint of detection speed, which results in high computational complexity and the inability to meet the requirements of rapid detection. Therefore, based on the human visual system, this study proposes a single-stage lightweight camouflage object detection network using multilevel feature fusion, integrating features of various feature layers and receptive field sizes. Using three benchmark datasets for normal camouflaged objects, the lightweight network (LINet) model demonstrated an accuracy superior to those of six existing mainstream camouflaged object detection methods. Its detection speed, 126.3 frames per second, is significantly higher than those of the existing mainstream methods, enabling rapid detection with a maximum increase of 187.62%. The accuracy of LINet is the minimum and maximum for Resnet101 and Resnet152, respectively. These findings pave the way for diverse applications of camouflaged target detection algorithms.

Astroconformer: The prospects of analysing stellar light curves with transformer-based deep learning models

ABSTRACT Stellar light curves contain valuable information about oscillations and granulation, offering insights into stars’ internal structures and evolutionary states. Traditional asteroseismic techniques, primarily focused on power spectral analysis, often overlook the crucial phase information in these light curves. Addressing this gap, recent machine learning applications, particularly those using Convolutional Neural Networks (CNNs), have made strides in inferring stellar properties from light curves. However, CNNs are limited by their localized feature extraction capabilities. In response, we introduce Astroconformer, a Transformer-based deep learning framework, specifically designed to capture long-range dependencies in stellar light curves. Our empirical analysis centres on estimating surface gravity (log g), using a data set derived from single-quarter Kepler light curves with log g values ranging from 0.2 to 4.4. Astroconformer demonstrates superior performance, achieving a root-mean-square-error (RMSE) of 0.017 dex at log g ≈ 3 in data-rich regimes and up to 0.1 dex in sparser areas. This performance surpasses both K-nearest neighbour models and advanced CNNs. Ablation studies highlight the influence of receptive field size on model effectiveness, with larger fields correlating to improved results. Astroconformer also excels in extracting νmax with high precision. It achieves less than 2 per cent relative median absolute error for 90-d red giant light curves. Notably, the error remains under 3 per cent for 30-d light curves, whose oscillations are undetectable by a conventional pipeline in 30 per cent cases. Furthermore, the attention mechanisms in Astroconformer align closely with the characteristics of stellar oscillations and granulation observed in light curves.

GABAergic Inhibition Controls Receptive Field Size, Sensitivity, and Contrast Preference of Direction Selective Retinal Ganglion Cells Near the Threshold of Vision.

Information about motion is encoded by direction-selective retinal ganglion cells (DSGCs). These cells reliably transmit this information across a broad range of light levels, spanning moonlight to sunlight. Previous work indicates that adaptation to low light levels causes heterogeneous changes to the direction tuning of ON-OFF (oo)DSGCs and suggests that superior-preferring ON-OFF DSGCs (s-DSGCs) are biased toward detecting stimuli rather than precisely signaling direction. Using a large-scale multielectrode array, we measured the absolute sensitivity of ooDSGCs and found that s-DSGCs are 10-fold more sensitive to dim flashes of light than other ooDSGCs. We measured their receptive field (RF) sizes and found that s-DSGCs also have larger receptive fields than other ooDSGCs; however, the size difference does not fully explain the sensitivity difference. Using a conditional knock-out of gap junctions and pharmacological manipulations, we demonstrate that GABA-mediated inhibition contributes to the difference in absolute sensitivity and receptive field size at low light levels, while the connexin36-mediated gap junction coupling plays a minor role. We further show that under scotopic conditions, ooDSGCs exhibit only an ON response, but pharmacologically removing GABA-mediated inhibition unmasks an OFF response. These results reveal that GABAergic inhibition controls and differentially modulates the responses of ooDSGCs under scotopic conditions.

Research on Retinal Vessel Segmentation Algorithm Based on a Modified U-Shaped Network

Due to the limitations of traditional retinal blood vessel segmentation algorithms in feature extraction, vessel breakage often occurs at the end. To address this issue, a retinal vessel segmentation algorithm based on a modified U-shaped network is proposed in this paper. This algorithm can extract multi-scale vascular features and perform segmentation in an end-to-end manner. First, in order to improve the low contrast of the original image, pre-processing methods are employed. Second, a multi-scale residual convolution module is employed to extract image features of different granularities, while residual learning improves feature utilization efficiency and reduces information loss. In addition, a selective kernel unit is incorporated into the skip connections to obtain multi-scale features with varying receptive field sizes achieved through soft attention. Subsequently, to further extract vascular features and improve processing speed, a residual attention module is constructed at the decoder stage. Finally, a weighted joint loss function is implemented to address the imbalance between positive and negative samples. The experimental results on the DRIVE, STARE, and CHASE_DB1 datasets demonstrate that MU-Net exhibits better sensitivity and a higher Matthew’s correlation coefficient (0.8197, 0.8051; STARE: 0.8264, 0.7987; CHASE_DB1: 0.8313, 0.7960) compared to several state-of-the-art methods.

A Multitask Network for Joint Multispectral Pansharpening on Diverse Satellite Data.

Despite the rapid advance in multispectral (MS) pansharpening, existing convolutional neural network (CNN)-based methods require training on separate CNNs for different satellite datasets. However, such a single-task learning (STL) paradigm often leads to overlooking any underlying correlations between datasets. Aiming at this challenging problem, a multitask network (MTNet) is presented to accomplish joint MS pansharpening in a unified framework for images acquired by different satellites. Particularly, the pansharpening process of each satellite is treated as a specific task, while MTNet simultaneously learns from all data obtained from these satellites following the multitask learning (MTL) paradigm. MTNet shares the generic knowledge between datasets via task-agnostic subnetwork (TASNet), utilizing task-specific subnetworks (TSSNets) to facilitate the adaptation of such knowledge to a certain satellite. To tackle the limitation of the local connectivity property of the CNN, TASNet incorporates Transformer modules to derive global information. In addition, band-aware dynamic convolutions (BDConvs) are proposed that can accommodate various ground scenes and bands by adjusting their respective receptive field (RF) size. Systematic experimental results over different datasets demonstrate that the proposed approach outperforms the existing state-of-the-art (SOTA) techniques.

A New Siamese Heterogeneous Convolutional Neural Networks Based on Attention Mechanism and Feature Pyramid.

Accuracy and speed are the most important indexes for evaluating many object tracking algorithms. However, when constructing a deep fully convolutional neural network (CNN), the use of deep network feature tracking will cause tracking drift due to the effects of convolution padding, receptive field (RF), and overall network step size. The speed of the tracker will also decrease. This article proposes a fully convolutional siamese network object tracking algorithm that combines the attention mechanism with the feature pyramid network (FPN), and uses heterogeneous convolution kernels to reduce the amount of calculations (FLOPs) and parameters. The tracker first uses a new fully CNN to extract image features, and introduces a channel attention mechanism in the feature extraction process to improve the representation ability of convolutional features. Then use the FPN to fuse the convolutional features of high and low layers, learn the similarity of the fused features, and train the fully CNNs. Finally, the heterogeneous convolutional kernel is used to replace the standard convolution kernel to improve the speed of the algorithm, thereby making up for the efficiency loss caused by the feature pyramid model. In this article, the tracker is experimentally verified and analyzed on the VOT-2017, VOT-2018, OTB-2013, and OTB-2015 datasets. The results show that our tracker has achieved better results than the state-of-the-art trackers.

ResGEM: Multi-scale Graph Embedding Network for Residual Mesh Denoising.

Mesh denoising is a crucial technology that aims to recover a high-fidelity 3D mesh from a noise-corrupted one. Deep learning methods, particularly graph convolutional networks (GCNs) based mesh denoisers, have demonstrated their effectiveness in removing various complex real-world noises while preserving authentic geometry. However, it is still a quite challenging work to faithfully regress uncontaminated normals and vertices on meshes with irregular topology. In this paper, we propose a novel pipeline that incorporates two parallel normal-aware and vertex-aware branches to achieve a balance between smoothness and geometric details while maintaining the flexibility of surface topology. We introduce ResGEM, a new GCN, with multi-scale embedding modules and residual decoding structures to facilitate normal regression and vertex modification for mesh denoising. To effectively extract multi-scale surface features while avoiding the loss of topological information caused by graph pooling or coarsening operations, we encode the noisy normal and vertex graphs using four edge-conditioned embedding modules (EEMs) at different scales. This allows us to obtain favorable feature representations with multiple receptive field sizes. Formulating the denoising problem into a residual learning problem, the decoder incorporates residual blocks to accurately predict true normals and vertex offsets from the embedded feature space. Moreover, we propose novel regularization terms in the loss function that enhance the smoothing and generalization ability of our network by imposing constraints on normal consistency. Comprehensive experiments have been conducted to demonstrate the superiority of our method over the state-of-the-art on both synthetic and real-scanned datasets.

An Enhanced Dual-Stream Network Using Multi-Source Remote Sensing Imagery for Water Body Segmentation

Accurate surface water mapping is crucial for rationalizing water resource utilization and maintaining ecosystem sustainability. However, the diverse shapes and scales of water bodies pose challenges in automatically extracting them from remote sensing images. Existing methods suffer from inaccurate lake boundary extraction, inconsistent results, and failure to detect small rivers. In this study, we propose a dual-stream parallel feature aggregation network to address these limitations. Our network effectively combines global information interaction from the Swin Transformer network with deep local information integration from Convolutional Neural Networks (CNNs). Moreover, we introduce a deformable convolution-based attention mechanism module (D-CBAM) that adaptively adjusts receptive field size and shape, highlights important channels in feature maps automatically, and enhances the expressive ability of our network. Additionally, we incorporate a Feature Pyramid Attention (FPA) module during the advanced coding stage for multi-scale feature learning to improve segmentation accuracy for small water bodies. To verify the effectiveness of our method, we chose the Yellow River Basin in China as the research area and used Sentinel-2 and Sentinel-1 satellite images as well as manually labelling samples to construct a dataset. On this dataset, our method achieves a 93.7% F1 score, which is a significant improvement compared with other methods. Finally, we use the proposed method to map the seasonal and permanent water bodies in the Yellow River Basin in 2021 and compare it with existing water bodies. The results show that our method has certain advantages in mapping large-scale water bodies, which not only ensures the overall integrity but also retains local details.

Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

U structured network with three encoding paths for breast tumor segmentation

Breast ultrasound segmentation remains challenging because of the blurred boundaries, irregular shapes, and the presence of shadowing and speckle noise. The majority of approaches stack convolutional layers to extract advanced semantic information, which makes it difficult to handle multiscale issues. To address those issues, we propose a three-path U-structure network (TPUNet) that consists of a three-path encoder and an attention-based feature fusion block (AFF Block). Specifically, instead of simply stacking convolutional layers, we design a three-path encoder to capture multiscale features through three independent encoding paths. Additionally, we design an attention-based feature fusion block to weight and fuse feature maps in spatial and channel dimensions. The AFF Block encourages different paths to compete with each other in order to synthesize more salient feature maps. We also investigate a hybrid loss function for reducing false negative regions and refining the boundary segmentation, as well as the deep supervision to guide different paths to capture the effective features under the corresponding receptive field sizes. According to experimental findings, our proposed TPUNet achieves more excellent results in terms of quantitative analysis and visual quality than other rival approaches.

Poster Session I: Digital dual-Purkinje-image eye tracking enables precise determination of visual receptive fields in fixating macaques.

Understanding the relationship between visual stimuli and neural activity is a fundamental goal in visual neuroscience. However, the study of visual neurophysiology in awake primates is complicated by the constant occurrence of eye movements, even during periods of nominal fixation. To address this challenge, we adapted a recently developed high-resolution digital dual-Purkinje-image (dDPI) eye tracker (Wu et al., 2023) for use with macaque monkeys. In addition to tracking the Purkinje images, we simultaneously estimate the pupil center and size: a first for video eye tracking. We then sought to evaluate the efficacy of dDPI eye tracking for studying visual processing in fixating macaques by recording single neurons from the lateral geniculate nucleus while a spatially-correlated noise stimulus was displayed. Our analyses show that, as a result of properly accounting for eye movements post-hoc, the predictive performance of generalized linear models improves and the estimates center radii contract to values equal to or smaller than those reported in the literature. Notably, correcting for eye movements using the locations of the pupil center and corneal reflection-the standard method in video eye tracking-yielded worse model fits and larger receptive field sizes. This finding implies that the pupil center is an inaccurate reporter of small eye movements, while the Purkinje images may be veridical during fixations.

PEA-Net: A progressive edge information aggregation network for vessel segmentation

Automatic vessel segmentation is a critical area of research in medical image analysis, as it can greatly assist doctors in accurately and efficiently diagnosing vascular diseases. However, accurately extracting the complete vessel structure from images remains a challenge due to issues such as uneven contrast and background noise. Existing methods primarily focus on segmenting individual pixels and often fail to consider vessel features and morphology. As a result, these methods often produce fragmented results and misidentify vessel-like background noise, leading to missing and outlier points in the overall segmentation. To address these issues, this paper proposes a novel approach called the progressive edge information aggregation network for vessel segmentation (PEA-Net). The proposed method consists of several key components. First, a dual-stream receptive field encoder (DRE) is introduced to preserve fine structural features and mitigate false positive predictions caused by background noise. This is achieved by combining vessel morphological features obtained from different receptive field sizes. Second, a progressive complementary fusion (PCF) module is designed to enhance fine vessel detection and improve connectivity. This module complements the decoding path by combining features from previous iterations and the DRE, incorporating nonsalient information. Additionally, segmentation-edge decoupling enhancement (SDE) modules are employed as decoders to integrate upsampling features with nonsalient information provided by the PCF. This integration enhances both edge and segmentation information. The features in the skip connection and decoding path are iteratively updated to progressively aggregate fine structure information, thereby optimizing segmentation results and reducing topological disconnections. Experimental results on multiple datasets demonstrate that the proposed PEA-Net model and strategy achieve optimal performance in both pixel-level and topology-level metrics.

Organization of pRF size along the AP axis of the hippocampus and adjacent medial temporal cortex is related to specialization for scenes versus faces.

The hippocampus is largely recognized for its integral contributions to memory processing. By contrast, its role in perceptual processing remains less clear. Hippocampal properties vary along the anterior-posterior (AP) axis. Based on past research suggesting a gradient in the scale of features processed along the AP extent of the hippocampus, the representations have been proposed to vary as a function of granularity along this axis. One way to quantify such granularity is with population receptive field (pRF) size measured during visual processing, which has so far received little attention. In this study, we compare the pRF sizes within the hippocampus to its activation for images of scenes versus faces. We also measure these functional properties in surrounding medial temporal lobe (MTL) structures. Consistent with past research, we find pRFs to be larger in the anterior than in the posterior hippocampus. Critically, our analysis of surrounding MTL regions, the perirhinal cortex, entorhinal cortex, and parahippocampal cortex shows a similar correlation between scene sensitivity and larger pRF size. These findings provide conclusive evidence for a tight relationship between the pRF size and the sensitivity to image content in the hippocampus and adjacent medial temporal cortex.

Effect of theanine on the hyperexcitability of trigeminal secondary nociceptive neurons following orofacial inflammation in rats.

The present in vivo study investigated whether systemic administration of theanine attenuates the inflammation-induced hyperexcitability of trigeminal spinal nucleus caudalis (SpVc) neurons associated with hyperalgesia. Complete Freund's adjuvant (CFA) was injected into the whisker pads of 24 rats to induce inflammation, and then mechanical stimulation was applied to the orofacial area to assess the threshold of escape. The mechanical threshold was statistically significantly lower in CFA-inflamed rats compared to uninjected naïve rats, and this lowered threshold returned to control levels after 2 days of theanine administration. The mean discharge frequency of SpVc wide-dynamic range (WDR) neurons to mechanical stimuli in anesthetized CFA-inflamed rats was statistically significantly lower after two days of theanine administration. In addition, the increased mean spontaneous discharge of SpVc WDR neurons in CFA-inflamed rats statistically significantly decreased after theanine administration. Similarly, theanine restored the expanded mean receptive field size in CFA-inflamed rats to control levels. Taken together, these results suggest that administration of theanine attenuates inflammatory hyperalgesia associated with hyperexcitability of nociceptive SpVc WDR neurons. These findings support the potential of theanine as a therapeutic agent in complementary alternative medicine strategies to prevent inflammatory hyperalgesia.

E2LNet: An Efficient and Effective Lightweight Network for Panoramic Depth Estimation.

Monocular panoramic depth estimation has various applications in robotics and autonomous driving due to its ability to perceive the entire field of view. However, panoramic depth estimation faces two significant challenges: global context capturing and distortion awareness. In this paper, we propose a new framework for panoramic depth estimation that can simultaneously address panoramic distortion and extract global context information, thereby improving the performance of panoramic depth estimation. Specifically, we introduce an attention mechanism into the multi-scale dilated convolution and adaptively adjust the receptive field size between different spatial positions, designing the adaptive attention dilated convolution module, which effectively perceives distortion. At the same time, we design the global scene understanding module to integrate global context information into the feature maps generated using the feature extractor. Finally, we trained and evaluated our model on three benchmark datasets which contains the virtual and real-world RGB-D panorama datasets. The experimental results show that the proposed method achieves competitive performance, comparable to existing techniques in both quantitative and qualitative evaluations. Furthermore, our method has fewer parameters and more flexibility, making it a scalable solution in mobile AR.