Receptive Field Size Research Articles

Receptive Field Size as a Key Design Parameter for Ultrasound Image Segmentation with U-Net.

Automatic and accurate segmentation of medical images is an important task due to the direct impact of this procedure on both disease diagnosis and treatment. Segmentation of ultrasound (US) imaging is particularly challenging due to the presence of speckle noise. Recent deep learning approaches have demonstrated remarkable findings in image segmentation tasks, including segmentation of US images. However, many of the newly proposed structures are either task specific and suffer from poor generalization, or are computationally expensive. In this paper, we show that the receptive field plays a more significant role in the network's performance compared to the network's depth or the number of parameters. We further show that by controlling the size of the receptive field, a deep network can instead be replaced by a shallow network.

ResDUnet: Residual Dilated UNet for Left Ventricle Segmentation from Echocardiographic Images.

Echocardiography is the modality of choice for the assessment of left ventricle function. Left ventricle is responsible for pumping blood rich in oxygen to all body parts. Segmentation of this chamber from echocardiographic images is a challenging task, due to the ambiguous boundary and inhomogeneous intensity distribution. In this paper we propose a novel deep learning model named ResDUnet. The model is based on U-net incorporated with dilated convolution, where residual blocks are employed instead of the basic U-net units to ease the training process. Each block is enriched with squeeze and excitation unit for channel-wise attention and adaptive feature re-calibration. To tackle the problem of left ventricle shape and size variability, we chose to enrich the process of feature concatenation in U-net by integrating feature maps generated by cascaded dilation. Cascaded dilation broadens the receptive field size in comparison with traditional convolution, which allows the generation of multi-scale information which in turn results in a more robust segmentation. Performance measures were evaluated on a publicly available dataset of 500 patients with large variability in terms of quality and patients pathology. The proposed model shows a dice similarity increase of 8.4% when compared to deeplabv3 and 1.2% when compared to the basic U-net architecture. Experimental results demonstrate the potential use in clinical domain.

Retinal Image Formation and Sampling in a Three-Dimensional World.

In this review, I develop an empirically based model of optical image formation by the human eye, followed by neural sampling by retinal ganglion cells, to demonstrate the perceptual effects of blur, aliasing, and distortion of visual space in the brain. The optical model takes account of ocular aberrations and their variation across the visual field, in addition to variations of defocus due to variation of target vergence in three-dimensional scenes. Neural sampling by retinal ganglion cells with receptive field size and spacing that increases with eccentricity is used to visualize the neural image carried by the optic nerve to the brain. Anatomical parameters are derived from psychophysical studies of sampling-limited visual resolution of sinusoidal interference fringes. Retinotopic projection of the neural image onto brainstem nuclei reveals features of the neural image in a perceptually uniform brain space where location and size of visual objects may be measured by counting neurons.

The pericyte connectome: spatial precision of neurovascular coupling is driven by selective connectivity maps of pericytes and endothelial cells and is disrupted in diabetes

Functional hyperemia, or the matching of blood flow with activity, directs oxygen and nutrients to regionally firing neurons. The mechanisms responsible for this spatial accuracy remain unclear but are critical for brain function and establish the diagnostic resolution of BOLD-fMRI. Here, we described a mosaic of pericytes, the vasomotor capillary cells in the living retina. We then tested whether this net of pericytes and surrounding neuroglia predicted a connectivity map in response to sensory stimuli. Surprisingly, we found that these connections were not only selective across cell types, but also highly asymmetric spatially. First, pericytes connected predominantly to other neighboring pericytes and endothelial cells, and less to arteriolar smooth muscle cells, and not to surrounding neurons or glia. Second, focal, but not global stimulation evoked a directional vasomotor response by strengthening connections along the feeding vascular branch. This activity required local NO signaling and occurred by means of direct coupling via gap junctions. By contrast, bath application of NO or diabetes, a common microvascular pathology, not only weakened the vascular signaling but also abolished its directionality. We conclude that the exclusivity of neurovascular interactions may thus establish spatial accuracy of blood delivery with the precision of the neuronal receptive field size, and is disrupted early in diabetes.

Learning lightweight Multi-Scale Feedback Residual network for single image super-resolution

In the past years, convolutional neural networks (CNNs) have demonstrated great success for single image super-resolution (SISR). However, existing CNNs for SISR generally have two limitations: (1) the network depth is very deep, which not only weakens the information flow from bottom to top but also has a heavy model capacity; (2) the network architectures are often feed-forward, which prevent the previous layers capturing the useful information from the following layers, limiting the feature learning capability. To address these issues, this paper presents a lightweight Multi-scale Feedback Residual network for SISR. Specifically, we design a lightweight feedback-based recurrent neural network (FRNN) tailored to SISR. The FRNN is consists of a series of recursive Densely-Connected Blocks (DCBs) with the Low-Resolution (LR) image features and the output of the former DCB as inputs. Each DCB adaptively fuses multi-level features from the side-output intermediate feature maps to generate a powerful feature representation. Meanwhile, the DCB cascades a set of Multi-scale Residual Blocks (MRBs), each of which has an enlarged field of view to fully capture multi-scale context information. Moreover, the MRB has a novel Multi-Kernel Fusion Block (MKFB) design, which can dynamically adjust the receptive field size of the output feature representation based on the multi-scale inputs. The whole network of our MFRSR is lightweight with only ∼4.5M parameters, but achieves favorable performance on five benchmark datasets compared to the state-of-the-art methods in terms of PSNR and SSIM.

Multibranch Selective Kernel Networks for Hyperspectral Image Classification

Convolutional neural networks (CNNs) have demonstrated excellent performance in hyperspectral image (HSI) classification. However, tuning some critical hyperparameters of a CNN-such as the receptive field (RF) size-presents a major challenge due to the presence of features with different scales in HSIs. Contrary to the conventional design of CNNs, which fixes the RF size, it has been proven that the RF size is modulated by the stimulus and hence, depends on the scene being considered. Such a dilemma has been rarely considered in CNN design. In this letter, a new multibranch selective kernel network (MSKNet) is introduced, in which the input image is convolved using different RF sizes to create multiple branches so that the effect of each branch is adjusted by an attention mechanism according to the input contrast. As a result, our newly developed MSKNet is capable of modeling different scales. Our experimental results, conducted on three widely used HSIs, reveal that the MSKNet can outperform state-of-the-art CNNs in the context of HSI classification problems. The source code of our newly developed MSKNet is available from: https://github.com/mhaut/MSKNet-HSI.

Event-Based Eccentric Motion Detection Exploiting Time Difference Encoding.

Attentional selectivity tends to follow events considered as interesting stimuli. Indeed, the motion of visual stimuli present in the environment attract our attention and allow us to react and interact with our surroundings. Extracting relevant motion information from the environment presents a challenge with regards to the high information content of the visual input. In this work we propose a novel integration between an eccentric down-sampling of the visual field, taking inspiration from the varying size of receptive fields (RFs) in the mammalian retina, and the Spiking Elementary Motion Detector (sEMD) model. We characterize the system functionality with simulated data and real world data collected with bio-inspired event driven cameras, successfully implementing motion detection along the four cardinal directions and diagonally.

Generative Feedback Explains Distinct Brain Activity Codes for Seen and Mental Images

The relationship between mental imagery and vision is a long-standing problem in neuroscience. Currently, it is not known whether differences between the activity evoked during vision and reinstated during imagery reflect different codes for seen and mental images. To address this problem, we modeled mental imagery in the human brain as feedback in a hierarchical generative network. Such networks synthesize images by feeding abstract representations from higher to lower levels of the network hierarchy. When higher processing levels are less sensitive to stimulus variation than lower processing levels, as in the human brain, activity in low-level visual areas should encode variation in mental images with less precision than seen images. To test this prediction, we conducted an fMRI experiment in which subjects imagined and then viewed hundreds of spatially varying naturalistic stimuli. To analyze these data, we developed imagery-encoding models. These models accurately predicted brain responses to imagined stimuli and enabled accurate decoding of their position and content. They also allowed us to compare, for every voxel, tuning to seen and imagined spatial frequencies, as well as the location and size of receptive fields in visual and imagined space. We confirmed our prediction, showing that, in low-level visual areas, imagined spatial frequencies in individual voxels are reduced relative to seen spatial frequencies and that receptive fields in imagined space are larger than in visual space. These findings reveal distinct codes for seen and mental images and link mental imagery to the computational abilities of generative networks.

Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy

In this paper, we present a new Deep Convolutional Neural Networks (CNNs) dedicated to fully automatic segmentation of Glioblastoma brain tumors with high- and low-grade. The proposed CNNs model is inspired by the Occipito-Temporal pathway which has a special function called selective attention that uses different receptive field sizes in successive layers to figure out the crucial objects in a scene. Thus, using selective attention technique to develop the CNNs model, helps to maximize the extraction of relevant features from MRI images. We have also addressed two more issues: class-imbalance, and the spatial relationship among image Patches. To address the first issue, we propose two steps: an equal sampling of images Patches and an experimental analysis of the effect of weighted cross-entropy loss function on the segmentation results. In addition, to overcome the second issue, we have studied the effect of Overlapping Patches against Adjacent Patches where the Overlapping Patches show better segmentation results due to the introduction of the global context as well as the local features of the image Patches compared to the conventionnel Adjacent Patches. Our experiment results are reported on BRATS-2018 dataset where our End-to-End Deep Learning model achieved state-of-the-art performance. The median Dice score of our fully automatic segmentation model is 0.90, 0.83, 0.83 for the whole tumor, tumor core, and enhancing tumor respectively compared to the Dice score of radiologist, that is in the range 74% - 85%. Moreover, our proposed CNNs model is not only computationally efficient at inference time, but it could segment the whole brain on average 12 seconds. Finally, the proposed Deep Learning model provides an accurate and reliable segmentation result, and that makes it suitable for adopting in research and as a part of different clinical settings.

Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution

Spectral super-resolution (SSR) aims at generating a hyperspectral image (HSI) from a given RGB image. Recently, a promising direction is to learn a complicated mapping function from the RGB image to the HSI counterpart using a deep convolutional neural network. This essentially involves mapping the RGB context within a size-specific receptive field centered at each pixel to its spectrum in the HSI. The focus thereon is to appropriately determine the receptive field size and establish the mapping function from RGB context to the corresponding spectrum. Due to their differences in category or spatial position, pixels in HSIs often require different-sized receptive fields and distinct mapping functions. However, few efforts have been invested to explicitly exploit this prior.To address this problem, we propose a pixel-aware deep function-mixture network for SSR, which is composed of a new class of modules, termed function-mixture (FM) blocks. Each FM block is equipped with some basis functions, i.e., parallel subnets of different-sized receptive fields. Besides, it incorporates an extra subnet as a mixing function to generate pixel-wise weights, and then linearly mixes the outputs of all basis functions with those generated weights. This enables us to pixel-wisely determine the receptive field size and the mapping function. Moreover, we stack several such FM blocks to further increase the flexibility of the network in learning the pixel-wise mapping. To encourage feature reuse, intermediate features generated by the FM blocks are fused in late stage, which proves to be effective for boosting the SSR performance. Experimental results on three benchmark HSI datasets demonstrate the superiority of the proposed method.

Spatial resolution of local field potential signals in macaque V4

Objective. An important challenge for the development of cortical visual prostheses is to generate spatially localized percepts of light, using artificial stimulation. Such percepts are called phosphenes, and the goal of prosthetic applications is to generate a pattern of phosphenes that matches the structure of the retinal image. A preliminary step in this process is to understand how the spatial positions of phosphene-like visual stimuli are encoded in the distributed activity of cortical neurons. The spatial resolution with which the distributed responses discriminate positions puts a limit on the capability of visual prosthesis devices to induce phosphenes at multiple positions. While most previous prosthetic devices have targeted the primary visual cortex, the extrastriate cortex has the advantage of covering a large part of the visual field with a smaller amount of cortical tissue, providing the possibility of a more compact implant. Here, we studied how well ensembles of Local Field Potentials (LFPs) and Multiunit activity (MUA) responses from extrastriate cortical visual area V4 of a behaving macaque monkey can discriminate between two-dimensional spatial positions. Approach. We used support vector machines (SVM) to determine the capabilities of LFPs and MUA to discriminate responses to phosphene-like stimuli (probes) at different spatial separations. We proposed a selection strategy based on the combined responses of multiple electrodes and used the linear learning weights to find the minimum number of electrodes for fine and coarse discriminations. We also measured the contribution of correlated trial-to-trial variability in the responses to the discrimination performance for MUA and LFP. Main results. We found that despite the large receptive field sizes in V4, the combined responses from multiple sites, whether MUA or LFP, are capable of fine and coarse discrimination of positions. Our electrode selection procedure significantly increased discrimination performance while reducing the required number of electrodes. Analysis of noise correlations in MUA and LFP responses showed that noise correlations in LFPs carry more information about spatial positions. Significance. This study determined the coding strategy for fine discrimination, suggesting that spatial positions could be well localized with patterned stimulation in extrastriate area V4. It also provides a novel approach to build a compact prosthesis with relatively few electrodes, which has the potential advantage of reducing tissue damage in real applications.

Dynamic Update Siamese Networks with Deeper Features

Visual object tracking has been a concern topic these years, and many trackers have achieved good results in various fields. These researches and breakthroughs have made many improvements to solve problems such as drift, lighting, deformation and occlusion. In this paper, we improve the structure of the AlexNet[1] network by designing the three important influencing factors of the receptive field size, total network step size, and feature filling of the twin network. Apart from this, we add a smoothing matrices and a background suppression matrices to effectively learn the features of the first few frames as much as possible. Fuse multilayer feature elements can learn online about target appearance changes and background suppression, and we train them by using continuous video sequences.

Social media sentiment analysis through parallel dilated convolutional neural network for smart city applications

Deep Learning is considered to leverage smart cities through social media sentiment analysis. The digital content in social media can be used for many smart city applications (SCAs)11Smart city applications.. Classical convolutional neural networks (CNNs) are challenging to parallelize and insufficient to capture long term contextual semantic features for sentiment analysis. In this perspective, this paper initially proposes a domain-specific distributed word representation (DS-DWR)22Domain Specific distributed word representation. with a considerably small corpus size induced from textual resources in social media. In DS-DWR, different Distributed Word Representations are concatenated to builds rich representations over the input sequence, which is worthwhile for infrequent and unseen terms. Second, a dilated convolutional neural network (D-CNN)33Dilated Convolutional Neural Network., which is composed of three parallel dilated convolutional neural network (PD-CNN)44Parallel dilated convolutional neural network. layers and a global average pooling (GAP)55Global Average Pooling. layer. Our considered parallel dilated convolution reduces dimension and incorporates an extension in the size of receptive fields without the loss of local information. Further, the long-term contextual semantic information is achieved by the use of different dilation rates. Experiments demonstrate that our architecture accomplishes comparable results with multiple hyperparameters tuning for better parallelism which leads to the minimized computational cost.

Retina inspired tone mapping method for high dynamic range images.

The limited dynamic range of regular screens restricts the display of high dynamic range (HDR) images. Inspired by retinal processing mechanisms, we propose a tone mapping method to address this problem. In the retina, horizontal cells (HCs) adaptively adjust their receptive field (RF) size based on the local stimuli to regulate the visual signals absorbed by photoreceptors. Using this adaptive mechanism, the proposed method compresses the dynamic range locally in different regions, and has the capability of avoiding halo artifacts around the edges of high luminance contrast. Moreover, the proposed method introduces the center-surround antagonistic RF structure of bipolar cells (BCs) to enhance the local contrast and details. Extensive experiments show that the proposed method performs robustly well on a wide variety of images, providing competitive results against the state-of-the-art methods in terms of visual inspection, objective metrics and observer scores.

Scale and translation-invariance for novel objects in human vision

Though the range of invariance in recognition of novel objects is a basic aspect of human vision, its characterization has remained surprisingly elusive. Here we report tolerance to scale and position changes in one-shot learning by measuring recognition accuracy of Korean letters presented in a flash to non-Korean subjects who had no previous experience with Korean letters. We found that humans have significant scale-invariance after only a single exposure to a novel object. The range of translation-invariance is limited, depending on the size and position of presented objects. To understand the underlying brain computation associated with the invariance properties, we compared experimental data with computational modeling results. Our results suggest that to explain invariant recognition of objects by humans, neural network models should explicitly incorporate built-in scale-invariance, by encoding different scale channels as well as eccentricity-dependent representations captured by neurons’ receptive field sizes and sampling density that change with eccentricity. Our psychophysical experiments and related simulations strongly suggest that the human visual system uses a computational strategy that differs in some key aspects from current deep learning architectures, being more data efficient and relying more critically on eye-movements.

Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network

Urban land cover classification for high-resolution images is a fundamental yet challenging task in remote sensing image analysis. Recently, deep learning techniques have achieved outstanding performance in high-resolution image classification, especially the methods based on deep convolutional neural networks (DCNNs). However, the traditional CNNs using convolution operations with local receptive fields are not sufficient to model global contextual relations between objects. In addition, multiscale objects and the relatively small sample size in remote sensing have also limited classification accuracy. In this paper, a relation-enhanced multiscale convolutional network (REMSNet) method is proposed to overcome these weaknesses. A dense connectivity pattern and parallel multi-kernel convolution are combined to build a lightweight and varied receptive field sizes model. Then, the spatial relation-enhanced block and the channel relation-enhanced block are introduced into the network. They can adaptively learn global contextual relations between any two positions or feature maps to enhance feature representations. Moreover, we design a parallel multi-kernel deconvolution module and spatial path to further aggregate different scales information. The proposed network is used for urban land cover classification against two datasets: the ISPRS 2D semantic labelling contest of Vaihingen and an area of Shanghai of about 143 km2. The results demonstrate that the proposed method can effectively capture long-range dependencies and improve the accuracy of land cover classification. Our model obtains an overall accuracy (OA) of 90.46% and a mean intersection-over-union (mIoU) of 0.8073 for Vaihingen and an OA of 88.55% and a mIoU of 0.7394 for Shanghai.

Unique Spatial Integration in Mouse Primary Visual Cortex and Higher Visual Areas.

Neurons in the visual system integrate over a wide range of spatial scales. This diversity is thought to enable both local and global computations. To understand how spatial information is encoded across the mouse visual system, we use two-photon imaging to measure receptive fields (RFs) and size-tuning in primary visual cortex (V1) and three downstream higher visual areas (HVAs: LM (lateromedial), AL (anterolateral), and PM (posteromedial)) in mice of both sexes. Neurons in PM, compared with V1 or the other HVAs, have significantly larger RF sizes and less surround suppression, independent of stimulus eccentricity or contrast. To understand how this specialization of RFs arises in the HVAs, we measured the spatial properties of V1 inputs to each area. Spatial integration of V1 axons was remarkably similar across areas and significantly different from the tuning of neurons in their target HVAs. Thus, unlike other visual features studied in this system, specialization of spatial integration in PM cannot be explained by specific projections from V1 to the HVAs. Further, the differences in RF properties could not be explained by differences in convergence of V1 inputs to the HVAs. Instead, our data suggest that distinct inputs from other areas or connectivity within PM may support the area's unique ability to encode global features of the visual scene, whereas V1, LM, and AL may be more specialized for processing local features.SIGNIFICANCE STATEMENT Surround suppression is a common feature of visual processing whereby large stimuli are less effective at driving neuronal responses than smaller stimuli. This is thought to enhance efficiency in the population code and enable higher-order processing of visual information, such as figure-ground segregation. However, this comes at the expense of global computations. Here we find that surround suppression is not equally represented across mouse visual areas: primary visual cortex has substantially more surround suppression than higher visual areas, and one higher area has significantly less suppression than two others examined, suggesting that these areas have distinct functional roles. Thus, we have identified a novel dimension of specialization in the mouse visual cortex that may enable both local and global computations.

Spatially-Adaptive Filter Units for Compact and Efficient Deep Neural Networks

Convolutional neural networks excel in a number of computer vision tasks. One of their most crucial architectural elements is the effective receptive field size, that has to be manually set to accommodate a specific task. Standard solutions involve large kernels, down/up-sampling and dilated convolutions. These require testing a variety of dilation and down/up-sampling factors and result in non-compact representations and excessive number of parameters. We address this issue by proposing a new convolution filter composed of displaced aggregation units (DAU). DAUs learn spatial displacements and adapt the receptive field sizes of individual convolution filters to a given problem, thus eliminating the need for hand-crafted modifications. DAUs provide a seamless substitution of convolutional filters in existing state-of-the-art architectures, which we demonstrate on AlexNet, ResNet50, ResNet101, DeepLab and SRN-DeblurNet. The benefits of this design are demonstrated on a variety of computer vision tasks and datasets, such as image classification (ILSVRC 2012), semantic segmentation (PASCAL VOC 2011, Cityscape) and blind image de-blurring (GOPRO). Results show that DAUs efficiently allocate parameters resulting in up to four times more compact networks at similar or better performance.

Electrical Coupling of Heterotypic Ganglion Cells in the Mammalian Retina.

Electrical coupling has been reported to occur only between homotypic retinal ganglion cells, in line with the concept of parallel processing in the early visual system. Here, however, we show reciprocal correlated firing between heterotypic ganglion cells in multielectrode array recordings during light stimulation in retinas of adult guinea pigs of either sex. Heterotypic coupling was further confirmed via tracer spread after intracellular injections of single cells with neurobiotin. Both electrically coupled cell types were sustained ON center ganglion cells but showed distinct light response properties and receptive field sizes. We identified one of the involved cell types as sustained ON α-ganglion cells. The presence of electrical coupling between heterotypic ganglion cells introduces a network motif in which the signals of distinct ganglion cell types are partially mixed at the output stage of the retina.SIGNIFICANCE STATEMENT The visual information is split into parallel pathways, before it is sent to the brain via the output neurons of the retina, the ganglion cells. Ganglion cells can form electrical synapses between dendrites of neighboring cells in support of lateral information exchange. To date, ganglion-to-ganglion cell coupling is thought to occur only between cells of the same type. Here, however, we show that electrical coupling between different types of ganglion cells exists in the mammalian retina. We provide functional and anatomical evidence that two different types of ganglion cells share information via electrical coupling. This new network motif extends the impact of the heavily studied coding benefits of homotypic coupling to heterotypic coupling across parallel neuronal pathways.

Hartley Spectral Pooling for Deep Learning

In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly so-called pooling. Maximation and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convolution are popular pooling methods. Since the pooling is a lossy procedure, a motivation of our work is to design a new pooling approach for less lossy in the dimensionality reduction. Inspired by the Fourier spectral pooling(FSP) proposed by Rippel et. al. [1], we present the Hartley transform based spectral pooling method in CNNs. Compared with FSP, the proposed spectral pooling avoids the use of complex arithmetic for frequency representation and reduces the computation. Spectral pooling preserves more structure features for network's discriminability than max and average pooling. We empirically show that Hartley spectral pooling gives rise to the convergence of training CNNs on MNIST and CIFAR-10 datasets.