Small Receptive Fields Research Articles

SSD-Resnet50: Research on Pedestrian Detection Technology

In the era of artificial intelligence, with the rapid development of computer hardware, the defects of neural network algorithm have been further improved, the technology of artificial intelligence is progressing, the algorithm and technology of target detection have been further improved, and pedestrian detection technology is an important research direction of target detection technology. The main research content of this paper consists of three parts: the improvement of the original SSD algorithm, and the fundamental composition and characteristics of the neural network; Changing VGG16, the basis of the original SSD algorithm, to Resnet50 can improve the detection speed and accuracy. A feature fusion module is introduced to fuse the feature map with large receptive field and small receptive field, so as to improve the overall expression ability of the feature map to the image.

Multi-scale-based joint super-resolution and inverse tone-mapping with data synthesis for UHD HDR video

The performance of current joint super-resolution (SR) and inverse tone-mapping (ITM) frameworks is limited since they only account for information in small local receptive fields. Moreover, the SDR images in the existing dataset for joint SR-ITM are oversaturated compared to the HDR ground truths. The models trained on this dataset produce HDR images with color shift. This paper proposes a multi-scale-based joint SR-ITM model to reconstruct HR HDR videos. Image features are downsampled to different resolutions to increase the local receptive fields. Thus the proposed model can react to more complex patterns of input images. And we design a novel multi-path residual dense block (MRDB) as the model’s fundamental component to extract features. The proposed MRDBs can improve performance by combining dense feature learning and novel multi-path residual learning. Experiments show that the proposed model outperforms the state-of-the-art methods in terms of quantitative and qualitative results. Furthermore, we propose a data synthesis pipeline to generate perceptually identical SDR images from their HDR counterparts. These SDR-HDR pairs are used to create a new dataset. Experiments demonstrate that models trained with our new dataset prevent color shift and preserve creative intent.

LKC-Net: large kernel convolution object detection network

Deep learning-based object detection methods have achieved great performance improvement. However, since small kernel convolution has been widely used, the semantic feature is difficult to obtain due to the small receptive fields, and the key information cannot be highlighted, resulting in a series of problems such as wrong detection, missing detection, and repeated detection. To overcome these problems, we propose a large kernel convolution object detection network based on feature capture enhancement and vast receptive field attention, called LKC-Net. Firstly, a feature capture enhancement block based on large kernel convolution is proposed to improve the semantic feature capturing ability, and depth convolution is used to reduce the number of parameters. Then, the vast receptive filed attention mechanism is constructed to enhance channel direction information extraction ability, and it is more compatible with the proposed backbone than other existing attention mechanisms. Finally, the loss function is improved by introducing the SIoU, which can overcome the angle mismatch problem between the ground truth and prediction box. Experiments are conducted on Pascal VOC and MS COCO datasets for demonstrating the performance of LKC-Net.

URTSegNet: A real-time segmentation network of unstructured road at night based on thermal infrared images for autonomous robot system

When it comes to poor lighting conditions such as night, backlight, haze, etc., the detection method of unstructured roads based on visible light cameras often fails. Therefore, this paper aims to improve the detection performance of unstructured roads based on thermal infrared images for autonomous robot systems. For this purpose, we propose a real-time network for unstructured road segmentation at night based on thermal infrared images (URTSegNet) in this paper. Like BiSeNet V2, we adopt a two-branch structure, including Detail Branch and Semantic Branch, to extract spatial detail and contextual features. The Detail Branch maintains a small receptive field and high capacity and adjusts the computational cost by employing depth-wise separable convolutions. In the Semantic Branch, we modify Stem Block to obtain more information because of the characteristics of infrared images. Dilated convolution expands the receptive field while keeping the number of parameters unchanged. Finally, we design a more concise and efficient Aggregation Block using a gate mechanism. An infrared unstructured road dataset, including total darkness at night, low light, backlight, haze, and other scenes, is collected to get the real-world dataset and effectively evaluate performance. We got 97.5% Intersection over Union (IoU) and reached an inference speed of 129 FPS on a single NVIDIA GTX 1660S card on our dataset. In addition, extensive evaluation experiments are conducted on two public datasets to verify the generalization ability of the model and the role of each module. This research addresses the problem of road detection for unmanned platforms in unstructured environments at night or under poor lighting conditions. The algorithm has high accuracy and a fast inference speed so that it can be implemented on various embedded devices.

A Patch Information Supplement Transformer for Person Re-Identification

Extracting fine-grained features from person images has proven crucial in person re-identification (re-ID). Although the research of convolutional neural networks (CNN) has been very successful in person re-ID, due to the small receptive field and downsampling operation, the existing CNNs cannot solve the problem of information loss. The multi-head attention modules in transformer can solve the above problems well. However, since dicing operations destroy the spatial correlation between patches, transformer still loses some local features. in this paper, we propose the scheme of the patch information supplement transformer (PIT) to extract fine-grained features in the dicing stage. Patch pyramid network (PPN) is introduced to solve the problem of local information loss. This is accomplished by dividing the image into different scales through the dicing operation and adding them together from top to bottom according to the pyramid structure. In addition, we insert a learnable identity information-embedding module (IDE) to reduce the feature bias of clothing and camera perspective. Experiments verify the superiority and effectiveness of PIT compared to state-of-the-art methods.

Disentangled convolution for optimizing receptive field

Convolutional neural network (CNN) is one of the primary techniques for high-performance image recognition. The convolution operation with small-sized filters is a key ingredient of CNN and the receptive field of the whole CNN is enlarged by stacking lots of convolution layers. The convolution layer, however, is problematic in terms of the receptive field. It provides fixed small receptive field due to the fixed filter size in convolution, which requires us to manually control it in advance. Besides, the larger-sized convolution filters significantly increase computation cost. Thus, in this study, we propose a method to adaptively tune the receptive field of the convolution operation in an end-to-end manner as well as to enlarge the receptive field in a low computation cost. Based on the biological studies and scale-space theory, we can disentangle convolution operation into Gaussian envelope filtering for smoothing and derivative-related filtering, both of which are heterogeneously parameterized. Those two types of filters are jointly optimized in the end-to-end CNN training and the receptive field of the convolution is adequately optimized via learning the Gaussian envelope with a low extra computation cost. The experimental results on image classification tasks demonstrate that the proposed method effectively enlarges receptive field to improve performance.

A new convolutional dual-channel Transformer network with time window concatenation for remaining useful life prediction of rolling bearings

Deep learning has achieved numerous breakthroughs in bearing predicting remaining useful life (RUL). However, the current mainstream deep learning framework inevitably has flaws, including the disadvantage of the small receptive field, the difficulty of learning long-term dependencies and the singularity of feature extraction domains, etc. Given the challenges mentioned above, we propose a new convolutional dual-channel Transformer network (CDCT) for remaining useful life prediction of rolling bearings. In the CDCT, the causal convolution operation is applied to extract local features from the time and frequency domains and add positional encoding to the input signal, while the transformer block is utilized for extracting bidirectional features and fusing them. The CDCT not only has a global receptive field but also can learn long-term dependencies regardless of sequence length. Besides, the time window concatenation is adopted to ameliorate the problem of large amounts of trainable parameters of the Transformer-based models. In the experiments, we conduct a detailed analysis of each crucial element and hyperparameter of the CDCT and compare it to multiple basic and advanced methods. The experimental results highlight the superiority of the CDCT in bearing RUL prediction and demonstrate the effectiveness of crucial elements in the CDCT.

A cortical zoom-in operation underlies covert shifts of visual spatial attention

Shifting the focus of attention without moving the eyes poses challenges for signal coding in visual cortex in terms of spatial resolution, signal routing, and cross-talk. Little is known how these problems are solved during focus shifts. Here, we analyze the spatiotemporal dynamic of neuromagnetic activity in human visual cortex as a function of the size and number of focus shifts in visual search. We find that large shifts elicit activity modulations progressing from highest (IT) through mid-level (V4) to lowest hierarchical levels (V1). Smaller shifts cause those modulations to start at lower levels in the hierarchy. Successive shifts involve repeated backward progressions through the hierarchy. We conclude that covert focus shifts arise from a cortical coarse-to-fine process progressing from retinotopic areas with larger toward areas with smaller receptive fields. This process localizes the target and increases the spatial resolution of selection, which resolves the above issues of cortical coding.

YOLO-FR: A YOLOv5 Infrared Small Target Detection Algorithm Based on Feature Reassembly Sampling Method.

The loss of infrared dim-small target features in the network sampling process is a major factor affecting its detection accuracy. In order to reduce this loss, this paper proposes YOLO-FR, a YOLOv5 infrared dim-small target detection model, based on feature reassembly sampling, which refers to scaling the feature map size without increasing or decreasing the current amount of feature information. In this algorithm, an STD Block is designed to reduce the loss of features during down-sampling by saving spatial information to the channel dimension, and the CARAFE operator, which increases the feature map size without changing the feature mapping mean, is adopted to ensure that features are not distorted by relational scaling. In addition, in order to make full use of the detailed features extracted by the backbone network, the neck network is improved in this study so that the feature extracted after one down-sampling of the backbone network is fused with the top-level semantic information by the neck network to obtain the target detection head with a small receptive field. The experimental results show that the YOLO-FR model proposed in this paper achieved 97.4% on mAP50, which is a 7.4% improvement compared to the original network, and it also outperformed J-MSF and YOLO-SASE.

Dead pixel test using effective receptive field

Deep neural networks have been used in various fields, but their internal behavior in how they understand images is not well known. In this study, we discuss two counterintuitive properties of convolutional neural networks (CNNs). First, we evaluated the size of the receptive field of CNNs with their classification accuracy. Previous studies have attempted to increase the size of the receptive field for performance gain. However, we observed that some CNNs with a smaller receptive field can achieve higher classification accuracy. In this regard, we claim that a larger receptive field does not guarantee improved classification accuracy. Second, using the effective receptive field, we examined the contribution of each pixel to the output of CNN. Intuitively, each pixel is expected to equally contribute to the final output, but we found that there exist pixels in a partially dead state with little contribution to the output. We reveal that the reason for dead pixels lies in even stride operations with odd-sized kernels in CNN and propose a kernel padding method to remove the dead pixels. We demonstrated the vulnerability of CNNs with dead pixels when we detect a noise or small box that is on dead pixels. Our findings on dead pixels should be understood and considered in practical applications of CNN.

Dual parallel net: A novel deep learning model for rectal tumor segmentation via CNN and transformer with Gaussian Mixture prior

Segmentation of rectal cancerous regions from Magnetic Resonance (MR) images can help doctor define the extent of the rectal cancer and judge the severity of rectal cancer, so rectal tumor segmentation is crucial to improve the accuracy of rectal cancer diagnosis. However, accurate segmentation of rectal cancerous regions remains a challenging task due to the shape of rectal tumor has significant variations and the tumor and surrounding tissue are indistinguishable. In addition, in the early research on rectal tumor segmentation, most deep learning methods were based on convolutional neural networks (CNNs), and traditional CNN have small receptive field, which can only capture local information while ignoring the global information of the image. Nevertheless, the global information plays a crucial role in rectal tumor segmentation, so traditional CNN-based methods usually cannot achieve satisfactory segmentation results. In this paper, we propose an encoder-decoder network named Dual Parallel Net (DuPNet), which fuses transformer and classical CNN for capturing both global and local information. Meanwhile, as for capture features at different scales as well as to avoid accuracy loss and parameters reduction, we design a feature adaptive block (FAB) in skip connection between encoder and decoder. Furthermore, in order to utilize the apriori information of rectal tumor shape effectively, we design a Gaussian Mixture prior and embed it in self-attention mechanism of the transformer, leading to robust feature representation and accurate segmentation results. We have performed extensive ablation experiments to verify the effectiveness of our proposed dual parallel encoder, FAB and Gaussian Mixture prior on the dataset from the Shanxi Cancer Hospital. In the experimental comparison with the state-of-the-art methods, our method achieved a Mean Intersection over Union (MIoU) of 89.34% on the test set. In addition to that, we evaluated the generalizability of our method on the dataset from Xinhua Hospital, the promising results verify the superiority of our method.

Monocular Depth Estimation Network Based on Swin Transformer

Estimating depth from a single image is challenging because a single 2D image may correspond to many different 3D scenes with the same depth. While most deep learning based depth prediction methods extract depth features using small convolutional kernels with small receptive fields, which results in deformed depth edges and inaccurate depth values of distant objects in the depth estimation results. Aiming at this problem, we propose a depth estimation network based on Swin Transformer and the encoder-decoder structure. We construct the encoder using the Swin Transformer network, which can encode long-range spatial dependency and extract features on various scales and across different channels. The decoder of the proposed network is in charge of fusing the features from the encoder by the operations of interpolation, concatenation, and convolution. Experiments on KITTI and NYUv2 datasets show that our proposed network can get more accurate depth edges and depth values than the state-of-the-art methods.

Target area distillation and section attention segmentation network for accurate 3D medical image segmentation.

3D medical image segmentation has an essential role in medical image analysis, while attention mechanism has improved the performance by a large margin. However, existing methods obtained the attention coefficient in a small receptive field, resulting in possible performance limitations. Radiologists usually scan all the slices first to have an overall idea of the target, and then analyze regions of interest in multiple 2D views in clinic practice. We simulate radiologists' recognition process and propose to exploit the 3D context information in a deeper manner for accurate 3D medical images segmentation. Due to the similarity of human body structure, medical images of different populations have highly similar shape and location information, so we use target region distillation to extract the common segmented region information. Particularly, we proposed two optimizations including Target Area Distillation and Section Attention. Target Area Distillation adds positions information to the original input to let the network has an initial attention of the target, while section attention performs attention extraction in three 2D sections thus with large range of receptive field. We compare our method against several popular networks in two public datasets including ImageCHD and COVID-19. Experimental results show that our proposed method improves the segmentation Dice score by 2-4% over the state-of-the-art methods. Our code has been released to the public (Anonymous link).

Unsafe Mining Behavior Identification Method Based on an Improved ST-GCN

Aiming to solve the problems of large environmental interference and complex types of personnel behavior that are difficult to identify in the current identification of unsafe behavior in mining areas, an improved spatial temporal graph convolutional network (ST-GCN) for miners’ unsafe behavior identification network in a transportation roadway (NP-AGCN) was proposed. First, the skeleton spatial-temporal map constructed using multi-frame human key points was used for behavior recognition to reduce the interference caused by the complex environment of the coal mine. Second, aiming to solve the problem that the original graph structure cannot learn the association relationship between the non-naturally connected nodes, which leads to the low recognition rate of climbing belts, fighting and other behaviors, the graph structure was reconstructed and the original partitioning strategy was changed to improve the recognition ability of the model for multi-joint interaction behaviors. Finally, in order to alleviate the problem that the graph convolution network has difficulty learning global information due to the small receptive field, multiple self-attention mechanisms were introduced into the graph convolution to improve the recognition ability of the model for unsafe behaviors. In order to verify the detection ability of the model regarding identifying unsafe behaviors of personnel in a coal mine belt area, our model was tested on the public datasets NTU-RGB + D and the self-built datasets of unsafe behaviors in a coal mine belt area. The recognition accuracies of the proposed model in the above datasets were 94.7% and 94.1%, respectively, which were 6.4% and 7.4% higher than the original model, which verified that the proposed model had excellent recognition accuracies.

Self-Supervised Image Denoising for Real-World Images With Context-Aware Transformer

In recent years, the development of deep learning has been pushing image denoising to a new level. Among them, self-supervised denoising is increasingly popular because it does not require any prior knowledge. Most of the existing self-supervised methods are based on convolutional neural networks (CNN), which are restricted by the locality of the receptive field and would cause color shifts or textures loss. In this paper, we propose a novel Denoise Transformer for real-world image denoising, which is mainly constructed with Context-aware Denoise Transformer (CADT) units and Secondary Noise Extractor (SNE) block. CADT is designed as a dual-branch structure, where the global branch uses a window-based Transformer encoder to extract the global information, while the local branch focuses on the extraction of local features with small receptive field. By incorporating CADT as basic components, we build a hierarchical network to directly learn the noise distribution information through residual learning and obtain the first stage denoised output. Then, we design SNE in low computation for secondary global noise extraction. Finally the blind spots are collected from the Denoise Transformer output and reconstructed, forming the final denoised image. Extensive experiments on the real-world SIDD benchmark achieve 50.62/0.990 for PSNR/SSIM, which is competitive with the current state-of-the-art method and only 0.17/0.001 lower. Visual comparisons on public sRGB, Raw-RGB and greyscale datasets prove that our proposed Denoise Transformer has a competitive performance, especially on blurred textures and low-light images, without using additional knowledge, e.g., noise level or noise type, regarding the underlying unknown noise.

F2SRGAN: A Lightweight Approach Boosting Perceptual Quality in Single Image Super-Resolution via a Revised Fast Fourier Convolution

With the successful development of deep learning, single image super-resolution (SISR) has advanced significantly in recent years. However, in practice, excessive convolutions limit super-resolution applications on platforms with limited resources like mobile devices or embedded systems. Besides, existing lightweight models have a problem with small receptive fields and only consider local features for the reconstruction. Previous models try to stack more convolutions layers to address this problem, but this prolongs the execution time due to increasing the number of parameters. Therefore, this paper proposes a novel approach named F2SRGAN using a revised Fast Fourier Convolution to enlarge the receptive field, enabling this model to learn global features better, neither introducing too many parameters nor prolonging the model inference time. The experimental results show that our proposed F2SRGAN significantly improves perceptual image quality among the lightweight SISR methods while maintaining an acceptable inference time.

Direction selectivity of the retinotectal system of fish: Findings based on microelectrode extracellular recordings of the tectum opticum

Vision in fish plays an important role in different forms of visually guided behavior. The visual system of fish is available for research by different methods; it is a convenient experimental model for studying and understanding the mechanisms of vision in general. Responses of retinal direction-selective (DS) ganglion cells (GCs) are recorded extracellularly from their axon terminals in the superficial layers of the tectum opticum (TO). They can be divided into three distinct groups according to the preferred directions of stimulus movement: caudorostral, dorsoventral and ventrodorsal. Each of these groups comprises both ON and OFF units in equal proportions. Relatively small receptive fields (3-8?) and fine spatial resolution characterize retinal DS units as local motion detectors. Conversely, the responses of direction-selective tectal neurons (DS TNs) are recorded at two different tectal levels, deeper than the zone of retinal DS afferents. They are characterized by large receptive fields (up to 60?) and are indifferent to any sign of contrast, i.e., they can be considered as ON-OFF-type units. Four types of ON-OFF DS TNs preferring different directions of motion have been recorded. The preferred directions of three types of DS TNs match the preferred directions of three types of DS GCs. Matching the three preferred directions of ON and OFF DS GCs and ON-OFF DS TNs has allowed us to hypothesize that the GCs with caudorostral, ventrodorsal and dorsoventral preferences are input neurons for the corresponding types of DS TNs. On the other hand, the rostrocaudal preference in the fourth type of DS TNs, recorded exclusively in the deep tectal zone, is an emergent property of the TO. In this review, our findings are compared with the results of other authors examining direction selectivity in the fish retinotectal system.

DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition

As a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">de facto</i> solution, the vanilla Vision Transformers (ViTs) are encouraged to model long-range dependencies between arbitrary image patches while the global attended receptive field leads to quadratic computational cost. Another branch of Vision Transformers exploits local attention inspired by CNNs, which only models the interactions between patches in small neighborhoods. Although such a solution reduces the computational cost, it naturally suffers from small attended receptive fields, which may limit the performance. In this work, we explore effective Vision Transformers to pursue a preferable trade-off between the computational complexity and size of the attended receptive field. By analyzing the patch interaction of global attention in ViTs, we observe two key properties in the shallow layers, namely locality and sparsity, indicating the redundancy of global dependency modeling in shallow layers of ViTs. Accordingly, we propose Multi-Scale Dilated Attention (MSDA) to model <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sparse</i> patch interaction within the sliding window. With a pyramid architecture, we construct a Multi-Scale Dilated Transformer (DilateFormer) by stacking MSDA blocks at low-level stages and global multi-head self-attention blocks at high-level stages. Our experiment results show that our DilateFormer achieves state-of-the-art performance on various vision tasks. On ImageNet-1 K classification task, DilateFormer achieves comparable performance with 70% fewer FLOPs compared with existing state-of-the-art models. Our DilateFormer-Base achieves 85.6% top-1 accuracy on ImageNet-1 K classification task, 53.5% box mAP/46.1% mask mAP on COCO object detection/instance segmentation task and 51.1% MS mIoU on ADE20 K semantic segmentation task.

Improving Generalized Zero-Shot Learning SSVEP Classification Performance From Data-Efficient Perspective.

Generalized zero-shot learning (GZSL) has significantly reduced the training requirements for steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs). Traditional methods require complete class data sets for training, but GZSL allows for only partial class data sets, dividing them into 'seen' (those with training data) and 'unseen' classes (those without training data). However, inefficient utilization of SSVEP data limits the accuracy and information transfer rate (ITR) of existing GZSL methods. To this end, we proposed a framework for more effective utilization of SSVEP data at three systematically combined levels: data acquisition, feature extraction, and decision-making. First, prevalent SSVEP-based BCIs overlook the inter-subject variance in visual latency and employ fixed sampling starting time (SST). We introduced a dynamic sampling starting time (DSST) strategy at the data acquisition level. This strategy uses the classification results on the validation set to find the optimal sampling starting time (OSST) for each subject. In addition, we developed a Transformer structure to capture the global information of input data for compensating the small receptive field of existing networks. The global receptive fields of the Transformer can adequately process the information from longer input sequences. For the decision-making level, we designed a classifier selection strategy that can automatically select the optimal classifier for the seen and unseen classes, respectively. We also proposed a training procedure to make the above solutions in conjunction with each other. Our method was validated on three public datasets and outperformed the state-of-the-art (SOTA) methods. Crucially, we also outperformed the representative methods that require training data for all classes.

The structure and diversity of retinal ganglion cells in the masked greenling Hexagrammos octogrammus Pallas, 1814 (Pisces: Scorpaeniformes: Hexagrammidae).

The authors studied the structure and diversity of retinal ganglion cells (GC) in the masked greenling Hexagrammos octogrammus. In vivo labelling with horseradish peroxidase revealed GCs of various structures in retinal wholemounts. A total of 154 cells were camera lucida drawn, and their digital models were generated. Each cell was characterized by 17 structural and topological parameters. Using nine clustering algorithms, a variety of clusterings were obtained. The optimum clustering was found using silhouette analysis. It was based on a set of three variables associated with dendritic field size and dendrite stratification depth in the retina. A total of nine cell types were discovered. A number of non-parametric tests showed significant pair-wise between-cluster differences in at least four parameters with medium and large effect sizes. Three large-field types differed mainly in dendritic field size, total dendrite length, level of dendrite stratification in the retina and position of somata. Six medium- to small-field types differed mainly in the structural complexity of dendritic arbors and level of dendrite arborization. Cells similar and obviously homologous to types 1-4 were identified in many fish species, including teleosts. Potential homologues of type 5 cells were identified in fewer teleost species. Cells similar to types 6-9 in relative dendritic field size and dendrite arborization pattern were also described in several teleostean species. Nonetheless, their homology is more questionable as their stratification patterns do not match so well as they do in large types. Potential functional matches of the GC types were identified in a number of teleostean species. Type 1 and 2 cells probably match spontaneously active units with the large receptive field centre, so-called dimming and lightening detectors; type 4 may be a counterpart of changing contrast detectors with medium receptive field centre size preferring fast-moving stimuli. Type 3 (biplexiform) cells have no obvious functional matches. Probable functional matches of types 6, 8 and 9 belong to ON-centre elements with small receptive fields such as ON-type direction-selective cells, ON-type spot detectors or ON-type spontaneously active units. Type 5 and 7 cells may match ON-OFF type units, in particular, changing contrast detectors or orientation-selective units. Potential functional matches of GC types presently described are involved in a wide spectrum of visual reactions related to adaptation to gradual change in illumination, predator escape, prey detection and capture, habitat selection and social behaviour.