Small Receptive Fields Research Articles

Bridging Global Context Interactions for High-Fidelity Pluralistic Image Completion.

We introduce PICFormer, a novel framework for Pluralistic Image Completion using a transFormer based architecture, that achieves both high quality and diversity at a much faster inference speed. Our key contribution is to introduce a code-shared codebook learning using a restrictive CNN on small and non-overlapping receptive fields (RFs) for the local visible token representation. This results in a compact yet expressive discrete representation, facilitating efficient modeling of global visible context relations by the transformer. Unlike the prevailing autoregressive approaches, we proposed to sample all tokens simultaneously, leading to more than 100× faster inference speed. To enhance appearance consistency between visible and generated regions, we further propose a novel attention-aware layer (AAL), designed to better exploit distantly related high-frequency features. Through extensive experiments, we demonstrate that the PICFormer efficiently learns semantically-rich discrete codes, resulting in significantly improved image quality. Moreover, our diverse image completion framework surpasses State-of-the-Art methods on multiple image completion datasets.

Hierarchical-Concatenate Fusion TDNN for sound event classification.

Semantic feature combination/parsing issue is one of the key problems in sound event classification for acoustic scene analysis, environmental sound monitoring, and urban soundscape analysis. The input audio signal in the acoustic scene classification is composed of multiple acoustic events, which usually leads to low recognition rate in complex environments. To address this issue, this paper proposes the Hierarchical-Concatenate Fusion(HCF)-TDNN model by adding HCF Module to ECAPA-TDNN model for sound event classification. In the HCF module, firstly, the audio signal is converted into two-dimensional time-frequency features for segmentation. Then, the segmented features are convolved one by one for improving the small receptive field in perceiving details. Finally, after the convolution is completed, the two adjacent parts are combined before proceeding with the next convolution for enlarging the receptive field in capturing large targets. Therefore, the improved model further enhances the scalability by emphasizing channel attention and efficient propagation and aggregation of feature information. The proposed model is trained and validated on the Urbansound8K dataset. The experimental results show that the proposed model can achieve the best classification accuracy of 95.83%, which is an approximate improvement of 5% (relatively) over the ECAPA-TDNN model.

Prior-free 3D human pose estimation in a video using limb-vectors

Estimating accurate 3D human poses from a monocular video is fundamental to various computer vision tasks. Existing methods exploit 2D-to-3D pose lifting, multiview images, and depth sensors to model spatio-temporal dependencies. However, depth ambiguities, occlusions, and larger temporal receptive fields pose challenges to these approaches. To address this, we propose a novel prior-free DCNN-based 3D human pose estimation method for monocular image sequences using limb vectors. Our method comprises two subnetworks: a limb direction estimator and a limb length estimator. The limb direction estimator utilizes a fully convolutional network to model limb direction vectors across a temporal window. We show that network complexity can be significantly reduced by utilizing dilated convolutional operations and a relatively smaller receptive field while maintaining estimation accuracy. Moreover, the limb length estimator captures stable limb length estimations from a reliable frame set. Our model has shown superior performance compared to existing methods on the Human3.6M and MPI-INF-3DHP datasets.

Power Quality Transient Disturbance Diagnosis Based on Dynamic Large Convolution Kernel and Multi-Level Feature Fusion Network

Power quality is an important metric for the normal operation of a power system, and the accurate identification of transient signals is of great significance for the improvement of power quality. The diverse types of power system transient signals and strong characteristic coupling brings new challenges to the analysis and identification of power system transient signals. In order to enhance the identification accuracy of transient signals, one method of power system transient signal identification is proposed based on a dynamic large convolution kernel and multilevel feature fusion network. First, the more fine-grained and more informative features of the transient signals are extracted by the dynamic large convolution kernel feature extraction module. Then, the multi-scale local features are adaptively fused by the multilevel feature fusion module. Finally, the fused features are reduced in dimension by the fully connected layer in the classification module and fed into the SoftMax layer for transient signal type detection. The proposed method can effectively improve the small receptive field problem of convolutional neural networks and the lack of ability of Transformer network in extracting local context information. Compared with five other power quality transient disturbance identification models, the experimental results show that the proposed method has better diagnostic accuracy and anti-noise capability.

Convolutional neural network models applied to neuronal responses in macaque V1 reveal limited nonlinear processing.

Computational models of the primary visual cortex (V1) have suggested that V1 neurons behave like Gabor filters followed by simple nonlinearities. However, recent work employing convolutional neural network (CNN) models has suggested that V1 relies on far more nonlinear computations than previously thought. Specifically, unit responses in an intermediate layer of VGG-19 were found to best predict macaque V1 responses to thousands of natural and synthetic images. Here, we evaluated the hypothesis that the poor performance of lower layer units in VGG-19 might be attributable to their small receptive field size rather than to their lack of complexity per se. We compared VGG-19 with AlexNet, which has much larger receptive fields in its lower layers. Whereas the best-performing layer of VGG-19 occurred after seven nonlinear steps, the first convolutional layer of AlexNet best predicted V1 responses. Although the predictive accuracy of VGG-19 was somewhat better than that of standard AlexNet, we found that a modified version of AlexNet could match the performance of VGG-19 after only a few nonlinear computations. Control analyses revealed that decreasing the size of the input images caused the best-performing layer of VGG-19 to shift to a lower layer, consistent with the hypothesis that the relationship between image size and receptive field size can strongly affect model performance. We conducted additional analyses using a Gabor pyramid model to test for nonlinear contributions of normalization and contrast saturation. Overall, our findings suggest that the feedforward responses of V1 neurons can be well explained by assuming only a few nonlinear processing stages.

Towards accurate binocular vision of satellites: A cascaded multi-scale pyramid network for stereo matching on satellite imagery

Stereo matching of high-resolution satellite stereo images is a fundamental and challenging task in photogrammetry, geoscience, and remote sensing. Some deep learning-based methods have shown great potential in solving this task recently. However, these methods have not attracted much attention as their accuracy is still unsatisfactory, especially in regions with intractable factors (e.g., occlusions, repetitive patterns, low texture, non-texture, and disparity discontinuities). To tackle these challenging factors and improve disparity accuracy, we propose an end-to-end stereo matching network named Cascaded Multi-Scale Pyramid Network (CMSP-Net). Firstly, to fully utilize the multi-level spatial context information and mitigate the ambiguous matching caused by repetitive patterns, low texture, non-texture, and occlusions, a cost volume pyramid is constructed with multi-scale image features. Then, each cost volume in the pyramid is aggregated with an hourglass structure to exploit the multi-scale context in a single cost volume. Secondly, considering that high-resolution cost volumes contain more detailed information but have small receptive fields, while low-resolution ones have large receptive fields but lack details, we design an Attention-Guided Cost Fusion (AGCF) strategy to fuse pyramid cost volumes, which effectively integrates the advantages of multi-scale cost volumes while mitigating the backfiring caused by semantic inconsistency across different scales. Thirdly, a disparity refinement module is designed to improve overall disparity accuracy further, which incorporates the reconstruction error and edge clues of input satellite stereo images. Extensive experimental results on multiple satellite datasets demonstrate that the proposed method can effectively reduce matching ambiguities and recover sharp disparity discontinuities, achieving clear superiority over existing state-of-the-art methods.

DL-YOLOX: Real-time object detection via adjustable dilated enhancement for autonomous driving scene

In the domain of autonomous driving, object detection presents several complex challenges, particularly concerning the accurate identification of small and salient objects. This paper introduces DL-YOLOX (Dilated Enhancement YOLOX), which flexibly uses dilated convolution to enhance features to achieve the purpose of improving small objects and silent objects. As we all know, a large receptive field covers a larger area and has greater contextual information, which is more advantageous for detecting large targets. A small receptive field helps capture local details and has better detection capabilities for detecting small targets. To bolster the representation of objects across various scales, we propose the integration of Dilated Adaptive Feature Fusion (DAFF) which has the ability to adaptively fuse features with different receptive fields. This innovative fusion mechanism allows for a more comprehensive understanding of objects, enabling improved detection accuracy even for objects of varying sizes. In addition, we tackle the issue of small object loss during feature propagation by introducing Stack Dilated Module (SDM), a powerful module that mitigates this phenomenon and contributes to better detection performance. Moreover, we endeavor to enhance small object detection further by replacing the conventional Intersection over Union (IoU) metric with Normalized Gaussian Wasserstein Distance (NWD), a novel distance metric that proves to be more effective in accurately gauging small object detection, thus elevating the precision of our algorithm. To thoroughly evaluate the robustness and generalization capabilities of our proposed method, we conduct extensive experiments on two benchmark datasets, namely MS COCO 2017 and BDD100K. The results from our evaluation not only affirm the significant improvements achieved in multi-scale object detection but also highlight the real-time capability of our approach. The impressive performance across these datasets demonstrates the promising potential of DL-YOLOX in revolutionizing object detection techniques in the context of autonomous driving.

TTMRI: Multislice texture transformer network for undersampled MRI reconstruction

AbstractMagnetic resonance imaging (MRI) is a non‐interposition imaging technique that provides rich anatomical and physiological information. Yet it is limited by the long imaging time. Recently, deep neural networks have shown potential to significantly accelerate MRI. However, most of these approaches ignore the correlation between adjacent slices in MRI image sequences. In addition, the existing deep learning‐based methods for MRI are mainly based on convolutional neural networks (CNNs). They fail to capture long‐distance dependencies due to the small receptive field. Inspired by the feature similarity in adjacent slices and impressive performance of Transformer for exploiting the long‐distance dependencies, a novel multislice texture transformer network is presented for undersampled MRI reconstruction (TTMRI). Specifically, the proposed TTMRI is consisted of four modules, namely the texture extraction, correlation calculation, texture transfer and texture synthesis. It takes three adjacent slices as inputs, in which the middle one is the target image to be reconstructed, and the other two are auxiliary images. The multiscale features are extracted by the texture extraction module and their inter‐dependencies are calculated by the correlation calculation module, respectively. Then the relevant features are transferred by the texture transfer module and fused by the texture synthesis module. By considering inter‐slice correlations and leveraging the Transformer architecture, the joint feature learning across target and adjacent slices are encouraged. Moreover, TTMRI can be stacked with multiple layers to recover more texture information at different levels. Extensive experiments demonstrate that the proposed TTMRI outperforms other state‐of‐the‐art methods in both quantitative and qualitative evaluationsions.

High-Order Temporal Convolutional Network for Improving Classification Performance of SSVEP-EEG

Background and objectiveSteady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) aim to detect target frequencies corresponding to specific commands in electroencephalographic (EEG) signals by classification algorithms to achieve the desired control. However, SSVEP signals suffer from low signal-to-noise ratio and large differences in brain activity. Moreover, the existing CNN models have small receptive fields, which make it difficult to receive large range of feature information and limit the effectiveness of classification algorithms. MethodsTo this end, we proposed a high-order temporal convolutional neural network (HOT-CNN) model for enhancing the performance of SSVEP target recognition. Specifically, the SSVEP-EEG signals was divided into equal-length time segments and a time-slice attention module was designed to capture the correlation between time slices. The module improves the local characterization of signals and reduces biological noise interference by automatically assigning high weights to locally relevant temporal sampling cues and lower weights to other temporal cues. Moreover, for global features, a temporal convolutional network module was designed to increases the receptive field of the network and to extract more comprehensive time domain features by using dilated causal convolution. Finally, the fusion and analysis of local and global features are achieved by designing a feature fusion and classification module to accomplish accurate classification of SSVEP signals. ResultsOur method was evaluated on large publicly available datasets containing 35 subjects and 40 categories. Experimental results indicated that HOT-CNN achieved encouraging performance compared with other advanced methods: the highest information transfer rate of 241.01bits/min was obtained using 0.5s stimuli, and the highest average accuracy of 96.39% was obtained using 1.0s stimuli. ConclusionsThe method effectively reinforced the global and local time-domain information and improved the classification performance of SSVEP, which has wide application prospects.

Hyper-S3NN: Spatial–spectral spiking neural network for hyperspectral image classification

Recently, convolutional neural networks (CNNs) have achieved an appealing performance in hyperspectral image (HSI) classification. Although CNNs excel at capturing local details, they fall short in modeling long-range dependencies due to small receptive fields. Spiking neural networks (SNNs) simulate the encoding and processing of information by the human brain, increasing the receptive field when extracting image features, which is expected to solve the previous problems. Therefore, we present further study on exploiting the power of SNNs for axial global information interaction. The proposed approach, termed Hyper-S3NN, combines CNNs and SNNs to leverage the fine localization information and global contexts. Specifically, based on the leaky integrate-and-fire (LIF) neurons in SNNs, three types of LIF branches are proposed to obtain the global interactive features along the horizontal, vertical, and channel dimensions. By combining convolutions for local information extraction and feature fusion, our Spatial–Spectral LIF (SSLIF) module can effectively integrate these three-dimensional features and enhance the local detail representation capability of the model. Extensive experiments conducted on four benchmark datasets for HSI task (Pavia University, Indian Pines, Trento and Houston 2013) indicate that our method achieves state-of-the-art (SOTA) performance compared with previous methods, which demonstrates the superiority and robust generalization capability of our Hyper-S3NN.

Finger Vein Identification Based on Large Kernel Convolution and Attention Mechanism.

FV (finger vein) identification is a biometric identification technology that extracts the features of FV images for identity authentication. To address the limitations of CNN-based FV identification, particularly the challenge of small receptive fields and difficulty in capturing long-range dependencies, an FV identification method named Let-Net (large kernel and attention mechanism network) was introduced, which combines local and global information. Firstly, Let-Net employs large kernels to capture a broader spectrum of spatial contextual information, utilizing deep convolution in conjunction with residual connections to curtail the volume of model parameters. Subsequently, an integrated attention mechanism is applied to augment information flow within the channel and spatial dimensions, effectively modeling global information for the extraction of crucial FV features. The experimental results on nine public datasets show that Let-Net has excellent identification performance, and the EER and accuracy rate on the FV_USM dataset can reach 0.04% and 99.77%. The parameter number and FLOPs of Let-Net are only 0.89M and 0.25G, which means that the time cost of training and reasoning of the model is low, and it is easier to deploy and integrate into various applications.

AML‐Net: Attention‐based multi‐scale lightweight model for brain tumour segmentation in internet of medical things

AbstractBrain tumour segmentation employing MRI images is important for disease diagnosis, monitoring, and treatment planning. Till now, many encoder‐decoder architectures have been developed for this purpose, with U‐Net being the most extensively utilised. However, these architectures require a lot of parameters to train and have a semantic gap. Some work tried to make a lightweight model and do channel pruning that made a small receptive field which compromised the accuracy. The authors propose an attention‐based multi‐scale lightweight model called AML‐Net in Internet of Medical Things to overcome the above issues. This model consists of three small encoder‐decoder architectures that are trained with different scale input images along with previously learned features to diminish the loss. Moreover, the authors designed an attention module which replaced the traditional skip connection. For the attention module, six different experiments were conducted, from which dilated convolution with spatial attention performed well. This attention module has three dilated convolutions which make a relatively large receptive field followed by spatial attention to extract global context from encoder low‐level features. Then these fine features are combined with the decoder's same layer of high‐level features. The authors perform the experiment on a low‐grade‐glioma dataset provided by the Cancer Genome Atlas which has at least Fluid‐Attenuated Inversion Recovery modality. The proposed model has 1/43.4, 1/30.3, 1/28.5, 1/20.2 and 1/16.7 fewer parameters than Z‐Net, U‐Net, Double U‐Net, BCDU‐Net and CU‐Net respectively. Moreover, the authors’ model gives results with IoU = 0.834, F1‐score = 0.909 and sensitivity = 0.939, which are greater than U‐Net, CU‐Net, RCA‐IUnet and PMED‐Net.

Focus Affinity Perception and Super-Resolution Embedding for Multifocus Image Fusion.

Despite the fact that there is a remarkable achievement on multifocus image fusion, most of the existing methods only generate a low-resolution image if the given source images suffer from low resolution. Obviously, a naive strategy is to independently conduct image fusion and image super-resolution. However, this two-step approach would inevitably introduce and enlarge artifacts in the final result if the result from the first step meets artifacts. To address this problem, in this article, we propose a novel method to simultaneously achieve image fusion and super-resolution in one framework, avoiding step-by-step processing of fusion and super-resolution. Since a small receptive field can discriminate the focusing characteristics of pixels in detailed regions, while a large receptive field is more robust to pixels in smooth regions, a subnetwork is first proposed to compute the affinity of features under different types of receptive fields, efficiently increasing the discriminability of focused pixels. Simultaneously, in order to prevent from distortion, a gradient embedding-based super-resolution subnetwork is also proposed, in which the features from the shallow layer, the deep layer, and the gradient map are jointly taken into account, allowing us to get an upsampled image with high resolution. Compared with the existing methods, which implemented fusion and super-resolution independently, our proposed method directly achieves these two tasks in a parallel way, avoiding artifacts caused by the inferior output of image fusion or super-resolution. Experiments conducted on the real-world dataset substantiate the superiority of our proposed method compared with state of the arts.

The super-resolution reconstruction algorithm of multi-scale dilated convolution residual network.

Aiming at the problems of traditional image super-resolution reconstruction algorithms in the image reconstruction process, such as small receptive field, insufficient multi-scale feature extraction, and easy loss of image feature information, a super-resolution reconstruction algorithm of multi-scale dilated convolution network based on dilated convolution is proposed in this paper. First, the algorithm extracts features from the same input image through the dilated convolution kernels of different receptive fields to obtain feature maps with different scales; then, through the residual attention dense block, further obtain the features of the original low resolution images, local residual connections are added to fuse multi-scale feature information between multiple channels, and residual nested networks and jump connections are used at the same time to speed up deep network convergence and avoid network degradation problems. Finally, deep network extraction features, and it is fused with input features to increase the nonlinear expression ability of the network to enhance the super-resolution reconstruction effect. Experimental results show that compared with Bicubic, SRCNN, ESPCN, VDSR, DRCN, LapSRN, MemNet, and DSRNet algorithms on the Set5, Set14, BSDS100, and Urban100 test sets, the proposed algorithm has improved peak signal-to-noise ratio and structural similarity, and reconstructed images. The visual effect is better.

Image Reconstruction for Accelerated MR Scan With Faster Fourier Convolutional Neural Networks.

High quality image reconstruction from undersampled k -space data is key to accelerating MR scanning. Current deep learning methods are limited by the small receptive fields in reconstruction networks, which restrict the exploitation of long-range information, and impede the mitigation of full-image artifacts, particularly in 3D reconstruction tasks. Additionally, the substantial computational demands of 3D reconstruction considerably hinder advancements in related fields. To tackle these challenges, we propose the following: 1) A novel convolution operator named Faster Fourier Convolution (FasterFC), aims at providing an adaptable broad receptive field for spatial domain reconstruction networks with fast computational speed. 2) A split-slice strategy that substantially reduces the computational load of 3D reconstruction, enabling high-resolution, multi-coil, 3D MR image reconstruction while fully utilizing inter-layer and intra-layer information. 3) A single-to-group algorithm that efficiently utilizes scan-specific and data-driven priors to enhance k -space interpolation effects. 4) A multi-stage, multi-coil, 3D fast MRI method, called the faster Fourier convolution based single-to-group network (FAS-Net), comprising a single-to-group k -space interpolation algorithm and a FasterFC-based image domain reconstruction module, significantly minimizes the computational demands of 3D reconstruction through split-slice strategy. Experimental evaluations conducted on the NYU fastMRI and Stanford MRI Data datasets reveal that the FasterFC significantly enhances the quality of both 2D and 3D reconstruction results. Moreover, FAS-Net, characterized as a method that can achieve high-resolution (320, 320, 256), multi-coil, (8 coils), 3D fast MRI, exhibits superior reconstruction performance compared to other state-of-the-art 2D and 3D methods.

GRA-Net: Global receptive attention network for surface defect detection

Surface defect detection is a key link in the production process of industrial products. However, it is still challenging to realize automatic detection due to the low contrast between the defect and background, the random spatial positions and shapes of defects, and the imbalance between positive and negative samples. Although convolutional neural network (CNN) has been successfully applied to defect detection, current CNN-based methods are still unable to cope with the above challenges because of the insufficient global representation ability, small receptive field, and difficulty in capturing tiny targets. To address these problems, this paper proposes a global receptive attention network (GRA-Net) for surface defect detection. This deep learning system introduces a backbone network combining CNN and transformers for feature extraction, so defects with low contrast can be identified. Then, a multi-level receptive field module is designed to achieve the overall perception for complex and changeable surface defects. Furthermore, a feature interaction attention module is developed to maximize the utilization of the comparatively few defect pixels. Finally, the defect boundary is refined by an adaptive boundary refinement block. Semantic segmentation experiments are conducted based on three datasets. The results show that the mean intersection-over-union and pixel accuracy of GRA-Net are 4–7 % and 2–5 % higher than the existing models, respectively.

Traffic-Sign-Detection Algorithm Based on SK-EVC-YOLO

Traffic sign detection is an important research direction in the process of intelligent transportation in the Internet era, and plays a crucial role in ensuring traffic safety. The purpose of this research is to propose a traffic-sign-detection algorithm based on the selective kernel attention (SK attention), explicit visual center (EVC), and YOLOv5 model to address the problems of small targets, incomplete detection, and insufficient detection accuracy in natural and complex road situations. First, the feature map with a smaller receptive field in the backbone network is fused with other scale feature maps to increase the small target detection layer. Then, the SK attention mechanism is introduced to extract and weigh features at different scales and levels, enhancing the attention to the target. By fusing the explicit visual center to gather local area features within the layer, the detection effect of small targets is improved. According to the experiment results, the mean average precision (mAP) on the Tsinghua-Tencent Traffic Sign Dataset (TT100K) for the proposed algorithm is 88.5%, which is 4.6% higher than the original model, demonstrating the practicality of the detection of small traffic signs.

SCGNet: Shifting and Cascaded Group Network

Many lightweight networks have been proposed for resource-limited applications, however, they cannot be efficiently applied to neural-network processing units (NPUs) due to the limited operations supported by the NPUs, and few works focus on efficient network design on the NPUs. The basic blocks of networks such as MobileNetV2 and RegNet use smaller convolution kernels with relatively small receptive fields, which are not conducive to capturing large-scale spatial information. To address this weakness, we propose Shifting and Cascaded Group (SCG) block, where we cascade group convolutions with larger kernels to exploit multi-scale information and propose shifting group convolution to communicate channel information between different groups. Besides, we carefully devise our architecture guided by some principles and finally build a very efficient network called Shifting and Cascaded Group Network (SCGNet) on NPUs. To verify the superiority of our method, we conduct extensive experiments on various tasks including image classification, object detection, human pose estimation, person re-identification, and semantic segmentation to comprehensively evaluate the performance. Results on widely used datasets such as ImageNet, PASCAL VOC, COCO, MPII, Market-1501, DukeMTMC-ReID, CUHK03, and Cityscapes demonstrate that the proposed network is a more effective network on the corresponding vision tasks.

Unsupervised registration for liver CT-MR images based on the multiscale integrated spatial-weight module and dual similarity guidance.

Multimodal registration is a key task in medical image analysis. Due to the large differences of multimodal images in intensity scale and texture pattern, it is a great challenge to design distinctive similarity metrics to guide deep learning-based multimodal image registration. Besides, since the limitation of the small receptive field, existing deep learning-based methods are mainly suitable for small deformation, but helpless for large deformation. To address the above issues, we present an unsupervised multimodal image registration method based on the multiscale integrated spatial-weight module and dual similarity guidance. In this method, a U-shape network with our multiscale integrated spatial-weight module is embedded into a multi-resolution image registration architecture to achieve end-to-end large deformation registration, where the spatial-weight module can effectively highlight the regions with large deformation and aggregate discriminative features, and the multi-resolution architecture further helps to solve the optimization problem of the network in a coarse-to-fine pattern. Furthermore, we introduce a special loss function based on dual similarity, which represents both global gray-scale similarity and local feature similarity, to optimize the unsupervised multimodal registration network. We verified the effectiveness of the proposed method on liver CT-MR images. Experimental results indicate that the proposed method achieves the optimal DSC value and TRE value of 92.70±1.75(%) and 6.52±2.94(mm), compared with other state-of-the-art registration algorithms. The proposed method can accurately estimate the large deformation field by aggregating multiscale features, and achieve higher registration accuracy and fast registration speed. Comparative experiments also demonstrate the effectiveness and generalization ability of the algorithm.

DBM-ViT: A multiscale features fusion algorithm for health status detection in CXR / CT lungs images

COVID-19 is a severe acute respiratory syndrome caused by SARS-CoV-2. It is highly contagious and spreads rapidly around the world. Although reverse transcription-polymerase chain reaction (RT-PCR) of viral nucleic acid is standard practice in confirmed patients. However, in the early stage of infection, RT-PCR has the disadvantages of low detection sensitivity and long detection cycles. Chest X-ray (CXR) and lung computed tomography (CT) scans combined with intelligent image recognition have the advantages of fast detection speed and low price. However, the traditional convolutional neural network (CNN) has a small receptive field in the image recognition process, which is not conducive to capturing the global features of the image. A single vision transformer (ViT) model lacks the characteristics of inductive bias. These affect the extraction of image fusion features. This paper proposed a DBM-ViT model for deep learning. The model utilized CXR/CT lung images for effective health detection of normal, COVID-19 and other types of pneumonia. The model employed depthwise convolutions with different expansion rates to efficiently captured global information from CXR/CT lung images. Then, the lung feature maps with combined sequences were fed into the ViT module to capture local information. Multi-scale features combined with global and local information ensure maximum feature learning. The results show that the detection accuracy of the DBM-ViT model in the CXR/CT image dataset reached 97.25%/98.36%. This method can effectively capture global and local information in lung images with high detection accuracy and can be used for rapid auxiliary diagnosis of pneumonia types.