Feature Map Fusion Research Articles

An Effective Multi-Scale Feature Network for Detecting Connector Solder Joint Defects

With the rapid development of industry, people’s requirements for the functionality, stability, and safety of electronic products are becoming higher and higher. As an important medium for power supply and information transmission functions of electronic products, high-quality soldering of cables and connectors ensures that the devices can operate normally. In this paper, we propose a multi-level feature detection network based on multi-level feature maps fusion and feature enhancement for detecting connector solder joints, classifying and locating qualified solder joints, and detecting seven common defective solder joints. This paper proposes a new feature map up-sampling algorithm and introduces a feature enhancement module, which better preserves the semantic information of higher-level feature maps, while at the same time enhancing the fused feature maps and weakening the effect of noise. Through comparison experiments, the mAP of the network proposed in this paper reaches 0.929 and the top-1 accuracy reaches 92%. The detection capability of each type of solder joint is greatly improved compared with the effect of other networks, which can assist engineers in the detection of weld joint quality and thus reduce the workload.

Analysis of Behavioral Image Recognition of Pan-Entertainment of Contemporary College Students’ Network

With the continuous update and iteration of network technology and technological innovation, the handheld smart media of college students will become more and more sensitive. With the advancement of economic globalization, various ideologies and cultures in the world will rapidly invade, and the “pan-entertainment” of online media may intensify. Only through the government’s supervision function and the self-discipline of the internet industry, we can strictly control and screen positive values. In order to better establish the correct employment value orientation of university students and further analyze the importance of the “pan-entertainment” behavior image recognition of college students, this study analyzes the related technology and basic theory of behavior recognition. After introducing several mainstream methods, the traditional dual-stream convolutional network method is improved, and the time information and spatial information extracted by the two channels are discussed for the weighted fusion of feature maps. Finally, using R(2 + 1)D structure and dual-stream network structure design, a deep learning-based spatiotemporal convolution behavior recognition algorithm is proposed. The proposed algorithm is tested and analyzed on the datasets UCF101 and HMDB51. The specific work content is as follows: (1) to summarize the widely used video behavior classification methods proposed so far and discuss the future development. Then, it mainly analyzes the existing technical bottlenecks of some methods based on deep learning methods and summarizes and explores an efficient, stable, and accurate spatiotemporal feature joint extraction and learning method theory. (2) The design of spatiotemporal convolutional network algorithm framework is proposed, the method of segmentation processing of long video is studied, the improvement of the dual-stream network decision-level fusion method is studied, and the R(2 + 1)D network is reorganized. The network algorithm is trained and tested on the UCF-101 dataset and HMDB-51 dataset under the condition of calling the pretrained model. Finally, the accuracy is compared with the existing classic algorithms to obtain better accuracy, which proves the effectiveness of the algorithm for the “pan-entertainment” behavioral image recognition of contemporary college students.

DermoExpert: Skin lesion classification using a hybrid convolutional neural network through segmentation, transfer learning, and augmentation

Background and Objective:Although automated Skin Lesion Classification (SLC) is a crucial integral step in computer-aided diagnosis, it remains challenging due to variability in textures, colors, indistinguishable boundaries, and shapes. Methods:This article proposes an automated dermoscopic SLC framework named Dermoscopic Expert (DermoExpert). It combines the pre-processing and hybrid Convolutional Neural Network (hybrid-CNN). The proposed hybrid-CNN has three distinct feature extractor modules, which are fused to achieve better-depth feature maps of the lesion. Those single and fused feature maps are classified using different fully connected layers, then ensembled to predict a lesion class. In the proposed pre-processing, we apply lesion segmentation, augmentation (geometry- and intensity-based), and class rebalancing (penalizing the majority class’s loss and merging additional images to the minority classes). Moreover, we leverage transfer learning from the pre-trained models. Finally, we deploy the weights of our DermoExpert to a possible web application. Results:We evaluate our DermoExpert on the ISIC-2016, ISIC-2017, and ISIC-2018 datasets, where the DermoExpert has achieved the area under the receiver operating characteristic curve (AUC) of 0.96, 0.95, and 0.97, respectively. The experimental results improve the state-of-the-art by the margins of 10.0% and 2.0%, respectively, for the ISIC-2016 and ISIC-2017 datasets in terms of AUC. The DermoExpert also outperforms by 3.0% for the ISIC-2018 dataset concerning a balanced accuracy. Conclusion:Since DermoExpert provides better classification outcomes on three different datasets, leading to a better recognition tool to assist dermatologists. Our source code and segmented masks for the ISIC-2018 dataset will be available as a public benchmark for future improvements.

Fine-Grained Recognition for Oriented Ship Against Complex Scenes in Optical Remote Sensing Images

Remote sensing ship recognition is widely used in civil and military applications, such as national defense construction, fishery management, and navigation supervision. Unfortunately, limited by the lack of public datasets for fine-grained ship recognition, current studies mainly focus on ship detection or coarse-grained ship recognition, whereas fine-grained recognition task is left out. The main challenges for fine-grained ship recognition include: 1) complex scenes and 2) ship characteristics of arbitrary orientation, dense distribution, and huge scale and appearance variation. Aiming at the challenges above, we propose a novel efficient information reuse network (EIRNet) and establish a public 20-class Dataset for Oriented Ship Recognition (DOSR). In our EIRNet, considering recognition robustness to multiscale ships, a dense feature fusion network (DFF-Net) with two fusion directions is designed to maximize the utilization of multilayer information and reduce information redundancy. Then, the fused feature maps are refined by a dual-mask attention module (DMAM) to improve performance in dense and clutter scenes by enhancing the distinction between ships and suppressing clutter. Furthermore, Mask-RPN improves the efficiency of generating proposals by reusing the attention mask. Finally, we introduce a concept of upper level class to mine interclass relationships, which further improves the recognition accuracy. Extensive experiments demonstrate that our EIRNet achieves state-of-the-art performance on DOSR and another popular public dataset called HRSC2016.

Pay Attention to Local Contrast Learning Networks for Infrared Small Target Detection

Infrared small target suffers from the lack of intrinsic features, context and samples. Conventional detection methods are usually unable to sufficiently and effectively extract the features of infrared small targets. Therefore, we propose a novel attention-based local contrast learning network (ALCL-Net). Considering the scarcity of intrinsic features of infrared small targets, we propose ResNet32, which enhances the ability to extract infrared small target features and avoids the problem that the target features are overwhelmed by the background features due to too deep network. At the same time, we construct a simplified bilinear interpolation attention module (SBAM), which is used for fusion of hierarchical feature maps. It has fast inference speed and can focus on the feature of the target in the lack of context. Furthermore, local contrast learning (LCL) is introduced, which adopts the local contrast idea of non-deep learning methods. It can alleviate the dependence on dataset samples, thereby improving detection accuracy on datasets with few samples. Compared with the state-of-the-art methods, the proposed ALCL-Net achieves superior performance with an intersection-over-union (IoU) of 0.792 and normalized IoU (nIoU) of 0.771 on the public SIRST dataset.

Scenario Context-Aware-Based Bidirectional Feature Pyramid Network for Remote Sensing Target Detection

Compared with ordinary optical images, the situation of remote sensing images is much more complicated. The problems caused by the shooting angles over the Earth’s surface are: 1) some target categories with more complex shooting environments greatly increase the difficulty of detection and 2) the remote sensing images with large and small targets at the same time leading to large changes in the target scale are difficult to handle. In this letter, we designed a novel scenario context-aware-based bidirectional feature pyramid network (SCBi-FPN) to address the above problems. There are two key modules of the proposed network: the scene context-aware module uses pyramid pooling to aggregate contextual information of the different regions to obtain better global contextual information. The bidirectional feature pyramid network (Bi-FPN) module with squeeze and excitation (SE) blocks connects feature layers at different scales in a cross-scale manner and performs weighted feature map fusion before passing through the SE blocks to enable the network to obtain more accurate information. The experiments demonstrate that our designed network has good results compared with the state-of-the-art methods. In particular, we achieved mean average precision (mAP) of 92.92 on the publicly available NWPU VHR-10 dataset.

Segmentation method of myocardial perfusion bull-eye for the degree of loss of cardiac ischemia

As one of the non-invasive imaging techniques, myocardial perfusion imaging provides a basis for the diagnosis of myocardial ischemia in coronary heart disease. Aiming at the bull-eye image in myocardial perfusion imaging, this paper proposed a branching structure, which included multi-layer transposed convolution up-sampling concatenate module and four-channel weighted channels attention module, and the output results of the branch structure were fused with the output results of trunk U-Net, to achieve accurate segmentation of the cardiac ischemia missing degree in myocardial perfusion bull-eye image. The experimental results show that the multi-layer transposed convolution up-sampling concatenate module realizes the fusion of different depth feature maps, and effectively reduces the interference of the severe sparse degree which is similar to the missing degree on the segmentation. Four-channel weighted attention module can further improve the ability to distinguish between the two similar degrees and the ability to learn edge details of the targets, and retain more abundant edge details features. The experimental data came from Tianjin Medical University General Hospital, Tianjin TEDA Hospital, Tianjin First Central Hospital and Third Central Hospital. The Jaccard scores in the self-built dataset was 5.00% higher than that of U-Net. The model presented in this paper is superior to other optimized models based on U-Net, and the subjective evaluation meets the accuracy requirements for clinical diagnosis.

CCAFFMNet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module

Dual-spectral (RGB-thermal) semantic segmentation is a fundamental task for visual perception of autonomous driving in harsh imaging environments (such as darkness, rain, and fog). In recent years, the encoder-decoder dual-spectral semantic segmentation networks have achieved satisfactory results. However, existing networks pay little attention to the feature-fusion strategy of infrared and RGB features at each feature-fusion stage, which limits the performance of semantic segmentation. This study proposes a novel encoder-decoder-based dual-spectral semantic segmentation network. Channel-coordinate attention feature-fusion modules (CCAFFMs) are designed and inserted into each feature-fusion stage to obtain the channel and spatial correlations between infrared and RGB features. Thus, fused feature maps are refined in this way. A down-up-connected decoder with skip connections is designed to restore the resolution of the feature map and ensure that it contains more object details and sharper boundary contours. Furthermore, we manually annotate and augment the RoadScene dataset to construct the RoadScene-seg dataset. In this way, dual-spectral semantic segmentation can be extended to diverse autonomous driving environments. The results of extensive experiments on the MF and RoadScene-seg datasets prove the superiority of the proposed network over state-of-the-art methods.

Concrete crack segmentation based on UAV-enabled edge computing

In recent years, the rapid development of UAV technology has greatly improved the efficiency of the detection of concrete bridge cracks. With the increase in the number of bridge inspection UAVs, the number of tasks handled by cloud services has increased linearly, resulting in increased computational pressure on cloud services. In order to reduce the computational load of cloud servers, we proposed a crack segmentation network based on UAV-enabled edge computing. However, due to the limitation of computational capability of edge computing and the strength inhomogeneity and background complexity of cracks, crack detection is still a challenging task. Thus, we proposed an effective concrete crack segmentation network based on UAV-enabled edge computing, the network used feature map fusion to fuse different levels of feature map information into lower-level features for crack detection. The atrous spatial pyramid pooling network was used to increase the low-resolution feature map receptive field information for cracks and to enhance the detection accuracy for cracks of different scales. In addition, loss functions for crack datasets were proposed to solve the problem of imbalance due to positive and negative samples in the concrete crack images. Experiments demonstrated that the proposed methods are better than the state-of-the-art edge detection and semantic segmentation methods in terms of accuracy and generality.

A Reconfigurable Neural Network Processor With Tile-Grained Multicore Pipeline for Object Detection on FPGA

In order to improve the computational efficiency of convolutional neural networks (CNNs) for object detection on reconfigurable platforms such as field-programmable gate arrays (FPGAs), we propose a CNN processor with hierarchical pipelining and multicore reconfigurable computing based on parallel parameter constraints. First, we propose a pipelined multicore processing architecture that can adapt to computations and on-chip memory requirements of different convolutional layers. We present the design of a CNN processor that can configure the computing units while adjusting the interconnection of multicore to improve the utilization of reconfigurable computing resources. Second, we propose an elastic on-chip buffer and a data access approach by dynamically configuring addresses to better utilize on-chip memory. Meanwhile, we present a cross-layer feature map fusion strategy based on computing near memory (CNM) to reduce off-chip memory accesses. Finally, we propose a scheduling algorithm for pipelined tasks to improve the computational efficiency and throughput of the proposed processor. For evaluation, the well-known object detection methods (RetinaNet-ResNet-50, MobileNetV2-SSDLite, and YOLOv3) performed using the proposed CNN processor on the ZCU102 platform and reached the throughput of 1503, 1066, and 809 GOPS and the computational efficiency of 0.79, 0.62, and 0.36 GOPS/DSP, respectively. The designed processor realized a better tradeoff between computing efficiency and detection accuracy compared with the recently proposed object detection CNNs on FPGA.

Fractional Wavelet-Based Generative Scattering Networks.

Generative adversarial networks and variational autoencoders (VAEs) provide impressive image generation from Gaussian white noise, but both are difficult to train, since they need a generator (or encoder) and a discriminator (or decoder) to be trained simultaneously, which can easily lead to unstable training. To solve or alleviate these synchronous training problems of generative adversarial networks (GANs) and VAEs, researchers recently proposed generative scattering networks (GSNs), which use wavelet scattering networks (ScatNets) as the encoder to obtain features (or ScatNet embeddings) and convolutional neural networks (CNNs) as the decoder to generate an image. The advantage of GSNs is that the parameters of ScatNets do not need to be learned, while the disadvantage of GSNs is that their ability to obtain representations of ScatNets is slightly weaker than that of CNNs. In addition, the dimensionality reduction method of principal component analysis (PCA) can easily lead to overfitting in the training of GSNs and, therefore, affect the quality of generated images in the testing process. To further improve the quality of generated images while keeping the advantages of GSNs, this study proposes generative fractional scattering networks (GFRSNs), which use more expressive fractional wavelet scattering networks (FrScatNets), instead of ScatNets as the encoder to obtain features (or FrScatNet embeddings) and use similar CNNs of GSNs as the decoder to generate an image. Additionally, this study develops a new dimensionality reduction method named feature-map fusion (FMF) instead of performing PCA to better retain the information of FrScatNets,; it also discusses the effect of image fusion on the quality of the generated image. The experimental results obtained on the CIFAR-10 and CelebA datasets show that the proposed GFRSNs can lead to better generated images than the original GSNs on testing datasets. The experimental results of the proposed GFRSNs with deep convolutional GAN (DCGAN), progressive GAN (PGAN), and CycleGAN are also given.

Defocus spread effect elimination method in multiple multi-focus image fusion for microscopic images

For a microscopic imaging scene, an all-in-focus image of the observation object is needed. Because of the limitation of the depth of field of the camera and the typically uneven surface of the observation object, an all-in-focus image is obtained through one shot with relative difficulty. In this case, an alternative method for obtaining the all-in-focus image is usually used, which is to fuse several images focusing on different depths with the help of multi-focus image fusion technology. Multi-focus image fusion is an important branch in the field of computer vision. It aims to use image processing technology to fuse the clear regions of multiple images, focusing on different objects in the same scene, and finally to obtain an all-in-focus fusion result. With the breakthrough of machine learning theory represented by deep learning, the convolutional neural network is widely adopted in the field of multi-focus image fusion. However, most methods only focus on improving network structure and use the simple one-by-one serial fusion method, which reduces the efficiency of multiple image fusion. In addition, the defocus spread effect in the fusion process, which causes blurred artifacts in the areas near focus map boundaries, can severely affect the quality of fusion results. In the application of microscopic imaging analysis, we proposed a maximum spatial frequency in the feature map (MSFIFM) fusion strategy. By adding a post-processing module in the convolution neural network based on unsupervised learning, the redundant feature extraction process in the one-by-one serial fusion is avoided. Experiments demonstrate that this strategy can significantly improve the efficiency of multi-focus image fusion with multiple images. In addition, we presented a correction strategy that can effectively alleviate the effect of defocus spread on the fusion result under the condition of ensuring the efficiency of the algorithm fusion.

Real-time semantic segmentation with weighted factorized-depthwise convolution

Semantic segmentation has achieved great success with the popularity of convolutional neural networks (CNNs). However, the huge computational burden restricts the application of most existing networks on edge devices with strict inference time constraints. To solve this problem, a weighted factorized-depthwise convolution network (WFDCNet) is presented in this paper, which contains full- dimensional continuous separation convolution (FCS) modules and a lateral asymmetric pyramid fusion (LAPF) module, aiming to obtain high accuracy without damaging inference speed. Specifically, the FCS module enables the calculation of each dimension to be completed independently in a continuous separation process and uses simplified SE (SSE) attention layer to adjust the channels, achieving the extensive extraction of feature information. The LAPF module is able to eliminate semantic divergence and fuse feature maps of three different scales to realize the combination of multiple information from the front-end and the back-end network. WFDCNet shows superior performance on Cityscapes, Camvid, Mapillary Vistas and COCO-Stuff datasets. Especially, the experimental results demonstrate that our network achieves 73.7% mIoU on Cityscapes dataset, with the inference speed of 102.6FPS on a single RTX 2080 Ti GPU, and 17.2FPS on Jetson TX2.

DCU-Net: a dual-channel U-shaped network for image splicing forgery detection.

The detection and location of image splicing forgery are a challenging task in the field of image forensics. It is to study whether an image contains a suspicious tampered area pasted from another image. In this paper, we propose a new image tamper location method based on dual-channel U-Net, that is, DCU-Net. The detection framework based on DCU-Net is mainly divided into three parts: encoder, feature fusion, and decoder. Firstly, high-pass filters are used to extract the residual of the tampered image and generate the residual image, which contains the edge information of the tampered area. Secondly, a dual-channel encoding network model is constructed. The input of the model is the original tampered image and the tampered residual image. Then, the deep features extracted from the dual-channel encoding network are fused for the first time, and then the tampered features with different granularity are extracted by dilation convolution, and then, the secondary fusion is carried out. Finally, the fused feature map is input into the decoder, and the predicted image is decoded layer by layer. The experimental results on Casia2.0 and Columbia datasets show that DCU-Net performs better than the latest algorithm and can accurately locate tampered areas. In addition, the attack experiments show that DCU-Net model has good robustness and can resist noise and JPEG recompression attacks.

A Densely Connected Network Based on U-Net for Medical Image Segmentation

The U-Net has become the most popular structure in medical image segmentation in recent years. Although its performance for medical image segmentation is outstanding, a large number of experiments demonstrate that the classical U-Net network architecture seems to be insufficient when the size of segmentation targets changes and the imbalance happens between target and background in different forms of segmentation. To improve the U-Net network architecture, we develop a new architecture named densely connected U-Net (DenseUNet) network in this article. The proposed DenseUNet network adopts a dense block to improve the feature extraction capability and employs a multi-feature fuse block fusing feature maps of different levels to increase the accuracy of feature extraction. In addition, in view of the advantages of the cross entropy and the dice loss functions, a new loss function for the DenseUNet network is proposed to deal with the imbalance between target and background. Finally, we test the proposed DenseUNet network and compared it with the multi-resolutional U-Net (MultiResUNet) and the classic U-Net networks on three different datasets. The experimental results show that the DenseUNet network has significantly performances compared with the MultiResUNet and the classic U-Net networks.

Predictive and discriminative localization of pathology using high resolution class activation maps with CNNs.

PurposeExisting class activation mapping (CAM) techniques extract the feature maps only from a single layer of the convolutional neural net (CNN), generally from the final layer and then interpolate to upsample to the original image resolution to locate the discriminative regions. Consequently these provide a coarse localization that may not be able to capture subtle abnormalities in medical images. To alleviate this, our work proposes a technique called high resolution class activation mapping (HR-CAMs) that can provide enhanced visual explainability to the CNN models.MethodsHR-CAMs fuse feature maps by training a network using the input from multiple layers of a trained CNN, thus gaining information from every layer that can localize abnormalities with greater details in original image resolution. The technique is validated qualitatively and quantitatively on a simulated dataset of 8,000 images followed by applications on multiple image analysis tasks that include (1) skin lesion classification (ISIC open dataset—25,331 cases) and (2) predicting bone fractures (MURA open dataset—40,561 images) (3) predicting Parkinson’s disease (PD) from neuromelanin sensitive MRI (small cohort-80 subjects).ResultsWe demonstrate that our model creates clinically interpretable subject specific high resolution discriminative localizations when compared to widely used CAMs and Gradient-CAMs.ConclusionHR-CAMs provide finer delineation of abnormalities thus facilitating superior explainability to CNNs as has been demonstrated from its rigorous validation.

Multi-granularity scale-aware networks for hard pixels segmentation of pulmonary nodules

Accurate automatic segmentation of pulmonary nodules can greatly assist in the early clinical diagnosis and analysis of lung cancer. However, it remains a challenging task due to (1) the variable scales, complex shapes, and textures heterogeneity of nodules and (2) the similar visual characteristics of nodules and the surrounding environment. These factors make it difficult for a model to capture the representative and distinguishing features of nodules. We define these pixels, which are difficult to segment accurately, as “hard pixels.” To comprehensively tackle these challenges, we focus on the complementarity between surrounding background information and salient nodules information. Accordingly, we propose a Multi-Granularity Scale-Aware Networks (MGSA-Net) for accurate pulmonary nodules segmentation with steps as follows. First, to effectively preserve both global contextual and local fine details information, we unify the representation of the feature about global and patch-level images in a single framework. Second, we introduce a deep scale-aware module (DSAM) in the global stream that could generate multi-scale feature maps with a uniform representational power, which can process more contextual information. Finally, we propose a multi-granularity feature map sharing learning to fuse feature maps from the dual branch at various scales. Benefiting from the rich background information and salient nodules information, the fused features can help simultaneously capture the similarity of nodules and the diversity of background to mine hard pixels. Especially, the two streams are jointly optimized, ensuring they are mutually reinforced and refining current pixels’ predictions by their similar structure boundaries. Extensive experiments on the LIDC-IDRI dataset have demonstrated that the proposed MGSA-Net could surpass most segmentation models and advance the state-of-the-art performance with the dice similarity coefficient (DSC) of 87.32% on the dataset. Code can be available at: https://github.com/ISSE-AILab/MGSA-Net.

MDC-net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information

Accurate segmentation of nuclei in digital pathology images can assist doctors in diagnosing diseases and evaluating subsequent treatments. Manual segmentation of nuclei from pathology images is time-consuming because of the large number of nuclei and is also error-prone. Therefore, accurate and automatic nucleus segmentation methods are required. Owing to the large variations in the characterization of nuclei, it is difficult to accurately segment nuclei using traditional methods. In this study, we propose a new method for nucleus segmentation. The proposed method uses a deep fully convolutional neural network to perform end-to-end segmentation on pathological tissue slices. Multiple short residual connections were used to fuse feature maps from different scales to better utilize the context information. Dilated convolutions with different dilation ratios were used to increase the receptive fields. In addition, we incorporated the distance map and contour information into the segmentation method to segment touching nuclei, which is difficult via traditional segmentation methods. Finally, post-processing was used to improve the segmentation results. The results demonstrate that our segmentation method can obtain comparable or better performance than other state-of-the-art methods on the public nuclei histopathology datasets.

FAGNet: Multi-Scale Object Detection Method in Remote Sensing Images by Combining MAFPN and GVR

<p indent=0mm>Remote sensing images of large scenes are complex, and have the characteristics of many categories of objects, different scales and changeable directions, which lead to the problem of multi-class, multi-scale and multi-oriented of objects in remote sensing images. A remote sensing image object detection method based on multi-scale attention feature pyramid network (MAFPN) and gliding vertex regression (GVR) mechanism is proposed. Firstly, multi-layer feature maps are extracted from backbone network as input of MAFPN, which combines feature fusion and attention mechanism. On the basis of fusing feature maps of multi-scale, channel attention and spatial attention mechanisms are used to suppress noise, enhance effective feature reuse, and improve the network’s adaptability to object multi-scale features. The fusion feature map output by MAFPN is input to the region proposal network to generate the regions of interest, and then they will be sent to the classification regression network. The GVR mechanism is used in the object classification regression network and the four vertex offset ratio parameters and rotation factors are added on the basis of predicting the horizontal boxes, which converts the horizontal boxes into a rotating box to reduce the redundant area in the bounding boxes, makes the predicted rotating bounding boxes fit the object more closely. The experimental results on the DOTA public dataset, compared with many classical detection algorithms based on convolutional neural networks, show that the average detection accuracy of the proposed method is significantly improved, which can detect objects of multi-scales and multi-oriented more accurately, and achieve the robust detection of multi-scale objects.

A 3D medical image registration method based on multi-scale feature fusion

Deformable medical image registration refers to finding a certain transformation so that the corresponding points of two medical images can be aligned in space. This has important clinical applications. In this article, we propose an unsupervised end-to-end medical image registration method. In this method, the fixed image and the moving image are concatenated in series and input into the convolution neural network to obtain the feature images of different scales. In order to improve the ability of neural network to capture global and local information, we fuse feature maps of different scales. The spatial transformation network uses the deformation field to deform the moving image, so as to realize the registration of the two pairs. We validate our method in the ABIDE data set and compare it with some classic state-of-the-art methods. The experimental results show that our method improves the registration accuracy of image pairs.