Switchable Normalization Research Articles

HLSNC-GAN: Medical Image Synthesis Using Hinge Loss and Switchable Normalization in CycleGAN

HLSNC-GAN: Medical Image Synthesis Using Hinge Loss and Switchable Normalization in CycleGAN

Imbalanced Fault Diagnosis Using Conditional Wasserstein Generative Adversarial Networks With Switchable Normalization

Imbalanced Fault Diagnosis Using Conditional Wasserstein Generative Adversarial Networks With Switchable Normalization

RGSB-UNet: Hybrid Deep Learning Framework for Tumour Segmentation in Digital Pathology Images.

Colorectal cancer (CRC) is a prevalent gastrointestinal tumour with high incidence and mortality rates. Early screening for CRC can improve cure rates and reduce mortality. Recently, deep convolution neural network (CNN)-based pathological image diagnosis has been intensively studied to meet the challenge of time-consuming and labour-intense manual analysis of high-resolution whole slide images (WSIs). Despite the achievements made, deep CNN-based methods still suffer from some limitations, and the fundamental problem is that they cannot capture global features. To address this issue, we propose a hybrid deep learning framework (RGSB-UNet) for automatic tumour segmentation in WSIs. The framework adopts a UNet architecture that consists of the newly-designed residual ghost block with switchable normalization (RGS) and the bottleneck transformer (BoT) for downsampling to extract refined features, and the transposed convolution and 1 × 1 convolution with ReLU for upsampling to restore the feature map resolution to that of the original image. The proposed framework combines the advantages of the spatial-local correlation of CNNs and the long-distance feature dependencies of BoT, ensuring its capacity of extracting more refined features and robustness to varying batch sizes. Additionally, we consider a class-wise dice loss (CDL) function to train the segmentation network. The proposed network achieves state-of-the-art segmentation performance under small batch sizes. Experimental results on DigestPath2019 and GlaS datasets demonstrate that our proposed model produces superior evaluation scores and state-of-the-art segmentation results.

MFE-ResNet: A new extraction framework for land cover characterization in mining areas

Obtaining accurate information about land cover is a critical aspect of environmental monitoring in mining areas. However, the land cover in these areas is often complex and poses a challenge for analysis compared to other landscapes. This results in the difficulty of acquiring effective features and an unbalanced proportion of available training samples. We introduce a new approach to obtaining effective image features and enhancing the extraction of open-pit (OP) and waste-dump area (WDA) by using a mining feature-enhanced ResNet (MFE-ResNet). Our framework utilizes image objects as the basic units of high spatial resolution images (HSRIs) to better incorporate spatial and geometric information. The MFE-ResNet is an improvement of the ResNet18 architecture, forming a 26-layer network, and reducing the computational complexity by changing the convolutional kernel size. A squeeze-and-excitation (SE) module is also introduced to form an SE-residual structure to improve the network’s ability to perceive the detailed mining area features. To alleviate the sample imbalance problem, a switchable normalization (SN) layer is applied to optimize the internal network structure. In addition, focal loss is introduced in the training process. We build an evaluation system that considers both the number of objects and their area. Compared with other six deep learning methods, our MFE-ResNet achieves the best overall accuracy in both large and small study areas. Specifically, MFE-ResNet obtained F values (F) of 87.73% and 97.28%, and quality (Q) of 81.37% and 95.18% in OP extraction. The F of WDA extraction improved by 0.99% and 2.13%, and the Q improved by 0.98% and 9.56% compared to the second-ranked model. We show that our framework exhibits promising performance in monitoring mining areas.

A Rolling Bearing Fault Diagnosis Method Based on Switchable Normalization and a Deep Convolutional Neural Network

Aiming to address the problems of a low fault detection rate and poor diagnosis performance under different loads and noise environments, a rolling bearing fault diagnosis method based on switchable normalization and a deep convolutional neural network (SNDCNN) is proposed. The method effectively extracted the fault features from the raw vibration signal and suppressed high-frequency noise by increasing the convolution kernel width of the first layer and stacking multiple layers’ convolution kernels. To avoid losing the intensity information of the features, the K-max pooling operation was adopted at the pooling layer. To solve the overfitting problem and improve the generalization ability, a switchable normalization approach was used after each convolutional layer. The proposed SNDCNN was evaluated with two sets of rolling bearing datasets and obtained a higher fault detection rate than SVM and BP, reaching a fault detection rate of over 90% under different loads and demonstrating a better anti-noise performance.

Depthwise separable convolution Unet for 3D seismic data interpolation

In seismic exploration, dense and evenly spatial sampled seismic traces are crucial for successful implementation of most seismic data processing and interpretation algorithms. Recently, numerous seismic data reconstruction approaches based on deep learning have been presented. High dimension-based methods have the benefit of making full use of seismic signal at different perspectives. However, with the transformation of data dimension from low to high, the parameter capacity and computation cost of training deep neural network increase significantly. In this paper, we introduce depthwise separable convolution instead of standard convolution to reduce the operation cost of Unet for 3D seismic data missing trace interpolation. The structural similarity (SSIM), L1 hybrid loss function, and switchable normalization further improve the reconstruction performance of the network. The comparative experiments on the synthetic and field seismic data show that depthwise separable convolution can effectively reduce the number of network parameters and computation intensity with the interpolation results comparable to the standard convolution results.

Block Attention and Switchable Normalization Based Deep Learning Framework for Segmentation of Retinal Vessels

The presence of high blood sugar levels damages blood vessels and causes an eye condition called diabetic retinopathy. The ophthalmologist can detect this disease by looking at the variations in retinal blood vasculature. Manual segmentation of vessels requires highly skilled specialists, and not possible for many patients to be done quickly in their daily routine. For these reasons, it is of great importance to isolate retinal vessels precisely, quickly, and accurately. The difficulty distinguishing the retinal vessels from the background, and the small number of samples in the databases make this segmentation problem difficult. In this study, we propose a novel network called Block Feature Map Distorted Switchable Normalization U-net with Global Context Informative Convolutional Block Attention Module (BFMD SN U-net with GCI- CBAM). We improve the traditional Fully Convolutional Segmentation Networks in multiple aspects with the proposed model; The model converges in earlier epochs, adapts more flexibly to different data, is more robust against overfitting, and gets better feature refinement at different dilation rates to cope with different sizes of retinal vessels. We evaluate the proposed network on two reference retinal datasets, DRIVE and CHASE DB1, and achieve state-of-the-art performances with 97.00 % accuracy and 98.71 % AUC in DRIVE and 97.62 % accuracy and 99.11 % AUC on CHASE DB1 databases. Additionally, the convergence step of the model is reduced and it has fewer parameters than the baseline U-net. In summary, the proposed model surpasses the U-net-based approaches used for retinal vessel separation in the literature.

MIC-Net: A deep network for cross-site segmentation of COVID-19 infection in the fog-assisted IoMT

The automatic segmentation of COVID-19 pneumonia from a computerized tomography (CT) scan has become a major interest for scholars in developing a powerful diagnostic framework in the Internet of Medical Things (IoMT). Federated deep learning (FDL) is considered a promising approach for efficient and cooperative training from multi-institutional image data. However, the nonindependent and identically distributed (Non-IID) data from health care remain a remarkable challenge, limiting the applicability of FDL in the real world. The variability in features incurred by different scanning protocols, scanners, or acquisition parameters produces the learning drift phenomena during the training, which impairs both the training speed and segmentation performance of the model. This paper proposes a novel FDL approach for reliable and efficient multi-institutional COVID-19 segmentation, called MIC-Net. MIC-Net consists of three main building modules: the down-sampler, context enrichment (CE) module, and up-sampler. The down-sampler was designed to effectively learn both local and global representations from input CT scans by combining the advantages of lightweight convolutional and attention modules. The contextual enrichment (CE) module is introduced to enable the network to capture the contextual representation that can be later exploited to enrich the semantic knowledge of the up-sampler through skip connections. To further tackle the inter-site heterogeneity within the model, the approach uses an adaptive and switchable normalization (ASN) to adaptively choose the best normalization strategy according to the underlying data. A novel federated periodic selection protocol (FED-PCS) is proposed to fairly select the training participants according to their resource state, data quality, and loss of a local model. The results of an experimental evaluation of MIC-Net on three publicly available data sets show its robust performance, with an average dice score of 88.90% and an average surface dice of 87.53%.

A Flexible Reference-Insensitive Spatiotemporal Fusion Model for Remote Sensing Images Using Conditional Generative Adversarial Network

Due to the tradeoff between spatial and temporal resolutions of remote sensing images, spatiotemporal fusion models were proposed to synthesize the high spatiotemporal image series. Currently, spatiotemporal fusion models usually employ one coarse-resolution image acquired on a prediction date and at least another pair of coarse–fine resolution images close to the prediction time as references to derive the fine-resolution image on the prediction date. After years of development, the model accuracy has gained a certain improvement, but nearly, all the models require at least three image inputs and rigid time constraints must be applied to the references to guarantee the fusion accuracy. However, it is not always that easy to collect adequate data pairs for fine-resolution image series simulation in practice because of the bad weather condition or the time inconsistency between the coarse–fine resolution data sources, which causes some difficulties in the actual application. This article introduces the conditional generative adversarial network (CGAN) and switchable normalization technique into the spatiotemporal fusion problem and proposes a flexible deep network named the GAN-based SpatioTemporal Fusion Model (GAN-STFM) to reduce the number of model inputs and broke the time restriction on reference image selection. The GAN-STFM just needs a coarse-resolution image on the prediction date and another fine-resolution reference image at an arbitrary time in the same area for model inputs. As far as we know, this is the first spatiotemporal fusion model that requires only two images as model inputs and puts no restriction on the acquisition time of references. Even so, the GAN-STFM performs on par or better than other classical fusion models in the experiments. With this improvement, the data preparation for spatiotemporal fusion tends to be much easier than before, showing a promising perspective for practical applications.

DFPGAN: Dual fusion path generative adversarial network for infrared and visible image fusion

Infrared and visible image fusion is an essential task for multi-sensor image fusion. Generative adversarial networks (GAN) have achieved remarkable performance in the fusion of infrared and visible image. Existing GAN based fusion methods merely using infrared and visible image as input for the fusion, while we found that differential images obtained by subtraction between two image sources could provide contrast information for the fusion. To this end, a novel dual fusion path generative adversarial network (DFPGAN) is proposed in this paper for infrared and visible image fusion. We divided the generator of generative adversarial network into two fusion paths namely infrared–visible path and differential path. The input of infrared–visible path concatenated two image sources to make infrared intensity and texture details keep balance fusion in this path. The input of differential path concatenated differential images obtained by subtraction between two image sources to make contrast information fusion in this path. The features extracted by two fusion paths are concatenated at the end of the generator to generate fused images with contrast effect and balanced information distribution. Meanwhile, we have implemented dual self-attention feature refine module (DSAM) on two fusion paths to refine feature maps in two fusion paths. We adopted switchable normalization layer (SN) substitute for batch normalization layer (BN) in the generator and discriminator to avoid fusion artifact. Furthermore, a mixed content loss is integrated in the generator loss functions to guide the generated image keep balanced information distribution and preserving contrast simultaneously. The adversarial training employed dual adversarial architecture to balance the distribution of infrared intensity and texture details. To verifying the improvement effect of fusion image on target detection, we introduce the Scaled-YOLOv4 target detection framework as evaluation framework, and use the proposed network to fuse RGB images and infrared images for target detection. The results of qualitative and quantitative experiments conducted on public datasets demonstrated the superiority of proposed network over other state-of-the-art methods and could generate fused images with distinctly contrast.

TSSN-Net: Two-step Sparse Switchable Normalization for learning correspondences with heavy outliers

In this paper, we solve the problem of feature matching by designing an end-to-end network (called TSSN-Net). Given putative correspondences of feature points in two views, existing deep learning based methods formulate the feature matching problem as a binary classification problem. In these methods, a normalizer plays an important role in the networks. However, they adopt the same normalizer in all normalization layers of the entire networks, which will result in suboptimal performance. To address this problem, we propose a Two-step Sparse Switchable Normalization Block, which involves the advantage of adaptive normalization for different convolution layers from Sparse Switchable Normalization and robust global context information from Context Normalization. Moreover, to capture local information of correspondences, we propose a Multi-Scale Correspondence Grouping algorithm, by defining a multi-scale neighborhood representation, to search for consistent neighbors of each correspondence. Finally, with a series of convolution layers, the end-to-end TSSN-Net is proposed to learn correspondences with heavy outliers for feature matching. Our experimental results have shown that our network achieves the state-of-the-art performance on benchmark datasets.

Switchable Normalization for Learning-to-Normalize Deep Representation

We address a learning-to-normalize problem by proposing Switchable Normalization (SN), which learns to select different normalizers for different normalization layers of a deep neural network. SN employs three distinct scopes to compute statistics (means and variances) including a channel, a layer, and a minibatch. SN switches between them by learning their importance weights in an end-to-end manner. It has several good properties. First, it adapts to various network architectures and tasks (see Fig. 1). Second, it is robust to a wide range of batch sizes, maintaining high performance even when small minibatch is presented (e.g., 2 images/GPU). Third, SN does not have sensitive hyper-parameter, unlike group normalization that searches the number of groups as a hyper-parameter. Without bells and whistles, SN outperforms its counterparts on various challenging benchmarks, such as ImageNet, COCO, CityScapes, ADE20K, MegaFace and Kinetics. Analyses of SN are also presented to answer the following three questions: (a) Is it useful to allow each normalization layer to select its own normalizer? (b) What impacts the choices of normalizers? (c) Do different tasks and datasets prefer different normalizers? We hope SN will help ease the usage and understand the normalization techniques in deep learning. The code of SN has been released at https://github.com/switchablenorms.

Bearing Fault Detection Using Scalogram and Switchable Normalization-Based CNN (SN-CNN)

Bearings play a vital role in all rotating machinery, and their failure is one of the significant causes of machine breakdown leading to a profound loss of safety and property. Therefore, the failure of rolling element bearings should be detected early while the machine fault is small. This paper presents the model that detects bearing failures using the continuous wavelet transform and classifies them using a switchable normalization-based convolutional neural network (SN-CNN). State-of-the-art accuracy was achieved with the proposed model using the Case Western Reserve University (CWRU) bearing dataset, which serves as the primary dataset for validating various algorithms for bearing failure detection. Batch normalization techniques were also employed and compared to the proposed model. The spectrogram images were also used as input for further comparison. Using switchable normalization, the proposed model achieved the testing accuracy in between 99.44% and 100% for different batch sizes and datasets.

Detection of panoramic vision pedestrian based on deep learning

As the AI (artificial intelligence) develops, driverless vehicle technology is widely concerned, and the important problem that needs to be solved in driverless technology is the detection of pedestrians in the panoramic vision of the vehicle, so the pedestrian detection technology of panoramic vision is explored based on the deep learning. Recently, anchor-free and one-stage detectors have been introduced into this area. However, their accuracies are unsatisfactory. Therefore, in order to enjoy the simplicity of anchor-free detectors and the accuracy of two-stage ones simultaneously, some adaptations based on a detector, CSP (center and scale prediction) are proposed. The original CSP of pedestrian detector is improved to make the training process more robust. For example, we use SN layers to replace all BN layers and ResNet-101 is used as backbone based on the research of deep learning. The main contributions of our paper are: (1) we improve the robustness of CSP and make it easier to train. (2) we propose a novel method to predict width, namely compressing width. (3) we achieve the second best performance on CityPersons benchmark, i.e. 9.3% log-average miss rate (MR) on reasonable set, 8.7% MR on partial set and 5.6% MR on bare set, which shows an anchor-free and one-stage detector can still have high accuracy. (4) we explore some capabilities of switchable normalization which are not mentioned in its original paper. This study will provide important theoretical support and practical basis for pedestrian detection

Automatic ischemic stroke lesion segmentation from computed tomography perfusion images by image synthesis and attention-based deep neural networks

Ischemic stroke lesion segmentation from Computed Tomography Perfusion (CTP) images is important for accurate diagnosis of stroke in acute care units. However, it is challenged by low image contrast and resolution of the perfusion parameter maps, in addition to the complex appearance of the lesion. To deal with this problem, we propose a novel framework based on synthesized pseudo Diffusion-Weighted Imaging (DWI) from perfusion parameter maps to obtain better image quality for more accurate segmentation. Our framework consists of three components based on Convolutional Neural Networks (CNNs) and is trained end-to-end. First, a feature extractor is used to obtain both a low-level and high-level compact representation of the raw spatiotemporal Computed Tomography Angiography (CTA) images. Second, a pseudo DWI generator takes as input the concatenation of CTP perfusion parameter maps and our extracted features to obtain the synthesized pseudo DWI. To achieve better synthesis quality, we propose a hybrid loss function that pays more attention to lesion regions and encourages high-level contextual consistency. Finally, we segment the lesion region from the synthesized pseudo DWI, where the segmentation network is based on switchable normalization and channel calibration for better performance. Experimental results showed that our framework achieved the top performance on ISLES 2018 challenge and: (1) our method using synthesized pseudo DWI outperformed methods segmenting the lesion from perfusion parameter maps directly; (2) the feature extractor exploiting additional spatiotemporal CTA images led to better synthesized pseudo DWI quality and higher segmentation accuracy; and (3) the proposed loss functions and network structure improved the pseudo DWI synthesis and lesion segmentation performance. The proposed framework has a potential for improving diagnosis and treatment of the ischemic stroke where access to real DWI scanning is limited.

SSN: Learning Sparse Switchable Normalization via SparsestMax

Normalization method deals with parameters training of convolution neural networks (CNNs) in which there are often multiple convolution layers. Despite the fact that layers in CNN are not homogeneous in the role they play at representing a prediction function, existing works often employ identical normalizer in different layers, making performance away from idealism. To tackle this problem and further boost performance, a recently-proposed switchable normalization (SN) provides a new perspective for deep learning: it learns to select different normalizers for different convolution layers of a ConvNet. However, SN uses softmax function to learn importance ratios to combine normalizers, not only leading to redundant computations compared to a single normalizer but also making model less interpretable. This work addresses this issue by presenting sparse switchable normalization (SSN) where the importance ratios are constrained to be sparse. Unlike $$\ell _1$$ and $$\ell _0$$ regularizations that impose difficulties in tuning layer-wise regularization coefficients, we turn this sparse-constrained optimization problem into feed-forward computation by proposing SparsestMax, which is a sparse version of softmax. SSN has several appealing properties. (1) It inherits all benefits from SN such as applicability in various tasks and robustness to a wide range of batch sizes. (2) It is guaranteed to select only one normalizer for each normalization layer, avoiding redundant computations and improving interpretability of normalizer selection. (3) SSN can be transferred to various tasks in an end-to-end manner. Extensive experiments show that SSN outperforms its counterparts on various challenging benchmarks such as ImageNet, COCO, Cityscapes, ADE20K, Kinetics and MegaFace. Models and code are available at https://github.com/switchablenorms/Sparse_SwitchNorm .

Parallel Structure Deep Neural Network Using CNN and RNN with an Attention Mechanism for Breast Cancer Histology Image Classification.

In this paper, we present a new deep learning model to classify hematoxylin–eosin-stained breast biopsy images into four classes (normal tissues, benign lesions, in situ carcinomas, and invasive carcinomas). Our model uses a parallel structure consist of a convolutional neural network (CNN) and a recurrent neural network (RNN) for image feature extraction, which is greatly different from the common existed serial method of extracting image features by CNN and then inputting them into RNN. Then, we introduce a special perceptron attention mechanism, which is derived from the natural language processing (NLP) field, to unify the features extracted by the two different neural network structures of the model. In the convolution layer, general batch normalization is replaced by the new switchable normalization method. And the latest regularization technology, targeted dropout, is used to substitute for the general dropout in the last three fully connected layers of the model. In the testing phase, we use the model fusion method and test time augmentation technology on three different datasets of hematoxylin–eosin-stained breast biopsy images. The results demonstrate that our model significantly outperforms state-of-the-art methods.

Bearing Fault Diagnosis Based on the Switchable Normalization SSGAN with 1-D Representation of Vibration Signals as Input.

The bearing is a component of the support shaft that guides the rotational movement of the shaft, widely used in the mechanical industry and also called a mechanical joint. In bearing fault diagnosis, the accuracy much depends on the feature extraction, which always needs a lot of training samples and classification in the commonly used methods. Neural networks are good at latent feature extraction and fault classification, however, they have problems with instability and over-fitting, and more labeled samples must be trained. Switchable normalization and semi-supervised learning are introduced to solve the above obstacles in this paper, which proposes a novel bearing fault diagnosis method based on switchable normalization semi-supervised generative adversarial networks (SN-SSGAN) with 1-dimensional representation of vibration signals as input. Experimental results showed that the proposed method has a desirable 99.93% classification accuracy in the case of less labeled data from the public data set of West Reserve University, which is better than the state-of-the-art methods.