Multi-scale Deep Convolutional Neural Network Research Articles

A Network Intrusion Detection Method Based on Deep Multi-scale Convolutional Neural Network

Network intrusion detection (NID) is an important method for network system administrators to detect various security holes. The performance of traditional NID methods can be affected when unknown or new attacks are detected. Compared with other machine learning methods, the intrusion detection method based on convolutional neural network (CNN) can significantly improve the accuracy of classification, but the convergence speed and generalization ability of CNN are not ideal in model training process resulting in a low true rate and a high false alarm rate. To solve the above problems, this paper proposes a deep multi-scale convolutional neural network (DMCNN) for network intrusion detection. Different levels of features in a large number of high-dimensional unlabeled original data are extracted by different scales convolution kernel. And the learning rate of network structure is optimized by batch normalization method to obtain the optimal feature representation of the raw data. We use NSL-KDD dataset as the benchmark thus we can compare the performance of our proposed method with other existing works. This dataset includes two testing sets which are the first one is KDDTest+ while the second one is $$\text {KDDTest}^{-21}$$ which is more difficult to be classified. The experimental results reveal that the AC and TPR are higher through our DMCNN model. Especially, in terms of DOS, the AC appropriately reaches to 98%. DMCNN has a high intrusion detection accuracy and a low false alarm rate, which overcomes the limitations of using the traditional detection methods and makes the new approach an attractive one for practical intrusion detection.

An Improved Topology Prediction of Alpha-Helical Transmembrane Protein Based on Deep Multi-Scale Convolutional Neural Network.

Alpha-helical proteins ( αTMPs) are essential in various biological processes. Despite their tertiary structures are crucial for revealing complex functions, experimental structure determination remains challenging and costly. In the past decades, various sequence-based topology prediction methods have been developed to bridge the gap between the sequences and structures by characterizing the structural features, but significant improvements are still required. Deep learning brings a great opportunity for its powerful representation learning capability from limited original data. In this work, we improved our αTMP topology prediction method DMCTOP using deep learning, which composed of two deep convolutional blocks to simultaneously extract local and global contextual features. Consequently, the inputs were simplified to reflect the original features of the sequence, including a protein sequence feature and an evolutionary conservation feature. DMCTOP can efficiently and accurately identify all topological types and the N-terminal orientation for an αTMP sequence. To validate the effectiveness of our method, we benchmarked DMCTOP against 13 peer methods according to the whole sequence, the transmembrane segment and the traditional criterion in testing experiments. All the results reveal that our method achieved the highest prediction accuracy and outperformed all the previous methods. The method is available at https://icdtools.nenu.edu.cn/dmctop.

CAST: A multi-scale convolutional neural network based automated hippocampal subfield segmentation toolbox

In this study, we developed a multi-scale Convolutional neural network based Automated hippocampal subfield Segmentation Toolbox (CAST) for automated segmentation of hippocampal subfields. Although training CAST required approximately three days on a single workstation with a high-quality GPU card, CAST can segment a new subject in less than 1 ​min even with GPU acceleration disabled, thus this method is more time efficient than current automated methods and manual segmentation. This toolbox is highly flexible with either a single modality or multiple modalities and can be easily set up to be trained with a researcher’s unique data. A 3D multi-scale deep convolutional neural network is the key algorithm used in the toolbox. The main merit of multi-scale images is the capability to capture more global structural information from down-sampled images without dramatically increasing memory and computational burden. The original images capture more local information to refine the boundary between subfields. Residual learning is applied to alleviate the vanishing gradient problem and improve the performance with a deeper network. We applied CAST with the same settings on two datasets, one 7T dataset (the UMC dataset) with only the T2 image and one 3T dataset (the MNI dataset) with both T1 and T2 images available. The segmentation accuracy of both CAST and the state-of-the-art automated method ASHS, in terms of the dice similarity coefficient (DSC), were comparable. CAST significantly improved the reliability of segmenting small subfields, such as CA2, CA3, and the entorhinal cortex (ERC), in terms of the intraclass correlation coefficient (ICC). Both ASHS and manual segmentation process some subfields (e.g. CA2 and ERC) with high DSC values but low ICC values, consequently increasing the difficulty of judging segmentation quality. CAST produces very consistent DSC and ICC values, with a maximal discrepancy of 0.01 (DSC-ICC) across all subfields. The pre-trained model, source code, and settings for the CAST toolbox are publicly available.

Deep multi-scale convolutional transfer learning network: A novel method for intelligent fault diagnosis of rolling bearings under variable working conditions and domains

Intelligent fault detection and diagnosis, as an important approach, play a crucial role in ensuring the stable, reliable and safe operation of rolling bearings, which is one of the most main components in the rotating machinery. However, the data distribution shift is inevitable in the practical scene due to changes in internal and external environments, it is still challenging to establish an effective fault diagnosis model that can eliminate the same distribution assumption. In light of the above demands, a novel transfer learning framework based on deep multi-scale convolutional neural network (MSCNN) is presented in this paper. First, a novel multi-scale module is ingenious established based on dilated convolution, which is used as the key part to obtain differential features through different perceptual fields. Then, in order to further reduce the complexity of the proposed model, a global average pooling technology is adopted to replace the traditional fully-connected layer. Finally, the architecture and weights of the MSCNN pre-trained on source domain are transferred to the other different but similar tasks with proper fine-tuning instead of training a network from scratch. The proposed MSCNN is evaluated by different transfer scenarios constructed on two famous rolling bearing test-bed. Three case studies show that the proposed framework not only has excellent performance on the source domain, but also has superior transferability on variable working conditions and domains.

A Novel System for Functional Determination of Variants of Uncertain Significance using Deep Convolutional Neural Networks

Many drugs are developed for commonly occurring, well studied cancer drivers such as vemurafenib for BRAF V600E and erlotinib for EGFR exon 19 mutations. However, most tumors also harbor mutations which have an uncertain role in disease formation, commonly called Variants of Uncertain Significance (VUS), which are not studied or characterized and could play a significant role in drug resistance and relapse. Therefore, the determination of the functional significance of VUS and their response to Molecularly Targeted Agents (MTA) is essential for developing new drugs and predicting response of patients. Here we present a multi-scale deep convolutional neural network (DCNN) architecture combined with an in-vitro functional assay to investigate the functional role of VUS and their response to MTA’s. Our method achieved high accuracy and precision on a hold-out set of examples (0.98 mean AUC for all tested genes) and was used to predict the oncogenicity of 195 VUS in 6 genes. 63 (32%) of the assayed VUS’s were classified as pathway activating, many of them to a similar extent as known driver mutations. Finally, we show that responses of various mutations to FDA approved MTAs are accurately predicted by our platform in a dose dependent manner. Taken together this novel system can uncover the treatable mutational landscape of a drug and be a useful tool in drug development.

Remaining useful life prediction using multi-scale deep convolutional neural network

Accurate and reliable remaining useful life (RUL) assessment result provides decision-makers valuable information to take suitable maintenance strategy to maximize the equipment usage and avoid costly failure. The conventional RUL prediction methods include model-based and data-driven. However, with the rapid development of modern industries, the physical model is becoming less capable of describing sophisticated systems, and the traditional data-driven methods have limited ability to learn sophisticated features. To overcome these problems, a multi-scale deep convolutional neural network (MS-DCNN) which have powerful feature extraction capability due to its multi-scale structure is proposed in this paper. This network constructs a direct relationship between Condition Monitoring (CM) data and ground-RUL without using any prior information. The MS-DCNN has three multi-scale blocks (MS-BLOCKs), where three different sizes of convolution operations are put on each block in parallel. This structure improves the network’s ability to learn complex features by extracting features of different scales. The developed algorithm includes three stages: data pre-processing, model training, and RUL prediction. After the min–max normalization pre-processing, the data is sent to the MS-DCNN network for parameter training directly, and the associated RUL value can be estimated base on the learned representations. Regularization helps to improve prediction accuracy and alleviate the overfitting problem. We evaluate the method on the available modular aero-propulsion system simulation data (C-MAPSS dataset) from NASA. The results show that the proposed method achieves good prognostics performance compared with other network architectures and state-of-the-art methods. RUL prediction result is obtained precisely without increasing the calculation burden.

DMCNN: A Deep Multiscale Convolutional Neural Network Model for Medical Image Segmentation

Medical image segmentation is one of the hot issues in the related area of image processing. Precise segmentation for medical images is a vital guarantee for follow-up treatment. At present, however, low gray contrast and blurred tissue boundaries are common in medical images, and the segmentation accuracy of medical images cannot be effectively improved. Especially, deep learning methods need more training samples, which lead to time-consuming process. Therefore, we propose a novelty model for medical image segmentation based on deep multiscale convolutional neural network (CNN) in this article. First, we extract the region of interest from the raw medical images. Then, data augmentation is operated to acquire more training datasets. Our proposed method contains three models: encoder, U-net, and decoder. Encoder is mainly responsible for feature extraction of 2D image slice. The U-net cascades the features of each block of the encoder with those obtained by deconvolution in the decoder under different scales. The decoding is mainly responsible for the upsampling of the feature graph after feature extraction of each group. Simulation results show that the new method can boost the segmentation accuracy. And, it has strong robustness compared with other segmentation methods.

Adversarial Adaptation From Synthesis to Reality in Fast Detector for Smoke Detection

Video smoke detection is a promising method for early fire prevention. However, it is still a challenging task for application of video smoke detection in real-world detection systems, as the limitations of smoke image samples for training and lack of efficient detection algorithm. This paper proposes a method based on two state-of-the-art fast detectors, a single-shot multi-box detector, and a multi-scale deep convolutional neural network, for smoke detection using synthetic smoke image samples. The virtual data can automatically offer rich samples with ground truth annotations. However, the learning of smoke representation in the detectors will be restricted by the appearance gap between real and synthetic smoke samples, which will cause a significant performance drop. To train a strong detector with synthetic smoke samples, we incorporate the domain adaptation into the fast detectors. A series of branches with the same structure as the detection branches are integrated into the fast detectors for domain adaptation. We design an adversarial training strategy to optimize the model of the adapted detectors, to learn a domain-invariant representation for smoke detection. The domain discrimination, domain confusion, and detection are combined in the iterative training procedure. The performance of the proposed approach surpasses the original baseline in our experiments.

Multi-scale deep context convolutional neural networks for semantic segmentation

Recent years have witnessed the great progress for semantic segmentation using deep convolutional neural networks (DCNNs). This paper presents a novel fully convolutional network for semantic segmentation using multi-scale contextual convolutional features. Since objects in natural images tend to be with various scales and aspect ratios, capturing the rich contextual information is very critical for dense pixel prediction. On the other hand, when going deeper in convolutional layers, the convolutional feature maps of traditional DCNNs gradually become coarser, which may be harmful for semantic segmentation. According to these observations, we attempt to design a multi-scale deep context convolutional network (MDCCNet), which combines the feature maps from different levels of network in a holistic manner for semantic segmentation. The segmentation outputs of MDCCNets are further enhanced using dense connected conditional random fields (CRF). The proposed network allows us to fully exploit local and global contextual information, ranging from an entire scene to every single pixel, to perform pixel-wise label estimation. The experimental results demonstrate that our method outperforms or is comparable to state-of-the-art methods on PASCAL VOC 2012 and SIFTFlow semantic segmentation datasets.

Single image depth estimation based on convolutional neural network and sparse connected conditional random field

Deep convolutional neural networks (DCNNs) have attracted significant interest in the computer vision community in the recent years and have exhibited high performance in resolving many computer vision problems, such as image classification. We address the pixel-level depth prediction from a single image by combining DCNN and sparse connected conditional random field (CRF). Owing to the invariance properties of DCNNs that make them suitable for high-level tasks, their outputs are generally not localized enough for detailed pixel-level regression. A multiscale DCNN and sparse connected CRF are combined to overcome this localization weakness. We have evaluated our framework using the well-known NYU V2 depth dataset, and the results show that the proposed method can improve the depth prediction accuracy both qualitatively and quantitatively, as compared to previous works. This finding shows the potential use of the proposed method in three-dimensional (3-D) modeling or 3-D video production from the given two-dimensional (2-D) images or 2-D videos.

Deep Quality: A Deep No-reference Quality Assessment System

Image quality assessment (IQA) continues to garner great interestin the research community, particularly given the tremendousrise in consumer video capture and streaming. Despite significantresearch effort in IQA in the past few decades, the area of noreferenceimage quality assessment remains a great challenge andis largely unsolved. In this paper, we propose a novel no-referenceimage quality assessment system called Deep Quality, which leveragesthe power of deep learning to model the complex relationshipbetween visual content and the perceived quality. Deep Qualityconsists of a novel multi-scale deep convolutional neural network,trained to learn to assess image quality based on training samplesconsisting of different distortions and degradations such as blur,Gaussian noise, and compression artifacts. Preliminary results usingthe CSIQ benchmark image quality dataset showed that DeepQuality was able to achieve strong quality prediction performance(89% patch-level and 98% image-level prediction accuracy), beingable to achieve similar performance as full-reference IQA methods.