Multi-scale Deep Neural Network Research Articles

Solving a class of multi-scale elliptic PDEs by Fourier-based mixed physics informed neural networks

Deep neural networks have garnered widespread attention due to their simplicity and flexibility in the fields of engineering and scientific calculation. In this study, we probe into solving a class of elliptic partial differential equations (PDEs) with multiple scales by utilizing Fourier-based mixed physics informed neural networks (dubbed FMPINN), its solver is configured as a multi-scale deep neural network. In contrast to the classical PINN method, a dual (flux) variable about the rough coefficient of PDEs is introduced to avoid the ill-condition of neural tangent kernel matrix caused by the oscillating coefficient of multi-scale PDEs. Therefore, apart from the physical conservation laws, the discrepancy between the auxiliary variables and the gradients of multi-scale coefficients is incorporated into the cost function, obtaining a satisfactory solution of PDEs by minimizing the defined loss through some optimization methods. Additionally, a trigonometric activation function is introduced for FMPINN, which is suited for representing the derivatives of complex target functions. Handling the input data by Fourier feature mapping will effectively improve the capacity of deep neural networks to solve high-frequency problems. Finally, to validate the efficiency and robustness of the proposed FMPINN algorithm, we present several numerical examples of multi-scale problems in various dimensional Euclidean spaces. These examples cover low-frequency and high-frequency oscillation cases, demonstrating the effectiveness of our approach. All code and data accompanying this manuscript will be publicly available at https://github.com/Blue-Giant/FMPINN.

Remote sensing scene classification using multi-domain sematic high-order network

Recently, convolutional neural networks (CNNs), which obtain powerful deep features in an end-to-end manner, have achieved powerful performance in remote sensing scene classification. However, the average or maximum pooling operations defined in the spatial domain and coarser-resolution features with high levels cannot extract reliable features and clear boundaries for small-scale targets in remote sensing scene imagery. This paper attempts to address these problems and proposes a multi-domain sematic high-order network for scene classification, named MSHNet. First, wavelet-spatial and detachable pooling blocks defined in the wavelet and spatial domains are inserted at the end of the convolutional block to learn the features in a more structural fusion manner. Second, multi-scale and multi-resolution semantic embedding modules are proposed to take full advantage of the complementary information and effectively maintain the spatial structures of learned deep features. Third, we employ a factorized bilinear coding approach to obtain compact and discriminative second-order features. MSHNet is thoroughly evaluated on two publicly available benchmarks, i.e., AID (Aerial Image Dataset) and NWPU-RESISC45 (Northwestern Polytechnical University-Remote Sensing Image Scene Classification 45). The extensive results illustrate that our MSHNet is competitive with other related multi-scale deep neural networks.

DefCor-Net: Physics-aware ultrasound deformation correction.

The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis. This article presents a novel physics-aware deformation correction approach based on a coarse-to-fine, multi-scale deep neural network (DefCor-Net). To achieve pixel-wise performance, DefCor-Net incorporates biomedical knowledge by estimating pixel-wise stiffness online using a U-shaped feature extractor. The deformation field is then computed using polynomial regression by integrating the measured force applied by the US probe. Based on real-time estimation of pixel-by-pixel tissue properties, the learning-based approach enables the potential for anatomy-aware deformation correction. To demonstrate the effectiveness of the proposed DefCor-Net, images recorded at multiple locations on forearms and upper arms of six volunteers are used to train and validate DefCor-Net. The results demonstrate that DefCor-Net can significantly improve the accuracy of deformation correction to recover the original geometry (Dice Coefficient: from 14.3±20.9 to 82.6±12.1 when the force is 6N). Code:https://github.com/KarolineZhy/DefCorNet.

Integrating Pre-Trained protein language model and multiple window scanning deep learning networks for accurate identification of secondary active transporters in membrane proteins

Secondary active transporters play pivotal roles in regulating ion and molecule transport across cell membranes, with implications in diseases like cancer. However, studying transporters via biochemical experiments poses challenges. We propose an effective computational approach to identify secondary active transporters from membrane protein sequences using pre-trained language models and deep learning neural networks.Our dataset comprised 290 secondary active transporters and 5,420 other membrane proteins from UniProt. Three types of features were extracted - one-hot encodings, position-specific scoring matrix profiles, and contextual embeddings from the ProtTrans language model. A multi-window convolutional neural network architecture scanned the ProtTrans embeddings using varying window sizes to capture multi-scale sequence patterns.The proposed model combining ProtTrans embeddings and multi-window convolutional neural networks achieved 86% sensitivity, 99% specificity and 98% overall accuracy in identifying secondary active transporters, outperforming conventional machine learning approaches.This work demonstrates the promise of integrating pre-trained language models like ProtTrans with multi-scale deep neural networks to effectively interpret transporter sequences for functional analysis. Our approach enables more accurate computational identification of secondary active transporters, advancing membrane protein research.

Disruption prediction on EAST with different wall conditions based on a multi-scale deep hybrid neural network

Plasma disruption is a very dangerous event for future tokamaks and fusion reactors. Therefore, predicting disruption is crucial for ensuring the safety and performance of reactors. In this study, the features of two deep learning algorithms are integrated to establish a multi-scale hybrid network disruption predictor. Firstly, 43 diagnostic signals are extracted by a convolutional neural network (CNN), and the time information is learned by a long short-term memory network. The predictor is trained and tested on a database containing non-all-metal wall discharges. Its area under the receiver operator characteristic curve (AUC), which is a common performance metric for deep learning algorithms, reaches 0.97, and the true positive rate is , while the false positive rate is . Since EAST was upgraded in 2020, the wall condition has been upgraded from non-all-metal to all-metal. To examine the robustness of the predictor, the EAST disruption predictor is migrated to the all-metal wall experiment for the first time. It is again tested under the all-metal wall experimental data, and its warning performance decreases significantly, with an AUC of only 0.79. To improve the robustness and sensitivity of the predictor against disruptions, the convolutional attention mechanism is introduced into the CNN. After training and testing with the same data set, the warning performance for the all-metal wall data is improved, with the AUC value increasing to 0.84. To further improve the robustness of the predictor, the t-distributed stochastic neighbor embedding algorithm is employed to explore the difference of the sample distribution before and after EAST upgradation, and the transfer learning algorithm is used to reduce this difference. By applying transfer learning to a small amount of discharges from the all-metal wall data, the warning performance in the all-metal wall experiment is further improved, and the AUC value increases to 0.93. Overall, the proposed data-driven predictor is promising for applications in future devices such as ITER.

A Correction and Comments on “Multi-Scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains CiCP, 28(5):1970–2001,2020”

A Correction and Comments on “Multi-Scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains CiCP, 28(5):1970–2001,2020”

A Multi-View Multi-Scale Neural Network for Multi-Label ECG Classification

The 12-lead electrocardiogram (ECG) is a common method used to diagnose cardiovascular diseases. Recently, ECG classification using deep neural networks has been more accurate and efficient than traditional methods. Most ECG classification methods usually connect the 12-lead ECG into a matrix and then input this matrix into a deep neural network. We propose a multi-view and multi-scale deep neural network for ECG classification tasks considering different leads as different views, taking full advantage of the diversity of different lead features in a 12-lead ECG. The proposed network utilizes a multi-view approach to effectively fuse different lead features, and uses a multi-scale convolutional neural network structure to obtain the temporal features of an ECG at different scales. In addition, the spatial information and channel relationships of ECG features are captured by coordinate attention to enhance the feature representation of the network. Since our network contains six view networks, to reduce the size of the network, we also explore the distillation of dark knowledge from the multi-view network into a single-view network. Experimental results on multiple multi-label datasets show that our multi-view network outperforms existing state-of-the-art networks in multiple tasks.

Linearized Learning with Multiscale Deep Neural Networks for Stationary Navier-Stokes Equations with Oscillatory Solutions

Linearized Learning with Multiscale Deep Neural Networks for Stationary Navier-Stokes Equations with Oscillatory Solutions

Subspace decomposition based DNN algorithm for elliptic type multi-scale PDEs

While deep learning algorithms demonstrate a great potential in scientific computing, its application to multi-scale problems remains to be a big challenge. This is manifested by the “frequency principle” that neural networks tend to learn low frequency components first. Novel architectures such as multi-scale deep neural network (MscaleDNN) were proposed to alleviate this problem to some extent. In this paper, we construct a subspace decomposition based DNN (dubbed SD2NN) architecture for a class of multi-scale problems by combining MscaleDNN algorithms with traditional numerical analysis ideas. The proposed architecture includes one low frequency normal DNN submodule, and one (or a few) high frequency MscaleDNN submodule(s), which are designed to capture the smooth part and the oscillatory part of the multi-scale solutions simultaneously. We demonstrate that the SD2NN outperforms existing models such as MscaleDNN, through several benchmark multi-scale problems in regular or perforated domains.

Typhoon cloud image prediction based on enhanced multi-scale deep neural network

Typhoons threaten individuals’ lives and property. The accurate prediction of typhoon activity is crucial for reducing those threats and for risk assessment. Satellite images are widely used in typhoon research because of their wide coverage, timeliness, and relatively convenient acquisition. They are also important data sources for typhoon cloud image prediction. Studies on typhoon cloud image prediction have rarely used multi-scale features, which cause significant information loss and lead to fuzzy predictions with insufficient detail. Therefore, we developed an enhanced multi-scale deep neural network (EMSN) to predict a 3-hour-advance typhoon cloud image, which has two parts: a feature enhancement module and a feature encode-decode module. The inputs of the EMSN were eight consecutive images, and a feature enhancement module was applied to extract features from the historical inputs. To consider that the images of different time steps had different contributions to the output result, we used channel attention in this module to enhance important features. Because of the spatially correlated and spatially heterogeneous information at different scales, the feature encode-decode module used ConvLSTMs to capture spatiotemporal features at different scales. In addition, to reduce information loss during downsampling, skip connections were implemented to maintain more low-level information. To verify the effectiveness and applicability of our proposed EMSN, we compared various algorithms and explored the strengths and limitations of the model. The experimental results demonstrated that the EMSN efficiently and accurately predicted typhoon cloud images with higher quality than in the literature.

Multi-scale deep neural network for fault diagnosis method of rotating machinery

In recent years, deep learning technology has shown great potential in the fault diagnosis of rotating machinery based on vibration signals. However, the feature extraction and noise robustness still need to be improved. To this end, we propose a multi-scale deep neural network fault diagnosis method. Firstly, multi-scale down sampling of time-domain vibration signals. Next, the attention long short-term memory network and the fully convolutional neural network of the multi-scale convolution kernel are used for feature extraction. Then, a fusion module is utilized to fuze the extracted features. The proposed method is evaluated on the public bearing datasets. Experimental results demonstrate that the proposed method can achieve high accuracy and noise robustness.

Multi-Scale Deep Learning-based Recurrent Neural Network for Improved Medical Image Restoration and Enhancement

Multi-Scale Deep Learning-based Recurrent Neural Network for Improved Medical Image Restoration and Enhancement

Ensemble probabilistic wind power forecasting with multi-scale features

The uncertainty of wind power forecasting has a significant impact on the decision-making process of power system operators. Herein, to comprehensively characterize this uncertainty, a novel ensemble probabilistic forecasting model is developed in this study. First, the best model inputs were selected to forecast future wind power based on the value of maximum information coefficient, which measures both linear and nonlinear relationships between two random variables. Second, to extract sufficient and useful features from historical wind power data, a multi-scale feature extraction module based on average pooling, convolutional neural network, bidirectional long short-term memory, and attention mechanism was proposed. Third, five multi-scale deep density neural networks with different distribution-based loss functions and the proposed multi-scale feature extraction module were designed as base probabilistic forecasters to characterize the uncertainty from various aspects. Finally, an ensemble framework that combines five multi-scale deep density neural networks was built to determine the optimal ensemble weights for all base forecasters by simultaneously minimizing the pinball loss and continuous ranked probability score. The proposed ensemble model was implemented on four real-world wind power datasets. The results demonstrated the effectiveness of the proposed ensemble strategy and the designed multi-scale feature extraction module for probabilistic wind power forecasting.

Multisensory collaborative damage diagnosis of a 10 MW floating offshore wind turbine tendons using multi-scale convolutional neural network with attention mechanism

An effective damage diagnosis and prognostic management method can considerably reduce operation and maintenance costs of floating wind turbines. In this research, an intelligent damage diagnosis framework, named “MS-ACNN”, has been developed using a multi-scale deep convolution neural network model fused with an attention mechanism. The framework is used to detect, localize, and quantify existing and potential damages on multibody floating wind turbine tendons. The MS-ACNN framework is fitted with two multi-scale extractors, designed to capture multi-scale information from raw wind turbine response signals measured using multi-sensor. The attention mechanism uses weight ratios of extracted damage feature to enhance the MS-ACNN's capability in offering a better generalization in damage diagnosis. The framework's performance is examined under normal and noisy environments and with a diagnosis accuracy of 80%, which is higher than those obtained using most generic industrial grade diagnostic tools (MS–CNN–I, MSCNN-II, CNN, CNN-LSTM and CNN-BiLSTM) by at least 10%. The framework is also fitted with a Majority Weighted Voting rule to reduce false alarms and ensure optimum performance of the multi-sensor during collaborative diagnosis. Further examination shows that the inclusion of a voting rule increases the diagnostic performance's F1 index from 90% for single sensor and 84% for multi-sensor results to 94%.

MSRCall: a multi-scale deep neural network to basecall Oxford Nanopore sequences.

MinION, a third-generation sequencer from Oxford Nanopore Technologies, is a portable device that can provide long-nucleotide read data in real-time. It primarily aims to deduce the makeup of nucleotide sequences from the ionic current signals generated when passing DNA/RNA fragments through nanopores charged with a voltage difference. To determine nucleotides from measured signals, a translation process known as basecalling is required. However, compared to NGS basecallers, the calling accuracy of MinION still needs to be improved. In this work, a simple but powerful neural network architecture called multi-scale recurrent caller (MSRCall) is proposed. MSRCall comprises a multi-scale structure, recurrent layers, a fusion block and a connectionist temporal classification decoder. To better identify both short-and long-range dependencies, the recurrent layer is redesigned to capture various time-scale features with a multi-scale structure. The results show that MSRCall outperforms other basecallers in terms of both read and consensus accuracies. MSRCall is available at: https://github.com/d05943006/MSRCall. Supplementary data are available at Bioinformatics online.

Multi-Scale Deep Neural Network Based on Dilated Convolution for Spacecraft Image Segmentation.

In recent years, image segmentation techniques based on deep learning have achieved many applications in remote sensing, medical, and autonomous driving fields. In space exploration, the segmentation of spacecraft objects by monocular images can support space station on-orbit assembly tasks and space target position and attitude estimation tasks, which has essential research value and broad application prospects. However, there is no segmentation network designed for spacecraft targets. This paper proposes an end-to-end spacecraft image segmentation network using the semantic segmentation network DeepLabv3+ as the basic framework. We develop a multi-scale neural network based on sparse convolution. First, the feature extraction capability is improved by the dilated convolutional network. Second, we introduce the channel attention mechanism into the network to recalibrate the feature responses. Finally, we design a parallel atrous spatial pyramid pooling (ASPP) structure that enhances the contextual information of the network. To verify the effectiveness of the method, we built a spacecraft segmentation dataset on which we conduct experiments on the segmentation algorithm. The experimental results show that the encoder+ attention+ decoder structure proposed in this paper, which focuses on high-level and low-level features, can obtain clear and complete masks of spacecraft targets with high segmentation accuracy. Compared with DeepLabv3+, our method is a significant improvement. We also conduct an ablation study to research the effectiveness of our network framework.

Prediction of seizure recurrence using electroencephalogram analysis with multiscale deep neural networks before withdrawal of antiepileptic drugs.

The decision to continue or discontinue antiepileptic drug (AED) treatment in patients who are seizure free for a prolonged time is critical. Studies have used certain risk factors or electroencephalogram (EEG) findings to predict seizure recurrence after the withdrawal of AEDs. However, applicable biomarkers to guide the withdrawal of AEDs are lacking. In this study, we used EEG analysis based on multiscale deep neural networks (MSDNN) to establish a method for predicting seizure recurrence after the withdrawal of AEDs. A total of 60 patients with epilepsy were divided into two groups (30 in the recurrence group and 30 in the non-recurrence group). All patients were seizure free for at least 2 years. Before AED withdrawal, an EEG was performed for each patient, which showed no epileptiform discharges. These EEG recordings were classified using MSDNN. We found that the performance indices of classification between recurrence and non-recurrence groups had a mean sensitivity, mean specificity, mean accuracy, and mean area under the receiver operating characteristic curve of 74.23%, 75.83%, 74.66%, and 82.66%, respectively. Our proposed method is a promising tool to help physicians to predict seizure recurrence after AED withdrawal among seizure-free patients.

Multi-scale deep neural network approach with attention mechanism for remaining useful life estimation

Prognostics and Health Management (PHM) is the core task in modern industries to provide the reliability and availability of mechanical systems. In recent years, the degradation behaviors have been extensively employed to estimate the remaining useful life in PMH technologies. In this research, a novel data-driven framework based on the multi-scale network structure, called MCA-BGRU, is proposed to provide the remaining useful life prediction, which combines multi-scale convolution neural network (CNN), bidirectional gated recurrent unit (BGRU), multi-head self-attention (MHSA) mechanism, and fully-connected layers. In this proposed structure, multi-scale CNN blocks and the MHSA mechanism are constructed to capture high-level representations from the multivariate input data automatically. Then, a BGRU layer is leveraged to learn various temporal tendencies between extracted features. Additionally, particle swarm optimization is adopted to simultaneously tune the hyperparameters of this framework. The superiority of the MCA-BGRU is validated by the well-known C-MAPSS dataset of NASA. The experimental results revealed that the presented approach achieves an improvement of 0.32% and 5.6% in terms of RMSE and Score values compared with the various existing studies.

Subspace Decomposition Based Dnn Algorithm for Elliptic-Type Multi-Scale Pdes

While deep learning algorithms demonstrate a great potential in scientific computing, its application to multi-scale problems remains to be a big challenge. This is manifested by the "frequency principle" that neural networks tend to learn low frequency components first. Novel architectures such as multi-scale deep neural network (MscaleDNN) were proposed to alleviate this problem to some extent. In this paper, we construct a subspace decomposition based DNN (dubbed SD$^2$NN) architecture for a class of multi-scale problems by combining traditional numerical analysis ideas and MscaleDNN algorithms. The proposed architecture includes one low frequency normal DNN submodule, and one (or a few) high frequency MscaleDNN submodule(s), which are designed to capture the smooth part and the oscillatory part of the multi-scale solutions, respectively. In addition, a novel trigonometric activation function is incorporated in the SD$^2$NN model. We demonstrate the performance of the SD$^2$NN architecture through several benchmark multi-scale problems in regular or irregular geometric domains. Numerical results show that the SD$^2$NN model is superior to existing models such as MscaleDNN.

Linearized Learning with Multiscale Deep Neural Networks for Stationary Navier-Stokes Equations with Oscillatory Solutions

Linearized Learning with Multiscale Deep Neural Networks for Stationary Navier-Stokes Equations with Oscillatory Solutions