Multi-scale Convolutional Neural Network Research Articles

A Hyperspectral Image Classification Method Based on the Nonlocal Attention Mechanism of a Multiscale Convolutional Neural Network.

Recently, convolution neural networks have been widely used in hyperspectral image classification and have achieved excellent performance. However, the fixed convolution kernel receptive field often leads to incomplete feature extraction, and the high redundancy of spectral information leads to difficulties in spectral feature extraction. To solve these problems, we propose a nonlocal attention mechanism of a 2D-3D hybrid CNN (2-3D-NL CNN), which includes an inception block and a nonlocal attention module. The inception block uses convolution kernels of different sizes to equip the network with multiscale receptive fields to extract the multiscale spatial features of ground objects. The nonlocal attention module enables the network to obtain a more comprehensive receptive field in the spatial and spectral dimensions while suppressing the information redundancy of the spectral dimension, making the extraction of spectral features easier. Experiments on two hyperspectral datasets, Pavia University and Salians, validate the effectiveness of the inception block and the nonlocal attention module. The results show that our model achieves an overall classification accuracy of 99.81% and 99.42% on the two datasets, respectively, which is higher than the accuracy of the existing model.

A Sound and Vibration Fusion Method for Fault Diagnosis of Rolling Bearings under Speed-Varying Conditions.

The fault diagnosis of rolling bearings is critical for the reliability assurance of mechanical systems. The operating speeds of the rolling bearings in industrial applications are usually time-varying, and the monitoring data available are difficult to cover all the speeds. Though deep learning techniques have been well developed, the generalization capacity under different working speeds is still challenging. In this paper, a sound and vibration fusion method, named the fusion multiscale convolutional neural network (F-MSCNN), was developed with strong adaptation performance under speed-varying conditions. The F-MSCNN works directly on raw sound and vibration signals. A fusion layer and a multiscale convolutional layer were added at the beginning of the model. With comprehensive information, such as the input, multiscale features are learned for subsequent classification. An experiment on the rolling bearing test bed was carried out, and six datasets under various working speeds were constructed. The results show that the proposed F-MSCNN can achieve high accuracy with stable performance when the speeds of the testing set are the same as or different from the training set. A comparison with other methods on the same datasets also proves the superiority of F-MSCNN in speed generalization. The diagnosis accuracy improves by sound and vibration fusion and multiscale feature learning.

Mosquito swarm counting via attention-based multi-scale convolutional neural network

Monitoring mosquito density to predict the risk of transmission of the virus and develop a response in advance is an important part of prevention efforts. This paper aims to estimate accurately the mosquito swarm count from a given image. To this end, we proposed an attention-based multi-scale mosquito swarm counting model that consists of the feature extraction network (FEN) and attention based multi-scale regression network (AMRN). The FEN uses VGG-16 network to extract low-level features of mosquitoes. The AMRN adopts a multi-scale convolutional neural network, and with a squeeze and excitation channel attention module in the branch with a 7 × 7 convolution kernel to extract high-level features, map the feature map to the mosquito swarm density map and estimate mosquitoes count. We collected and labelled a data set that includes 391 mosquito swarm images with 15,466 mosquitoes. Experiments show that our method performs well on the data set and achieves mean absolute error (MAE) of 1.810 and root mean square error (RMSE) of 3.467.

A Recurrent Attention Multi-Scale CNN–LSTM Network Based on Hyperspectral Image Classification

Since hyperspectral images contain a variety of ground objects of different scales, long-distance ground objects can fully extract the global spatial information of the image. However, most existing methods struggle to capture multi-scale information and global features simultaneously. Therefore, we combine two algorithms, MCNN and LSTM, and propose the MCNN–LSTM algorithm. The MCNN–LSTM model first performs multiple convolution operations on the image, and the result of each pooling layer is subjected to a feature fusion of the fully connected layer. Then, the results of fully connected layers at multiple scales and an attention mechanism are fused to alleviate the information redundancy of the network. Next, the results obtained by the fully connected layer are fed into the LSTM neural network, which enables the global information of the image to be captured more efficiently. In addition, to make the model meet the expected standard, a layer of loop control module is added to the fully connected layer of the LSTM network to share the weight information of multiple pieces of training. Finally, multiple public datasets are adopted for testing. The experimental results demonstrate that the proposed MCNN–LSTM model effectively extracts multi-scale features and global information of hyperspectral images, thus achieving higher classification accuracy.

NeuroPred-PLM: an interpretable and robust model for neuropeptide prediction by protein language model.

Neuropeptides are a diverse and complex class of signaling molecules that regulate a variety of biological processes. Neuropeptides provide many opportunities for the discovery of new drugs and targets for the treatment of a wide range of diseases, and thus, computational tools for the rapid and accurate large-scale identification of neuropeptides are of great significance for peptide research and drug development. Although several machine learning-based prediction tools have been developed, there is room for improvement in the performance and interpretability of the proposed methods. In this work, we developed an interpretable and robust neuropeptide prediction model, named NeuroPred-PLM. First, we employed a language model (ESM) of proteins to obtain semantic representations of neuropeptides, which could reduce the complexity of feature engineering. Next, we adopted a multi-scale convolutional neural network to enhance the local feature representation of neuropeptide embeddings. To make the model interpretable, we proposed a global multi-head attention network that could be used to capture the position-wise contribution to neuropeptide prediction via the attention scores. In addition, NeuroPred-PLM was developed based on our newly constructed NeuroPep 2.0 database. Benchmarks based on the independent test set show that NeuroPred-PLM achieves superior predictive performance compared with other state-of-the-art predictors. For the convenience of researchers, we provide an easy-to-install PyPi package (https://pypi.org/project/NeuroPredPLM/) and a web server (https://huggingface.co/spaces/isyslab/NeuroPred-PLM).

A Rolling Bearing Fault Diagnosis Method via 2D Feature Map of CSCoh After Denoising and MSCNN Under Different Conditions

The vibration signal of rolling bearing is interfered by the coupling of other various components and environment, which brings challenges to effective feature expression and the establishment of accurate predictor. A fault diagnosis method via 2D feature map of cyclic spectral coherence (CSCoh) after denoising and multi-scale convolutional neural network (MSCNN) under different conditions is proposed. The original vibration signal is processed by DTCWPT and threshold denoising, and a 2D feature map extraction model based on CSCoh is established to reflect hidden periodic characteristics. The feature differences can be highlighted and the coupling interference can be further eliminated. The multi-scale convolution kernels are proposed to build a parallel structure, and a fusion structure is followed. Benefit by the structures in MSCNN, the local and global features of 2D feature map can be preserved comprehensively, the learning ability and diagnostic performance can be improved. Finally, the verification experiments of rolling bearings in different positions under different operating conditions are carried out, and the experimental results show the proposed method has better generalization ability and comprehensive accuracy of more than 95% under different conditions.

Design of Deep Multi-Scale CNN for Scene Deblurring

Dynamic Scene deblurring is a difficult low-level vision problem induced by camera shaking, object motion, and other causes. Since the blur movements of moving objects and the backdrop in dynamic scenarios are different, segmentation of the motion blur is necessary for deblurring each blur motion precisely. In contrast to this limiting assumption, we tackle the issue of deblurring broad dynamic images with many moving objects and camera motion in the current study. A multilayer convolutional neural network (CNN) based video image deblurring technique is suggested as a solution to the issue that inter-frame information and spatiotemporal are easily lost during restoration of a digital video/image that has become blurred. We use the multiscale CNN-built adaptive Laplacian regularisation term to the problem of restoring images from videos. First, we provide a new restoration model by mixing several regularises, in particular by combining NLM regularise with the multiscale CNN, to extract redundant information from video images’ self-similarity. We utilised a basic gradient descent technique to solve the model for restoring images from videos. The experimental findings demonstrate the effective deblurring effect and some noise resilience of our approach.

Combining Multi-Scale Convolutional Neural Network with Long Short-Term Memory Neural Network for State of Charge Estimation of Lithium-ion Batteries

To develop safe and intelligent battery management systems for electric vehicles, it is necessary to accurately estimate the state of charge (SOC) of lithium-ion batteries. At present, deep learning methods have been broadly applied in the field of SOC estimation of lithium-ion batteries. However, existing deep SOC estimators are difficult to capture global trends due to being too sensitive to the changes of continuous time data points. In addition, a single non-linear neural network tends to ignore the linear features of the data, which makes the robustness of the estimation not good enough. To address these two issues, this paper proposes a SOC estimation method by combining multi-scale convolutional neural network with long short-term memory neural network (called MCNN-LSTM). Specifically, on the one hand, multiple one-dimension convolutions with different dilation rates are used to extract features at different time scales, and LSTM neural networks are used to acquire the long-term dependencies of the data. On the other hand, an additional fully connected layer is used to extract the linear features, which reduces the volatility of the estimation and enhances the robustness of the estimation. The results suggest that MCNN-LSTM has higher estimation accuracies and better robustness than DNN, LSTM, and CNN-LSTM.

A Weakly Supervised-Guided Soft Attention Network for Classification of Intracranial Hemorrhage

Intracranial hemorrhage (ICH) classification from computed tomography (CT) scans is important for accurate diagnosis of stroke in emergency centers. However, it is challenged by low image contrast and the complex appearance of the lesion. Therefore, the diagnosis results may vary from doctor to doctor under different situations. Existing auxiliary diagnosis algorithms treat the features of each location as equally important, and they do not impose any constraints on the learned features. The learned features used for classification in the model contain noise or features that have no diagnostic significance, which greatly limits the performance and reliability of systems. To facilitate the ICH treatment, we propose to use the auxiliary weak-segmentation-label supervision in multiscale CNN to classify ICH lesions on brain CT images, where the ICH lesions are segmented and used to guide the attention of the classification network. The weak segmentation labels we use are generated by an unsupervised algorithm and do not require additional annotations. We conduct experiments on the RSNA ICH detection data set, and the results demonstrate that our method can achieve an average ACC of 98.1% and an average <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F1$ </tex-math></inline-formula> score of 74.6% on the test set of nearly 80 000 CT images. The proposed weakly supervised-guided attention mechanism can accurately activate those neurons related to diagnosis, while inhibiting the activation of irrelevant areas and noise, thus achieving good performance.

MFAMNet: Multi-Scale Feature Attention Mixture Network for Short-Term Load Forecasting

Short-term load forecasting is an important prerequisite for smart grid controls. The current methods are mainly based on the convolution neural network (CNN) or long short-term memory (LSTM) model to realize load forecasting. For the multi-factor input sequence, the existing methods cannot obtain multi-scale features of the time series and the important parameters of the multi-factor, resulting in low accuracy and robustness. To address these problems, a multi-scale feature attention hybrid network is proposed, which uses LSTM to extract the time correlation of the sequence and multi-scale CNN to automatically extract the multi-scale feature of the load. This work realizes the integration of features by constructing a circular network. In the proposed model, a two-branch attention mechanism is further constructed to capture the important parameters of different influencing factors to improve the model’s robustness, which can make the network to obtain effective features at the curve changes. Comparative experiments on two open test sets show that the proposed multi-scale feature attention mixture network can achieve accurate short-term load forecasting and is superior to the existing methods.

DeepLBCEPred: A Bi-LSTM and multi-scale CNN-based deep learning method for predicting linear B-cell epitopes.

The epitope is the site where antigens and antibodies interact and is vital to understanding the immune system. Experimental identification of linear B-cell epitopes (BCEs) is expensive, is labor-consuming, and has a low throughput. Although a few computational methods have been proposed to address this challenge, there is still a long way to go for practical applications. We proposed a deep learning method called DeepLBCEPred for predicting linear BCEs, which consists of bi-directional long short-term memory (Bi-LSTM), feed-forward attention, and multi-scale convolutional neural networks (CNNs). We extensively tested the performance of DeepLBCEPred through cross-validation and independent tests on training and two testing datasets. The empirical results showed that the DeepLBCEPred obtained state-of-the-art performance. We also investigated the contribution of different deep learning elements to recognize linear BCEs. In addition, we have developed a user-friendly web application for linear BCEs prediction, which is freely available for all scientific researchers at: http://www.biolscience.cn/DeepLBCEPred/.

Determination of Shigella spp. via label-free SERS spectra coupled with deep learning

Accurate discrimination of Shigella spp. sits in the core of shigellosis prevention and control. As a label-free method, surface enhanced Raman spectroscopy (SERS) is being intensively investigated for bacterial diagnostics. In this study, we developed a novel method for rapid and accurate discrimination of Shigella spp. via label-free SERS coupling with multiscale deep-learning method. In particular, SERS spectral deconvolution was used to generate unique barcodes, revealing subtle differences in molecular composition between Shigella spp. Four supervised learning models based on Random Forest (RF), Support Vector Machine (SVM), Convolutional Neural Network (CNN), and One-Dimensional Multi-Scale CNN (1DMSCNN) were constructed and assessed for their predictive capacities of Shigella spp. The results showed that 1DMSCNN achieved the best performance, which could quickly distinguish four Shigella spp. accurately. Finally, we built a software embedded with 1DMSCNN model to predict raw SERS spectra of Shigella spp., which is freely available at https://github.com/4forfull/1DMSCNN_RAMAN_SHIGELLA.

Improved Text Matching Model Based on BERT

Text matching is a basic and important task in natural language understanding, this paper proposes a new model BBMC for the problem of insufficient feature extraction ability of existing text matching models, which integrates BiLSTM and multi-scale CNN on the basis of BERT. First, the word embedding representation of the text is obtained by the BERT, and then the semantic features of the text are further extracted by the double-layer BiLSTM, followed by the multi-scale CNN model, the key local features are extracted, and finally the linear and SoftMax function are used to classify. Experimental results on the LCQMC dataset show that the BBMC has been improved to a certain extent compared with other methods, and the accuracy on the test set can be best achieved 88.01%.

Remaining Useful Life Prediction of Milling Tool Based on Pyramid CNN

Remaining useful life prediction of a milling tool is one of the determinants in making scientific maintenance decision for the CNC machine tool. Predicting the RUL accurately can improve machining efficiency and the quality of product. Deep learning methods have strong learning capability in RUL prediction and are extensively used. Multiscale CNN, a typical deep learning model in RUL prediction, has a large number of parameters because of its parallel convolutional pathways, resulting in high computing cost. Besides, the MSCNN ignores various influences of different scales of degradation features on RUL prediction accuracy. To address the issue, a pyramid CNN (PCNN) is proposed for RUL prediction of the milling tool in this paper. Group convolution is used to replace parallel convolutional pathways to extract multiscale features without additional large number of parameters. And the channel attention with soft assignment is used to select the key degradation features, considering different sensors and scales. The milling tool wear experiments show that the score value of the proposed method achieved 51.248 ± 1.712 and the RMSE achieved 19.051 ± 0.804, confirming better performance of the proposed method compared with the traditional MSCNN and other deep learning methods. Besides, the number of parameters of the proposed method is reduced by 62.6% and 54.8% compared with the MSCNN with self-attention and the MSCNN methods, confirming its lower computing cost.

Text classification of Chinese news based on multi-scale CNN and LSTM hybrid model

Text classification of Chinese news based on multi-scale CNN and LSTM hybrid model

A self-attention multi-scale convolutional neural network method for SAR image despeckling

ABSTRACT The speckle noise found in synthetic aperture radar (SAR) images severely affects the efficiency of image interpretation, retrieval and other applications. Thus, effective methods for despeckling SAR image are required. The traditional methods for SAR image despeckling fail to balance in terms of the relationship between the intensity of speckle noise filtering and the retention of texture details. Deep learning based SAR image despeckling methods have been shown to have the potential to achieve this balance. Therefore, this study proposes a self-attention multi-scale convolution neural network (SAMSCNN) method for SAR image despeckling. The advantage of the SAMSCNN method is that it considers both multi-scale feature extraction and channel attention mechanisms for multi-scale fused features. In the SAMSCNN method, multi-scale features are extracted from SAR images through convolution layers with different depths. These are concatenated; then, and an attention mechanism is introduced to assign different weights to features of different scales, obtaining multi-scale fused features with weights. Finally, the despeckled SAR image is generated through global residual noise reduction and image structure fine-tuning. The despeckling experiments in this study involved a variety of scenes using simulated and real data. The performance of the proposed model was analysed using quantitative and qualitative evaluation methods and compared to probabilistic patch-based (PPB), SAR block-matching 3-D (SAR-BM3D) and SAR-CNN methods. The experimental results show that the method proposed in this paper improves the objective indexes and shows great advantages in visual effects compared to these classical methods. The method proposed in this study can provide key technical support for the practical application of SAR images.

FBMSNet: A Filter-Bank Multi-Scale Convolutional Neural Network for EEG-Based Motor Imagery Decoding.

Motor imagery (MI) is a mental process widely utilized as the experimental paradigm for brain-computer interfaces (BCIs) across a broad range of basic science and clinical studies. However, decoding intentions from MI remains challenging due to the inherent complexity of brain patterns relative to the small sample size available for machine learning. This paper proposes an end-to-end Filter-Bank Multiscale Convolutional Neural Network (FBMSNet) for MI classification. A filter bank is first employed to derive a multiview spectral representation of the EEG data. Mixed depthwise convolution is then applied to extract temporal features at multiple scales, followed by spatial filtering to mitigate volume conduction. Finally, with the joint supervision of cross-entropy and center loss, FBMSNet obtains features that maximize interclass dispersion and intraclass compactness. We compare FBMSNet with several state-of-the-art EEG decoding methods on two MI datasets: the BCI Competition IV 2a dataset and the OpenBMI dataset. FBMSNet significantly outperforms the benchmark methods by achieving 79.17% and 70.05% for four-class and two-class hold-out classification accuracy, respectively. These results demonstrate the efficacy of FBMSNet in improving EEG decoding performance toward more robust BCI applications. The FBMSNet source code is available at https://github.com/Want2Vanish/FBMSNet.

A Digital Twin Framework Embedded with POD and Neural Network for Flow Field Monitoring of Push-Plate Kiln

The push-plate kiln is a kind of kiln equipment widely used in the oxygen-free sintering of high-temperature alloy materials. Its flow field monitoring has an important application value for the manufacturing industry. However, traditional simulation methods cannot meet the requirements of real-time applications due to the high computational cost and being time-consuming. The rapid development of artificial intelligence technology will empower the traditional manufacturing industry. In this paper, we propose a data-driven digital twin framework for real-time flow field prediction by combining the CFD modeling simulation, IoT, and deep learning technologies. The framework integrates geometric, rule, physical, and neural network models to achieve the real-time simulation of physical and twin objects. The proper orthogonal decomposition (POD) and multiscale convolutional neural network (MCNN) are innovatively embedded into the framework. The POD is used to map high-dimensional data to low-dimensional features, and the MCNN is used to construct models predicting low-dimensional features for fast flow field prediction. The effectiveness of the proposed model is verified by the push-plate kiln case. The results show that the digital twin can quickly predict multi-physics fields based on the perceptual data to achieve the real-time evaluation of the operating state of the push-plate kiln.

A hybrid attention and time series network for enterprise sales forecasting under digital management and edge computing

Enterprises have both new opportunities and new challenges as a result of the rapid advancements in information technology that have accompanied the age of economic globalization. With the growth of internet of Things devices, data sizes have significantly increased. Further, the traditional cloud platform has been enriched with edge computing so that the huge data can be processed where it is collected. Therefore, businesses must adapt to new size requirements and rising standards for technical content. Forecasting corporate sales has emerged as a hot topic in the field of digital management. To successfully direct the future production and existence of enterprises, time series forecasting is of utmost importance and value. This is because it makes use of already-existing data to get the best predicting result. This work proposes a combination of enterprise sales forecasting from the perspective of digital management and neural networks, and proposes a network HATT-CNN-BiLSTM model for enterprise sales forecasting. First, this work combines multi-scale CNN (MSCNN) with improved BiLSTM (IBiLSTM) model. The MSCNN is utilized to extract spatial features with different scale, and it is often impossible to effectively explore the rules of time series features, and the processing of time series data is the strength of the LSTM network. Moreover, the IBiLSTM model can explore time series features in both directions, and therefore more useful information can be obtained. The MSCNN-IBiLSTM model, which is composed of MSCNN and IBiLSTM, can take advantage of strengths and avoid weaknesses, and give full play to the roles of the two models in different fields. Second, this work proposes a hybrid attention mechanism that combines self-attention, channel attention, and spatial attention. It enhances features extracted by MSCNN-IBiLSTM through a hybrid attention to build HATT-MSCNN-IBiLSTM network, which can extract more discriminative features. Third, this work conducts comprehensive and systematic experiments on HATT- MSCNN-IBiLSTM to verify feasibility of the proposed method. The proposed model is implemented over an edge computing platform that increases the model training speed and improve the response time.

MultiScale-CNN-4mCPred: a multi-scale CNN and adaptive embedding-based method for mouse genome DNA N4-methylcytosine prediction

N4-methylcytosine (4mC) is an important epigenetic mechanism, which regulates many cellular processes such as cell differentiation and gene expression. The knowledge about the 4mC sites is a key foundation to exploring its roles. Due to the limitation of techniques, precise detection of 4mC is still a challenging task. In this paper, we presented a multi-scale convolution neural network (CNN) and adaptive embedding-based computational method for predicting 4mC sites in mouse genome, which was referred to as MultiScale-CNN-4mCPred. The MultiScale-CNN-4mCPred used adaptive embedding to encode nucleotides, and then utilized multi-scale CNNs as well as long short-term memory to extract more in-depth local properties and contextual semantics in the sequences. The MultiScale-CNN-4mCPred is an end-to-end learning method, which requires no sophisticated feature design. The MultiScale-CNN-4mCPred reached an accuracy of 81.66% in the 10-fold cross-validation, and an accuracy of 84.69% in the independent test, outperforming state-of-the-art methods. We implemented the proposed method into a user-friendly web application which is freely available at: http://www.biolscience.cn/MultiScale-CNN-4mCPred/.