Multi-scale Convolution Research Articles

Research on Performance Degradation Estimation of Key Components of High-Speed Train Bogie Based on Multi-Task Learning.

The safe and comfortable operation of high-speed trains has attracted extensive attention. With the operation of the train, the performance of high-speed train bogie components inevitably degrades and eventually leads to failures. At present, it is a common method to achieve performance degradation estimation of bogie components by processing high-speed train vibration signals and analyzing the information contained in the signals. In the face of complex signals, the usage of information theory, such as information entropy, to achieve performance degradation estimations is not satisfactory, and recent studies have more often used deep learning methods instead of traditional methods, such as information theory or signal processing, to obtain higher estimation accuracy. However, current research is more focused on the estimation for a certain component of the bogie and does not consider the bogie as a whole system to accomplish the performance degradation estimation task for several key components at the same time. In this paper, based on soft parameter sharing multi-task deep learning, a multi-task and multi-scale convolutional neural network is proposed to realize performance degradation state estimations of key components of a high-speed train bogie. Firstly, the structure takes into account the multi-scale characteristics of high-speed train vibration signals and uses a multi-scale convolution structure to better extract the key features of the signal. Secondly, considering that the vibration signal of high-speed trains contains the information of all components, the soft parameter sharing method is adopted to realize feature sharing in the depth structure and improve the utilization of information. The effectiveness and superiority of the structure proposed by the experiment is a feasible scheme for improving the performance degradation estimation of a high-speed train bogie.

Multibranch and Multiscale Dynamic Convolutional Network for Small Sample Fault Diagnosis of Rotating Machinery

Deep learning methods have been widely used in the field of fault diagnosis of rotating machinery. As one of the deep learning methods, multi-scale convolutional neural network (MSCNN) can achieve accurate fault diagnosis by extracting feature information of vibration signals at different scales. However, in practical applications, a small number of fault samples limit the performance of MSCNN. To overcome this problem, a small sample rotating machinery fault diagnosis method based on multi-branch and multi-scale dynamic convolutional network (MBSDCN) is proposed. Firstly, a feature splitting strategy is proposed, which splits a single input into multi-branch inputs to ensure that each multi-scale convolutional layer is matched to the appropriate input. Secondly, a channel reconstruction (CR) attention mechanism is designed, and CR attention mechanism can form a direct mapping of high-dimensional space with the fully connected layer through the reconstruction processing of channels, thus making the connection between feature channels closer. Finally, a novel multi-scale feature extraction model is constructed, in which multi-scale convolutional layers are applied to extract multi-branch input features, and the CR attention mechanism is used to calibrate the contribution of multi-scale convolution layers. The performance of MBSDCN is evaluated with CWRU rolling bearing dataset and UOC gearbox dataset and is compared with some advanced deep learning methods. The comparison results indicate that the proposed MBSDCN can accurately diagnose faults under small sample conditions. In addition, the experimental results in noisy environments indicate that the proposed MBSDCN has satisfactory anti-noise performance under small sample conditions and noisy environments.

Multiscale Convolutional Neural Network Based on Channel Space Attention for Gearbox Compound Fault Diagnosis.

Gearboxes are one of the most widely used speed and power transfer elements in rotating machinery. Highly accurate compound fault diagnosis of gearboxes is of great significance for the safe and reliable operation of rotating machinery systems. However, traditional compound fault diagnosis methods treat compound faults as an independent fault mode in the diagnosis process and cannot decouple them into multiple single faults. To address this problem, this paper proposes a gearbox compound fault diagnosis method. First, a multiscale convolutional neural network (MSCNN) is used as a feature learning model, which can effectively mine the compound fault information from vibration signals. Then, an improved hybrid attention module, named the channel-space attention module (CSAM), is proposed. It is embedded into the MSCNN to assign weights to multiscale features for enhancing the feature differentiation processing ability of the MSCNN. The new neural network is named CSAM-MSCNN. Finally, a multilabel classifier is used to output single or multiple labels for recognizing single or compound faults. The effectiveness of the method was verified with two gearbox datasets. The results show that the method possesses higher accuracy and stability than other models for gearbox compound fault diagnosis.

Research on Response Parameters and Classification Identification Method of Concrete Vibration Process.

The vibration process applied to fresh concrete is an important link in the construction process, but the lack of effective monitoring and evaluation methods results in the quality of the vibration process being difficult to control and, therefore, the structural quality of the resulting concrete structures difficult to guarantee. In this paper, according to the sensitivity of internal vibrators to vibration acceleration changes under different vibration media, the vibration signals of vibrators in air, concrete mixtures, and reinforced concrete mixtures were collected experimentally. Based on a deep learning algorithm for load recognition of rotating machinery, a multi-scale convolution neural network combined with a self-attention feature fusion mechanism (SE-MCNN) was proposed for medium attribute recognition of concrete vibrators. The model can accurately classify and identify vibrator vibration signals under different working conditions with a recognition accuracy of up to 97%. According to the classification results of the model, the continuous working times of vibrators in different media can be further statistically divided, which provides a new method for accurate quantitative evaluation of the quality of the concrete vibration process.

Adversarial Training Collaborating Multi-Path Context Feature Aggregation Network for Maize Disease Density Prediction

Maize is one of the world’s major food crops, and its yields are closely related to the sustenance of people. However, its cultivation is hampered by various diseases. Meanwhile, maize diseases are characterized by spots of varying and irregular shapes, which makes identifying them with current methods challenging. Therefore, we propose an adversarial training collaborating multi-path context feature aggregation network for maize disease density prediction. Specifically, our multi-scale patch-embedding module uses multi-scale convolution to extract feature maps of different sizes from maize images and performs a patch-embedding operation. Then, we adopt the multi-path context-feature aggregation module, which is divided into four paths to further extract detailed features and long-range information. As part of the aggregation module, the multi-scale feature-interaction operation will skillfully integrate rough and detailed features at the same feature level, thereby improving prediction accuracy. By adding noise interference to the input maize image, our adversarial training method can produce adversarial samples. These samples will interfere with the normal training of the network—thus improving its robustness. We tested our proposed method on the Plant Village dataset, which contains three types of diseased and healthy maize leaves. Our method achieved an average accuracy of 99.50%, surpassing seven mainstream models and showing its effectiveness in maize disease density prediction. This research has theoretical and applied significance for the intelligent and accurate detection of corn leaf diseases.

MulCNN: An efficient and accurate deep learning method based on gene embedding for cell type identification in single-cell RNA-seq data.

Advancements in single-cell sequencing research have revolutionized our understanding of cellular heterogeneity and functional diversity through the analysis of single-cell transcriptomes and genomes. A crucial step in single-cell RNA sequencing (scRNA-seq) analysis is identifying cell types. However, scRNA-seq data are often high dimensional and sparse, and manual cell type identification can be time-consuming, subjective, and lack reproducibility. Consequently, analyzing scRNA-seq data remains a computational challenge. With the increasing availability of well-annotated scRNA-seq datasets, advanced methods are emerging to aid in cell type identification by leveraging this information. Deep learning neural networks have great potential for analyzing single-cell data. This paper proposes MulCNN, a multi-level convolutional neural network that uses a unique cell type-specific gene expression feature extraction method. This method extracts critical features through multi-scale convolution while filtering noise. Extensive testing using datasets from various species and comparisons with popular classification methods show that MulCNN has outstanding performance and offers a new and scalable direction for scRNA-seq analysis.

GL-Segnet: Global-Local representation learning net for medical image segmentation.

Medical image segmentation has long been a compelling and fundamental problem in the realm of neuroscience. This is an extremely challenging task due to the intensely interfering irrelevant background information to segment the target. State-of-the-art methods fail to consider simultaneously addressing both long-range and short-range dependencies, and commonly emphasize the semantic information characterization capability while ignoring the geometric detail information implied in the shallow feature maps resulting in the dropping of crucial features. To tackle the above problem, we propose a Global-Local representation learning net for medical image segmentation, namely GL-Segnet. In the Feature encoder, we utilize the Multi-Scale Convolution (MSC) and Multi-Scale Pooling (MSP) modules to encode the global semantic representation information at the shallow level of the network, and multi-scale feature fusion operations are applied to enrich local geometric detail information in a cross-level manner. Beyond that, we adopt a global semantic feature extraction module to perform filtering of irrelevant background information. In Attention-enhancing Decoder, we use the Attention-based feature decoding module to refine the multi-scale fused feature information, which provides effective cues for attention decoding. We exploit the structural similarity between images and the edge gradient information to propose a hybrid loss to improve the segmentation accuracy of the model. Extensive experiments on medical image segmentation from Glas, ISIC, Brain Tumors and SIIM-ACR demonstrated that our GL-Segnet is superior to existing state-of-art methods in subjective visual performance and objective evaluation.

Multimodal Affective States Recognition Based on Multiscale CNNs and Biologically Inspired Decision Fusion Model

There has been an encouraging progress in the affective states recognition models based on the single-modality signals as electroencephalogram (EEG) signals or peripheral physiological signals in recent years. However, multimodal physiological signals-based affective states recognition methods have not been thoroughly exploited yet. Here we propose Multiscale Convolutional Neural Networks (Multiscale CNNs) and a biologically inspired decision fusion model for multimodal affective states recognition. Firstly, the raw signals are pre-processed with baseline signals. Then, the High Scale CNN and Low Scale CNN in Multiscale CNNs are utilized to predict the probability of affective states output for EEG and each peripheral physiological signal respectively. Finally, the fusion model calculates the reliability of each single-modality signals by the Euclidean distance between various class labels and the classification probability from Multiscale CNNs, and the decision is made by the more reliable modality information while other modalities information is retained. We use this model to classify four affective states from the arousal valence plane in the DEAP and AMIGOS dataset. The results show that the fusion model improves the accuracy of affective states recognition significantly compared with the result on single-modality signals, and the recognition accuracy of the fusion result achieve 98.52% and 99.89% in the DEAP and AMIGOS dataset respectively.

Shuffle Attention U-Net for Speech Enhancement in Time Domain

Over the past 10 years, deep learning has enabled significant advancements in the improvement of noisy speech. In an end-to-end speech enhancement, the deep neural networks transform a noisy speech signal to a clean speech signal in the time domain directly without any conversion or estimation of mask. Recently, the U-Net-based models achieved good enhancement performance. Despite this, some of them may neglect context-related information and detailed features of input speech in case of ordinary convolution. To address the above issues, recent studies have upgraded the performance of the model by adding various network modules such as attention mechanisms, long and short-term memory (LSTM). In this work, we propose a new U-Net-based speech enhancement model using a novel lightweight and efficient Shuffle Attention (SA), Gated Recurrent Unit (GRU), residual blocks with dilated convolutions. Residual block will be followed by a multi-scale convolution block (MSCB). The proposed hybrid structure enables the temporal context aggregation in time domain. The advantage of shuffle attention mechanism is that the channel and spatial attention are carried out simultaneously for each sub-feature in order to prevent potential noises while also highlighting the proper semantic feature areas by combining the same features from all locations. MSCB is employed for extracting rich temporal features. To represent the correlation between neighboring noisy speech frames, a two Layer GRU is added in the bottleneck of U-Net. The experimental findings demonstrate that the proposed model outperformed the other existing models in terms of short-time objective intelligibility (STOI), and perceptual evaluation of the speech quality (PESQ).

CenFind: a deep-learning pipeline for efficient centriole detection in microscopy datasets

BackgroundHigh-throughput and selective detection of organelles in immunofluorescence images is an important but demanding task in cell biology. The centriole organelle is critical for fundamental cellular processes, and its accurate detection is key for analysing centriole function in health and disease. Centriole detection in human tissue culture cells has been achieved typically by manual determination of organelle number per cell. However, manual cell scoring of centrioles has a low throughput and is not reproducible. Published semi-automated methods tally the centrosome surrounding centrioles and not centrioles themselves. Furthermore, such methods rely on hard-coded parameters or require a multichannel input for cross-correlation. Therefore, there is a need for developing an efficient and versatile pipeline for the automatic detection of centrioles in single channel immunofluorescence datasets.ResultsWe developed a deep-learning pipeline termed CenFind that automatically scores cells for centriole numbers in immunofluorescence images of human cells. CenFind relies on the multi-scale convolution neural network SpotNet, which allows the accurate detection of sparse and minute foci in high resolution images. We built a dataset using different experimental settings and used it to train the model and evaluate existing detection methods. The resulting average F1-score achieved by CenFind is > 90% across the test set, demonstrating the robustness of the pipeline. Moreover, using the StarDist-based nucleus detector, we link the centrioles and procentrioles detected with CenFind to the cell containing them, overall enabling automatic scoring of centriole numbers per cell.ConclusionsEfficient, accurate, channel-intrinsic and reproducible detection of centrioles is an important unmet need in the field. Existing methods are either not discriminative enough or focus on a fixed multi-channel input. To fill this methodological gap, we developed CenFind, a command line interface pipeline that automates cell scoring of centrioles, thereby enabling channel-intrinsic, accurate and reproducible detection across experimental modalities. Moreover, the modular nature of CenFind enables its integration in other pipelines. Overall, we anticipate CenFind to prove critical for accelerating discoveries in the field.

Federal learning edge network based sentiment analysis combating global COVID-19

As one of the important research topics in the field of natural language processing, sentiment analysis aims to analyze web data related to COVID-19, e.g., supporting China government agencies combating COVID-19. There are popular sentiment analysis models based on deep learning techniques, but their performance is limited by the size and distribution of the dataset. In this study, we propose a model based on a federal learning framework with Bert and multi-scale convolutional neural network (Fed_BERT_MSCNN), which contains a Bidirectional Encoder Representations from Transformer modules and a multi-scale convolution layer. The federal learning framework contains a central server and local deep learning machines that train local datasets. Parameter communications were processed through edge networks. The weighted average of each participant’s model parameters was communicated in the edge network for final utilization. The proposed federal network not only solves the problem of insufficient data, but also ensures the data privacy of the social platform during the training process and improve the communication efficiency. In the experiment, we used datasets of six social platforms, and used accuracy and F1-score as evaluation criteria to conduct comparative studies. The performance of the proposed Fed_BERT_MSCNN model was generally superior than the existing models in the literature.

A Hyperspectral Image Classification Method Based on the Nonlocal Attention Mechanism of a Multiscale Convolutional Neural Network.

Recently, convolution neural networks have been widely used in hyperspectral image classification and have achieved excellent performance. However, the fixed convolution kernel receptive field often leads to incomplete feature extraction, and the high redundancy of spectral information leads to difficulties in spectral feature extraction. To solve these problems, we propose a nonlocal attention mechanism of a 2D-3D hybrid CNN (2-3D-NL CNN), which includes an inception block and a nonlocal attention module. The inception block uses convolution kernels of different sizes to equip the network with multiscale receptive fields to extract the multiscale spatial features of ground objects. The nonlocal attention module enables the network to obtain a more comprehensive receptive field in the spatial and spectral dimensions while suppressing the information redundancy of the spectral dimension, making the extraction of spectral features easier. Experiments on two hyperspectral datasets, Pavia University and Salians, validate the effectiveness of the inception block and the nonlocal attention module. The results show that our model achieves an overall classification accuracy of 99.81% and 99.42% on the two datasets, respectively, which is higher than the accuracy of the existing model.

Efficient Re-Parameterization Residual Attention Network for Nonhomogeneous Image Dehazing

Real-world nonhomogeneous haze brings challenges to image restoration. More efforts are needed to remove dense haze and thin haze simultaneously and efficiently. However, most existing dehazing methods do not pay attention to the complex distributions of haze and usually suffer from a low runtime speed. To tackle such problems, we present an efficient re-parameterization residual attention network (RRA-Net), whose design has three key aspects. Firstly, we propose a training-time multi-branch residual attention block (MRAB), where multi-scale convolutions in different branches cope with the nonuniformity of haze and are converted into a single-path convolution during inference. It also features local residual learning with improved spatial attention and channel attention, allowing dense and thin haze to be attended to differently. Secondly, our lightweight network structure cascades six MRABs followed by a long skip connection with attention and a fusion tail. Overall, our RRA-Net only has about 0.3M parameters. Thirdly, two new loss functions, namely the Laplace pyramid loss and the color attenuation loss, help train the network to recover details and colors. The experimental results show that the proposed RRA-Net performs favorably against state-of-the-art dehazing methods on real-world image datasets, including both nonhomogeneous haze and dense homogeneous haze. A runtime comparison under the same hardware setup also demonstrates the superior efficiency of the proposed network.

Lung Disease Classification using Dense Alex Net Framework with Contrast Normalisation and Five-Fold Geometric Transformation

lung disease is one of the leading causes of death worldwide. Most cases of lung diseases are found when the disease is in an advanced stage. Therefore, the development of systems and methods that begin to diagnose quickly and prematurely plays a vital role in today's world. Currently, in detecting differences in lung cancer, an accurate diagnosis of cancer types is needed. However, improving the accuracy and reducing training time of the diagnosis remains a challenge. In this study, we have developed an automated classification scheme for lung cancer presented in histopathological images using a dense Alex Net framework. The proposed methodology carries out several phases includes pre-processing, contrast normalization, data augmentation and classification. Initially, the pre-processing step is accompanied to diminish the noisy contents present in the image. Contrast normalization has been explored to maintain the same illumination factor among histopathological lung images next to pre-processing. Afterwards, data augmentation phase has been carried out to enhance the dataset further to avoid over-fitting problems. Finally, the Dense Alex Net is utilized for classification that comprises five convolutional layers, one multi-scale convolution layer, and three fully connected layers. In evaluation experiments, the proposed approach was trained using our original database to provide rich and meaningful features. The accuracy attained by the proposed methodology is93%, which is maximum compared with the existing algorithms

Long and short spatial-temporal relations model for automatic generation of Labanotation

As a major research field in computer vision, automatic generation of Labanotation has been attracting the attention of many researchers. These researchers use various methods to engage in dance education and protection, but the existing methods do not consider the spatial-temporal dependence modeling of dance movements. Therefore, they do not represent complex dance movements optimally. We propose an automatic generation network of Labanotation based on the long and short spatial-temporal relations. It includes local spatial-temporal feature extraction network, global spatial-temporal feature extraction network, and local and global feature fusion network. It can align the input and output sequences while modeling the skeletal spatial-temporal relation. The local spatial-temporal features among short interval skeleton sequence relationship are obtained through the multi-scale convolution of time and space. The global spatial-temporal features are learned through the transformer network to obtain the relationship among skeleton sequences with long interval. For the output of the two networks, we use the pyramid squeeze attention network to exchange information in long and short spatial-temporal information, achieving complementarity to improve the accuracy of action recognition. Experimental results show that the proposed method outperforms the state-of-the-art methods on laban16 and laban48, which are common datasets for the research of Labanotation.

A Rolling Bearing Fault Diagnosis Method via 2D Feature Map of CSCoh After Denoising and MSCNN Under Different Conditions

The vibration signal of rolling bearing is interfered by the coupling of other various components and environment, which brings challenges to effective feature expression and the establishment of accurate predictor. A fault diagnosis method via 2D feature map of cyclic spectral coherence (CSCoh) after denoising and multi-scale convolutional neural network (MSCNN) under different conditions is proposed. The original vibration signal is processed by DTCWPT and threshold denoising, and a 2D feature map extraction model based on CSCoh is established to reflect hidden periodic characteristics. The feature differences can be highlighted and the coupling interference can be further eliminated. The multi-scale convolution kernels are proposed to build a parallel structure, and a fusion structure is followed. Benefit by the structures in MSCNN, the local and global features of 2D feature map can be preserved comprehensively, the learning ability and diagnostic performance can be improved. Finally, the verification experiments of rolling bearings in different positions under different operating conditions are carried out, and the experimental results show the proposed method has better generalization ability and comprehensive accuracy of more than 95% under different conditions.

Automated detection of sleep-arousal using multi-scale convolution and self-attention mechanism

In clinical, manually scoring by technician is the major method for sleep arousal detection. This method is time-consuming and subjective. This study aimed to achieve an end-to-end sleep-arousal events detection by constructing a convolutional neural network based on multi-scale convolutional layers and self-attention mechanism, and using 1 min single-channel electroencephalogram (EEG) signals as its input. Compared with the performance of the baseline model, the results of the proposed method showed that the mean area under the precision-recall curve and area under the receiver operating characteristic were both improved by 7%. Furthermore, we also compared the effects of single modality and multi-modality on the performance of the proposed model. The results revealed the power of single-channel EEG signals in automatic sleep arousal detection. However, the simple combination of multi-modality signals may be counterproductive to the improvement of model performance. Finally, we also explored the scalability of the proposed model and transferred the model into the automated sleep staging task in the same dataset. The average accuracy of 73% also suggested the power of the proposed method in task transferring. This study provides a potential solution for the development of portable sleep monitoring and paves a way for the automatic sleep data analysis using the transfer learning method.

SS-TMNet: Spatial–Spectral Transformer Network with Multi-Scale Convolution for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a significant foundation for remote sensing image analysis, widely used in biology, aerospace, and other applications. Convolution neural networks (CNNs) and attention mechanisms have shown outstanding ability in HSI classification and have been widely studied in recent years. However, the existing CNN-based and attention mechanism-based methods cannot fully use spatial–spectral information, which is not conducive to further improving HSI classification accuracy. This paper proposes a new spatial–spectral Transformer network with multi-scale convolution (SS-TMNet), which can effectively extract local and global spatial–spectral information. SS-TMNet includes two key modules, i.e., multi-scale 3D convolution projection module (MSCP) and spatial–spectral attention module (SSAM). The MSCP uses multi-scale 3D convolutions with different depths to extract the fused spatial–spectral features. The spatial–spectral attention module includes three branches: height spatial attention, width spatial attention, and spectral attention, which can extract the fusion information of spatial and spectral features. The proposed SS-TMNet was tested on three widely used HSI datasets: Pavia University, IndianPines, and Houston2013. The experimental results show that the proposed SS-TMNet is superior to the existing methods.

DBFN: Double Branch Fusion Network for Vital Components and Defect Detection of Transmission Line

AbstractA double branch fusion network is proposed based on unmanned aerial vehicle (UAV) inspection images to increase the detection accuracy of vital components and defects in transmission lines. The backbone feature extraction network comprises a combination of a convolutional neural network (CNN) and a Transformer network. To be specific, the CNN should extract local information, and the Transformer network is responsible for the extraction of global information. Besides, global information and local information have semantic differences, while resulting in feature aliasing after fusion. To solve this problem, a multiscale convolution module and a multiscale pooling module are proposed to solve semantic differences and feature aliasing through the interaction between two types of information. In general, the enhanced feature extraction network comprises a residual‐like convolution module, which can reduce the loss of detailed information (e.g., edge contours) and further extract high‐level semantic information from the deep network. Besides, it performs feature fusion in multiple regions in the enhanced feature extraction network, such that the multi‐scale adaptability of the neural network is effectively enhanced. Last, the fused feature information at different scales is decoded, and the final detection results are yielded.

MSRConvNet: Classification of railway track defects using multi-scale residual convolutional neural network

The development of an automated rail line defect classification system is of great benefit, as railway tracks must be periodically monitored and inspected to guarantee the safety of rail transportation. In this paper, an effective multi-scale residual convolutional network (MSRConvNet) model is proposed to classify the different types of railway track defects. The skip connections with residual learning blocks are used to increase the effectiveness of the network. The multi-scale convolutions are connected with parallel and two skip connections in the structure to distribute detailed feature maps with each other. Therefore, different scale feature maps can be extracted. The data augmentation method is performed to ensure a balanced class distribution and to eliminate the negative effect of the imbalanced dataset. The proposed model is compared with both benchmark deep learning models and the different variations of the designed network. The results verify that the proposed model can reach superior classification fulfillment, and the MSRConvNet provides an overall accuracy of 99.83%, precision of 99.83%, sensitivity of 99.83%, specificity of 99.94%, F1-score of 99.83%, and Matthew’s correlation coefficient of 99.78% for four defect classes.