Multi-scale Convolution Module Research Articles

EDASNet: Efficient Dynamic Adaptive-scale Network for Infrared Pedestrian Detection

Abstract Infrared images are widely utilized due to their exceptional anti-interference capabilities. However, challenges such as low resolution and an absence of detailed texture can impede the effective recognition of multi-scale target information, particularly for small targets. To address these issues, we introduce a multi-scale detection framework named Efficient Dynamic Adaptive-scale Network (EDASNet), which focuses on enhancing the feature extraction of small objects while ensuring efficient detection of multi-scale. Firstly, we design a Lightweight dynamic enhance network (LDENet) as the backbone for feature extraction. It mainly includes a Lightweight adaptive-weight down sampling module and a dynamic enhancement convolution module. In addition, a multi-scale aggregation feature pyramid network (MSA-FPN) is proposed, which improves the perception effect of small objects through a multi-scale convolution module. Then, the Repulsion Loss term was introduced based on CIOU to effectively solve the missed detection problem caused by target overlap. Finally, the dynamic head (DyHead) was used as the network detection head, and through the superposition of dynamic convolution and multiple attention, the network was able to accurately realize multi-scale object detection. Comprehensive experiments show that EDASNet outperforms existing efficient models and achieves a good trade-off between speed and accuracy.

Side-channel attacks based on attention mechanism and multi-scale convolutional neural network

One widely utilized attack technique for side-channel attacks is deep learning. However, the efficiency of deep learning attacks can be impacted by the differences in data collected from various devices or environmental conditions. These differences arise from factors such as hardware discrepancies, environmental noise, and countermeasures. Such differences can disrupt the correlation between the leaked information from the target device during operations and sensitive data, leading to failures in the attacks. In order to better extract the feature information and temporal information of traces in this challenge, we propose an Attention Mechanism and Multi-Scale Convolutional Neural Network (AMCNNet). The critical components of the AMCNNet are the multi-scale convolution module and the feature extraction module. The multi-scale convolution module extracts feature of different scales from traces, improving the network's ability to capture high-level meanings and details. Then, the feature extraction module dynamically weights and rescales the features to better captures the long-term dependencies in the traces.. Through experiments conducted on publicly available datasets such as DPA Contest v4, ASCAD, and AES_HD, as well as analyzing the guessing entropy, the experimental results demonstrate that the network exhibits strong generalization capability and robustness across datasets with diverse features.

Plane-wave medical image reconstruction based on dynamic Criss-Cross attention and multi-scale convolution.

Plane-wave imaging is widely employed in medical imaging due to its ultra-fast imaging speed. However, the image quality is compromised. Existing techniques to enhance image quality tend to sacrifice the imaging frame rate. The study aims to reconstruct high-quality plane-wave images while maintaining the imaging frame rate. The proposed method utilizes a U-Net-based generator incorporating a multi-scale convolution module in the encoder to extract information at different levels. Additionally, a Dynamic Criss-Cross Attention (DCCA) mechanism is proposed in the decoder of the U-Net-based generator to extract both local and global features of plane-wave images while avoiding interference caused by irrelevant regions. In the reconstruction of point targets, the experimental images achieved a reduction in Full Width at Half Maximum (FWHM) of 0.0499 mm, compared to the Coherent Plane-Wave Compounding (CPWC) method using 75-beam plane waves. For the reconstruction of cyst targets, the simulated image achieved a 3.78% improvement in Contrast Ratio (CR) compared to CPWC. The proposed model effectively addresses the issue of unclear lesion sites in plane-wave images.

MSTSENet: Multiscale Spectral–Spatial Transformer with Squeeze and Excitation network for hyperspectral image classification

Hyperspectral image (HSI) classification pertains to the task of assigning a single label to each pixel by analyzing its spectral–spatial characteristics. Convolutional Neural Networks (CNNs) have garnered significant attention on account of their remarkable performance in feature representation. However, these approaches possess a restricted capacity to acquire highly similar spectral features. Recently, Attention-based approaches have been devised as a means to surmount the limitations inherent in CNNs. In this paper, we address a novel architecture for spectral–spatial feature extraction and HSI classification. The framework efficiently amalgamates the robustness and efficacy of three modern backbone networks, i.e., Multiscale CNN, Squeeze and Excitation (SE), and Transformer named MSTSENet. Initially, the multiscale convolution module integrates three branches of 3D convolution layers, each employing distinct kernel sizes, facilitating multiscale and multi-resolution feature extraction. Subsequently, the SE module enhances the inter-channel relationship by adaptively recalibrating the feature weight. The strategic fusion of the SE module with multiscale CNN strengthens deep feature extraction while introducing a minimal increment in overall parameters. Lastly, the Transformer module orchestrates information aggregation across different spectral–spatial regions, accurately modeling long-range contextual dependencies and capturing dominant feature representations. To evaluate the model performance, comprehensive experiments have been carried out on widely used benchmark HSI datasets. The overall accuracy of 98.74%, 99.83%, and 99.98% is achieved over the Indian Pines, Pavia University, and Salinas Valley datasets, respectively. The code of this work will be accessible at https://github.com/irfan01000/MSTSENet.

Multiscale Fusion Convolutional Network in Real-time Semantic Segmentation

To achieve semantic segmentation tasks in practical applications such as autonomous driving, networks need to efficiently process high-resolution images while maintaining high accuracy. This requires methods to effectively fuse spatial information in high-resolution images with semantic information in low-resolution images. To address this, this paper proposes a Multi-scale Fusion Convolutional Network (MFCNet) based on a single-branch network structure. In order to simultaneously handle information at different scales and assist the network in capturing a wide range of contextual information, separable Multi-Scale Convolution Modules (MSCM) are introduced to enable the network to obtain richer and more comprehensive feature representations. Additionally, considering that shallow-level information is difficult to directly restore resolution, a Dual-Attention Fusion Module (DAFM) is designed, introducing two attention mechanisms to respectively weight feature maps at different resolutions. Experimental results demonstrate that MFCNet achieves outstanding performance in real-time semantic segmentation tasks.

Multi-resolution sequence and structure feature extraction for binding site prediction

Circular ribonucleic acids (circRNAs) are single-stranded RNA molecules that form loops and are widely expressed in various cells and tissues. They interact with RNA-binding proteins (RBPs) and play a vital regulatory role in the onset and development of several diseases. Researchers have proposed various hybrid architecture prediction methods based on convolutional neural networks and recurrent neural networks to recognize the interactions and sites between circRNAs and RBPs and thus reveal the biological functions of circRNAs. However, existing methods usually ignore the structural information of circRNA, which may affect the modeling of circRNA and RBP binding modes. To address these problems, we propose a prediction model based on multi-resolution feature extraction. First, it generates sequence features using unsupervised word embedding and nucleotide density. Then, it uses implicit and explicit pseudo-secondary structure hybrid encoding to fuse sequence and structural information and better simulate circRNA-RBP binding patterns. Second, it uses an enhanced bidirectional sample convolution and interaction network encoder to capture and integrate high-order features of distinct resolutions from the multi-scale convolution module. This provides rich semantic input to the downstream bidirectional long short-term memory network to improve prediction accuracy. Experimental results on 37 circRNA and 31 linear RNA datasets show that our method has significant advantages in identifying RNA-RBP interactions. Furthermore, the four motifs learned by our method are verified against existing motif databases, indicating that it can discover biologically meaningful circRNA-RBP binding patterns.

Enhanced multi-scale networks for semantic segmentation

Multi-scale representation provides an effective answer to the scale variation of objects and entities in semantic segmentation. The ability to capture long-range pixel dependency facilitates semantic segmentation. In addition, semantic segmentation necessitates the effective use of pixel-to-pixel similarity in the channel direction to enhance pixel areas. By reviewing the characteristics of earlier successful segmentation models, we discover a number of crucial elements that enhance segmentation model performance, including a robust encoder structure, multi-scale interactions, attention mechanisms, and a robust decoder structure. The attention mechanism of the asymmetric non-local neural network (ANNet) is merged with multi-scale pyramidal modules to accelerate model segmentation while maintaining high accuracy. However, ANNet does not account for the similarity between pixels in the feature map channel direction, making the segmentation accuracy unsatisfactory. As a result, we propose EMSNet, a straightforward convolutional network architecture for semantic segmentation that consists of Integration of enhanced regional module (IERM) and Multi-scale convolution module (MSCM). The IERM module generates weights using four or five-stage feature maps, then fuses the input features with the weights and uses more computation. The similarity of the channel direction feature graphs is also calculated using ANNet’s auxiliary loss function. The MSCM module can more accurately describe the interactions between various channels, capture the interdependencies between feature pixels, and capture the multi-scale context. Experiments prove that we perform well in tests using the benchmark dataset. On Cityscapes test data, we get 82.2% segmentation accuracy. The mIoU in the ADE20k and Pascal VOC datasets are, respectively, 45.58% and 85.46%.

Electroencephalography recognition based on encephalic region and temporal sequence transformer

Stereoscopic vision is the key to good motor control and accurate cognition, and its formation is closely related to brain control. The early methods of measuring stereoscopic vision rely on the subject’s judgment, which might be influenced by inadvertent misjudgments. To solve this problem, we collected the Electroencephalography (EEG) of subjects watching dynamic random dot stereogram for stereogram recognition. To analyze stereogram EEG signals, this paper proposed a transformer-based encephalic region temporal sequence analysis network. Inspired by the concept of brain regions, this network designs an encephalic region Transformer module to capture global spatial features in each brain region and among the whole brain regions. Based on the spatial features of electrodes in different brain regions, the global spatial dependence of all electrodes can be further obtained. Then, the temporal sequence Transformer module is adopted to learn the global temporal EEG features. Finally, we utilize the spatial-temporal multi-scale convolution module to extract advanced spatial and temporal fusion features for recognition. The simulation results on two public EEG datasets illustrate the excellent classification performance of the proposed model, which is better than 9 existing comparison models in EEG recognition.

PMCT: Parallel Multiscale Convolutional Temporal model for MOOC dropout prediction

High dropout rates are a major challenge for Massive Open Online Courses (MOOCs), making dropout prediction a crucial task. Existing research often develops various prediction models based on student behavioral features. However, most of the existing methods rely on single-scale convolution operations, which can only extract single time scale information. This approach has limited ability in extracting multi-scale features and global information of student behavior, and cannot take into account student groups with different behavior patterns at the same time. To solve this limitation, we propose the Parallel Multiscale Convolutional Temporal model (PMCT) for modeling and predicting student dropout behavior. Specifically, our model uses a Parallel Multiscale Convolution module (PMC) to extract multiscale behavioral features in parallel based on the different features displayed by students at different time scales. Additionally, we employ a stacked residual block of Temporal Convolutional Network (TCN) to extract the potential correlation between scales from the behavior features and as a supplement to the timing information. Experimental results on two large datasets demonstrate that our proposed model outperforms the baseline methods in terms of predictive performance.

Short‐term origin‐destination demand forecasting in bus rapid transit based on dual attentive multi‐scale convolutional network

AbstractThe short‐term origin‐destination (OD) forecast in bus rapid transit (BRT) can provide future OD demand, which means that transport plan personnel can take measures in advance, such as adjusting the departure schedule, to avoid congestion. This emphasizes the importance of accurate and stable OD forecasting in real‐time. Deep learning methods have been widely applied due to their superior performance in capturing dynamic spatiotemporal and nonlinear features. However, current methods based on deep learning and data‐driven cannot simultaneously address the problems of data availability and sparsity, and their ability to capture dynamic spatiotemporal patterns is insufficient, which cannot provide satisfactory accuracy and stability. To address these problems, this paper proposes a method for forecasting the OD demand in BRT based on dual attention multi‐scale convolutional network (DAMSCN). Considering the three temporal properties of OD, that is, closeness, period and trend dependencies. DAMSCN uses multi‐scale convolution (MSC), dual attention mechanism (DAt) and stacked convolutional long short‐term memory networks (ConvLSTM) to model the three kinds of properties. The MSC module converts sparse OD information into dense useful features to solve the problem of data sparsity. The DAt and ConvLSTM module model the dependencies relationship in the spatial and temporal dimensions, which can improve the spatiotemporal features extraction ability. DAMSCN also combine inflow information to enhance data availability. The three properties are dynamically aggregated and further integrated with external influences (e.g. weather conditions etc.) to obtain the final forecasting result. The accuracy and stability are evaluated on the Xiamen BRT (XMBRT) and Shanghai Metro datasets (SHMetro). The experimental results show that the DAMSCN is superior to other method. Compared to the best baseline, on the XMBRT, MAE decreased by 8.60%, and RMSE decreased by 4.81%. On the SHMetro, MAE decreased by 7.22%, and RMSE decreased by 8.96%.

Automatic Assessment of Depression and Anxiety through Encoding Pupil-wave from HCI in VR Scenes

At present, there have been many studies on the methods of using the deep learning regression model to assess depression level based on behavioral signals (facial expression, speech, and language); however, the research on the assessment method of anxiety level using deep learning is absent. In this article, pupil-wave, a physiological signal collected by Human Computer Interaction (HCI) that can directly represent the emotional state, is developed to assess the level of depression and anxiety for the first time. In order to distinguish between different depression and anxiety levels, we use the HCI method to induce the participants’ emotional experience through three virtual reality (VR) emotional scenes of joyful, sad, and calm, and construct two differential pupil-waves of joyful and sad with the calm pupil-wave as the baseline. Correspondingly, a dual-channel fusion depression and anxiety level assessment model is constructed using the improved multi-scale convolution module and our proposed width-channel attention module for one-dimensional signal processing. The test results show that the MAE/RMSE of the depression and anxiety level assessment method proposed in this article is 3.05/4.11 and 2.49/1.85, respectively, which has better assessment performance than other related research methods. This study provides an automatic assessment technique based on human computer interaction and virtual reality for mental health physical examination.

A Cascade Network for Pattern Recognition Based on Radar Signal Characteristics in Noisy Environments

Target recognition mainly focuses on three approaches: optical-image-based, echo-detection-based, and passive signal-analysis-based methods. Among them, the passive signal-based method is closely integrated with practical applications due to its strong environmental adaptability. Based on passive radar signal analysis, we design an “end-to-end” model that cascades a noise estimation network with a recognition network to identify working modes in noisy environments. The noise estimation network is implemented based on U-Net, which adopts a method of feature extraction and reconstruction to adaptively estimate the noise mapping level of the sample, which can help the recognition network to reduce noise interference. Focusing on the characteristics of radar signals, the recognition network is realized based on the multi-scale convolutional attention network (MSCANet). Firstly, deep group convolution is used to isolate the channel interaction in the shallow network. Then, through the multi-scale convolution module, the finer-grained features of the signal are extracted without increasing the complexity of the model. Finally, the self-attention mechanism is used to suppress the influence of low-correlation and negative-correlation channels and spaces. This method overcomes the problem of the conventional method being seriously disturbed by noise. We validated the proposed method in 81 kinds of noise environment, achieving an average accuracy of 94.65%. Additionally, we discussed the performance of six machine learning algorithms and four deep learning algorithms. Compared to these methods, the proposed MSCANet achieved an accuracy improvement of approximately 17%. Our method demonstrates better generalization and robustness.

Multi-scale feature extraction and classification of motor imagery electroencephalography based on time series data enhancement

The brain-computer interface (BCI) based on motor imagery electroencephalography (MI-EEG) enables direct information interaction between the human brain and external devices. In this paper, a multi-scale EEG feature extraction convolutional neural network model based on time series data enhancement is proposed for decoding MI-EEG signals. First, an EEG signals augmentation method was proposed that could increase the information content of training samples without changing the length of the time series, while retaining its original features completely. Then, multiple holistic and detailed features of the EEG data were adaptively extracted by multi-scale convolution module, and the features were fused and filtered by parallel residual module and channel attention. Finally, classification results were output by a fully connected network. The application experimental results on the BCI Competition IV 2a and 2b datasets showed that the proposed model achieved an average classification accuracy of 91.87% and 87.85% for the motor imagery task, respectively, which had high accuracy and strong robustness compared with existing baseline models. The proposed model does not require complex signals pre-processing operations and has the advantage of multi-scale feature extraction, which has high practical application value.

PMGCN: Progressive Multi-Graph Convolutional Network for Traffic Forecasting

Traffic forecasting has always been an important part of intelligent transportation systems. At present, spatiotemporal graph neural networks are widely used to capture spatiotemporal dependencies. However, most spatiotemporal graph neural networks use a single predefined matrix or a single self-generated matrix. It is difficult to obtain deeper spatial information by only relying on a single adjacency matrix. In this paper, we present a progressive multi-graph convolutional network (PMGCN), which includes spatiotemporal attention, multi-graph convolution, and multi-scale convolution modules. Specifically, we use a new spatiotemporal attention multi-graph convolution that can extract extensive and comprehensive dynamic spatial dependence between nodes, in which multiple graph convolutions adopt progressive connections and spatiotemporal attention dynamically adjusts each item of the Chebyshev polynomial in graph convolutions. In addition, multi-scale time convolution was added to obtain an extensive and comprehensive dynamic time dependence from multiple receptive field features. We used real datasets to predict traffic speed and traffic flow, and the results were compared with a variety of typical prediction models. PMGCN has the smallest Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) results under different horizons (H = 15 min, 30 min, 60 min), which shows the superiority of the proposed model.

DBFN: Double Branch Fusion Network for Vital Components and Defect Detection of Transmission Line

AbstractA double branch fusion network is proposed based on unmanned aerial vehicle (UAV) inspection images to increase the detection accuracy of vital components and defects in transmission lines. The backbone feature extraction network comprises a combination of a convolutional neural network (CNN) and a Transformer network. To be specific, the CNN should extract local information, and the Transformer network is responsible for the extraction of global information. Besides, global information and local information have semantic differences, while resulting in feature aliasing after fusion. To solve this problem, a multiscale convolution module and a multiscale pooling module are proposed to solve semantic differences and feature aliasing through the interaction between two types of information. In general, the enhanced feature extraction network comprises a residual‐like convolution module, which can reduce the loss of detailed information (e.g., edge contours) and further extract high‐level semantic information from the deep network. Besides, it performs feature fusion in multiple regions in the enhanced feature extraction network, such that the multi‐scale adaptability of the neural network is effectively enhanced. Last, the fused feature information at different scales is decoded, and the final detection results are yielded.

An Effective Image-Based Tomato Leaf Disease Segmentation Method Using MC-UNet.

Tomato disease control is an urgent requirement in the field of intellectual agriculture, and one of the keys to it is quantitative identification and precise segmentation of tomato leaf diseases. Some diseased areas on tomato leaves are tiny and may go unnoticed during segmentation. Blurred edge also makes the segmentation accuracy poor. Based on UNet, we propose an effective image-based tomato leaf disease segmentation method called Cross-layer Attention Fusion Mechanism combined with Multi-scale Convolution Module (MC-UNet). First, a Multi-scale Convolution Module is proposed. This module obtains multiscale information about tomato disease by employing 3 convolution kernels of different sizes, and it highlights the edge feature information of tomato disease using the Squeeze-and-Excitation Module. Second, a Cross-layer Attention Fusion Mechanism is proposed. This mechanism highlights tomato leaf disease locations via gating structure and fusion operation. Then, we employ SoftPool rather than MaxPool to retain valid information on tomato leaves. Finally, we use the SeLU function appropriately to avoid network neuron dropout. We compared MC-UNet to the existing segmentation network on our self-built tomato leaf disease segmentation dataset and MC-UNet achieved 91.32% accuracy and 6.67M parameters. Our method achieves good results for tomato leaf disease segmentation, which demonstrates the effectiveness of the proposed methods.

Lightweight Multi-scale Convolutional Neural Network for Rice Leaf Disease燫ecognition

In the field of agricultural information, the identification and prediction of rice leaf disease have always been the focus of research, and deep learning (DL) technology is currently a hot research topic in the field of pattern recognition. The research and development of high-efficiency, high-quality and low-cost automatic identification methods for rice diseases that can replace humans is an important means of dealing with the current situation from a technical perspective. This paper mainly focuses on the problem of huge parameters of the Convolutional Neural Network (CNN) model and proposes a recognition model that combines a multi-scale convolution module with a neural network model based on Visual Geometry Group (VGG). The accuracy and loss of the training set and the test set are used to evaluate the performance of the model. The test accuracy of this model is 97.1% that has increased 5.87% over VGG. Furthermore, the memory requirement is 26.1 M, only 1.6% of the VGG. Experiment results show that this model performs better in terms of accuracy, recognition speed and memory size.

MCGRAM: Linking Multi-Scale CNN With a Graph-Based Recurrent Attention Model for Subject-Independent ERP Detection

Electroencephalogram (EEG)-based event-related potential (ERP) detection is expected to play an important role in many applications. However, limited by the low signal-to-noise ratio and large inter-subject variability of EEG signals, it is still challenging to design a high-precision ERP detection method in subject-independent scenarios. In this brief, a novel graph-based multi-scale convolutional recurrent attention model (MCGRAM) is proposed to extract the underlying invariant features of EEG signals for subject-independent ERP detection. Specifically, considering the frequency factors of ERP signals, the multi-scale convolution module is designed for learning the frequency representations. Then, the graph convolution module is developed by constructing the links between each EEG node and designing trainable weights of the links to extract the discriminative spatial features. Besides, two-layer LSTM combined with a self-attention model is utilized to extract temporal representations of different EEG slices. Compared to the existing state-of-the-art methods, MCGRAM achieves the best performance in terms of AUC and F1-Score on a rapid serial visual presentation benchmark dataset. Lastly, the ablation studies demonstrate the superiority of each component in MCGRAM.

Tool Wear Prediction Based on a Multi-Scale Convolutional Neural Network with Attention Fusion

Compared with traditional machine learning algorithms, the convolutional neural network (CNN) has an excellent automatic feature learning ability and can complete the nonlinear representation from original data input to output by itself. However, the CNN does not sufficiently mine the tool wear information contained in the multi-sensor data due to disregard of the differences in the contribution of different features when extracting features. In this paper, a tool wear prediction method based on a multi-scale convolutional neural network with attention fusion is proposed, which fuses the tool wear degradation information collected by different types of sensors. In the multi-scale convolution module, convolution kernels with different sizes are used to extract the degradation information of different scales in the wear information, and then the attention fusion module is constructed to fuse the multi-scale feature information. Finally, the mapping between tool wear and multi-sensor data is realized through the feature information obtained by residual connection and full connection layer. By comparing the multi-scale convolutional neural network with different attention mechanisms, the experiments demonstrated the effectiveness and superiority of the proposed method.

Prediction of protein secondary structure based on an improved channel attention and multiscale convolution module.

Prediction of the protein secondary structure is a key issue in protein science. Protein secondary structure prediction (PSSP) aims to construct a function that can map the amino acid sequence into the secondary structure so that the protein secondary structure can be obtained according to the amino acid sequence. Driven by deep learning, the prediction accuracy of the protein secondary structure has been greatly improved in recent years. To explore a new technique of PSSP, this study introduces the concept of an adversarial game into the prediction of the secondary structure, and a conditional generative adversarial network (GAN)-based prediction model is proposed. We introduce a new multiscale convolution module and an improved channel attention (ICA) module into the generator to generate the secondary structure, and then a discriminator is designed to conflict with the generator to learn the complicated features of proteins. Then, we propose a PSSP method based on the proposed multiscale convolution module and ICA module. The experimental results indicate that the conditional GAN-based protein secondary structure prediction (CGAN-PSSP) model is workable and worthy of further study because of the strong feature-learning ability of adversarial learning.