Convolutional Channel Research Articles

Detecting command injection attacks in web applications based on novel deep learning methods

Web command injection attacks pose significant security threats to web applications, leading to potential server information leakage or severe server disruption. Traditional detection methods struggle with the increasing complexity and obfuscation of these attacks, resulting in poor identification of malicious code, complicated feature extraction processes, and low detection efficiency. To address these challenges, a novel detection model, the Convolutional Channel-BiLSTM Attention (CCBA) model, is proposed, leveraging deep learning techniques to enhance the identification of web command injection attacks. The model utilizes dual CNN convolutional channels for comprehensive feature extraction and employs a BiLSTM network for bidirectional recognition of temporal features. An attention mechanism is also incorporated to assign weights to critical features, improving the model’s detection performance. Experimental results demonstrate that the CCBA model achieves 99.3% accuracy and 98.2% recall on a real-world dataset. To validate the robustness and generalization of the model, tests were conducted on two widely recognized public cybersecurity datasets, consistently achieving over 98% accuracy. Compared to existing methods, the proposed model offers a more effective solution for identifying web command injection attacks.

Task-Level Customized Pruning for Image Classification on Edge Devices

Convolutional neural networks (CNNs) are widely utilized in image classification. Nevertheless, CNNs typically require substantial computational resources, posing challenges for deployment on resource-constrained edge devices and limiting the spread of AI-driven applications. While various pruning approaches have been proposed to mitigate this issue, they often overlook a critical fact that edge devices are typically tasked with handling only a subset of classes rather than the entire set. Moreover, the specific combinations of subcategories that each device must discern vary, highlighting the need for fine-grained task-specific adjustments. Unfortunately, these oversights result in pruned models that still contain unnecessary category redundancies, thereby impeding the potential for further model optimization and lightweight design. To bridge this gap, we propose a task-level customized pruning (TLCP) method via utilizing task-level information, i.e., class combination information relevant to edge devices. Specifically, TLCP first introduces channel control gates to assess the importance of each convolutional channel for individual classes. These class-level control gates are then aggregated through linear combinations, resulting in a pruned model customized to the specific tasks of edge devices. Experiments on various customized tasks demonstrate that TLCP can significantly reduce the number of parameters, by up to 33.9% on CIFAR-10 and 14.0% on CIFAR-100, compared to other baseline methods, while maintaining almost the same inference accuracy.

A dual-channel network based on occlusion feature compensation for human pose estimation

Human pose estimation is an important technique in computer vision. Existing methods perform well in ideal environments, but there is room for improvement in occluded environments. The specific reasons are that the ambiguity of the features in the occlusion area makes the network pay insufficient attention to it, and the inadequate expressive ability of the features in the occlusion part cannot describe the true keypoint features. To address the occlusion issue, we propose a dual-channel network based on occlusion feature compensation. The dual channels are occlusion area enhancement channel based on convolution and occlusion feature compensation channel based on graph convolution, respectively. In the convolution channel, we propose an occlusion handling enhanced attention mechanism (OHE-attention) to improve the attention to the occlusion area. In the graph convolution channel, we propose a node feature compensation module that eliminates the obstacle features and integrates the shared and private attributes of the keypoints to improve the expressive ability of the node features. We conduct experiments on the COCO2017 dataset, COCO-Wholebody dataset, and CrowdPose dataset, achieving accuracy of 78.7%, 66.4%, and 77.9%, respectively. In addition, a series of ablation experiments and visualization demonstrations verify the performance of the dual-channel network in occluded environments.

Electromagnetic signal denoising model based on stacked ET layers structure

Electromagnetic signal denoising is crucial for safeguarding sensitive data from electromagnetic leakage. We introduce the Adaptive Stacked Temporal Pyramid Network (ASTPNet) for serial cable electromagnetic signals. ASTPNet combines dynamic convolution, pyramid channel convolution, and stacked ET layers structure to tackle non-linear and non-smooth noise. Dynamic convolution adjusts kernel parameters dynamically based on input signal variations, addressing signal non-smoothness. Pyramid channel convolution enhances the model’s capability to process non-linear features by analyzing key features across multiple scales. Stacked ET layers structure improve signal feature recognition by stacking multiple temporal network structures. Experimental results show that ASTPNet surpasses existing deep learning models in SNR, PSNR, and SI-SDR. The denoised signals can decode restored transmitted information, enhancing methodologies for the processing of electromagnetic signals.

DMGNet: Depth mask guiding network for RGB-D salient object detection

Though depth images can provide supplementary spatial structural cues for salient object detection (SOD) task, inappropriate utilization of depth features may introduce noisy or misleading features, which may greatly destroy SOD performance. To address this issue, we propose a depth mask guiding network (DMGNet) for RGB-D SOD. In this network, a depth mask guidance module (DMGM) is designed to pre-segment the salient objects from depth images and then create masks using pre-segmented objects to guide the RGB subnetwork to extract more discriminative features. Furthermore, a feature fusion pyramid module (FFPM) is employed to acquire more informative fused features using multi-branch convolutional channels with varying receptive fields, further enhancing the fusion of cross-modal features. Extensive experiments on nine benchmark datasets demonstrate the effectiveness of the proposed network.

A Clustering Pruning Method Based on Multidimensional Channel Information

Pruning convolutional neural networks offers a promising solution to mitigate the computational complexity challenges encountered during application deployment. However, prevalent pruning techniques primarily concentrate on model parameters or feature mapping analysis to devise static pruning strategies, often overlooking the underlying feature extraction capacity of convolutional kernels. To address this, the study first quantitatively expresses the feature extraction capability of convolutional channels from three aspects: global features, distribution metrics, and directional metrics. It explores the multi-dimensional information of the channels, calculates the overall expectation, variance, and cosine distance from the unit vector as the quantitative results of the channels. Subsequently, a clustering algorithm is employed to categorize the multidimensional information. This approach ensures that convolutional channels grouped within each cluster possess similar feature extraction capabilities. An enhanced differential evolutionary algorithm is utilized to optimize the number of clustering centers across all convolutional layers, ensuring optimal grouping. The final step involves achieving channel sparsification through the calculation of crowding distances for each sample within its designated cluster. This preserves a diverse subset of channels that are critical for maintaining model accuracy. Extensive empirical evaluations conducted on three benchmark image classification datasets demonstrate the efficacy of this method. For instance, on the ImageNet dataset, the ResNet-50 model experiences a substantial reduction in FLOPs by 58.43% while incurring a minimal decrease in TOP-1 accuracy of only 1.15%.

DA-YOLOv7: A Deep Learning-Driven High-Performance Underwater Sonar Image Target Recognition Model

Affected by the complex underwater environment and the limitations of low-resolution sonar image data and small sample sizes, traditional image recognition algorithms have difficulties achieving accurate sonar image recognition. The research builds on YOLOv7 and devises an innovative fast recognition model designed explicitly for sonar images, namely the Dual Attention Mechanism YOLOv7 model (DA-YOLOv7), to tackle such challenges. New modules such as the Omni-Directional Convolution Channel Prior Convolutional Attention Efficient Layer Aggregation Network (OA-ELAN), Spatial Pyramid Pooling Channel Shuffling and Pixel-level Convolution Bilat-eral-branch Transformer (SPPCSPCBiFormer), and Ghost-Shuffle Convolution Enhanced Layer Aggregation Network-High performance (G-ELAN-H) are central to its design, which reduce the computational burden and enhance the accuracy in detecting small targets and capturing local features and crucial information. The study adopts transfer learning to deal with the lack of sonar image samples. By pre-training the large-scale Underwater Acoustic Target Detection Dataset (UATD dataset), DA-YOLOV7 obtains initial weights, fine-tuned on the smaller Smaller Common Sonar Target Detection Dataset (SCTD dataset), thereby reducing the risk of overfitting which is commonly encountered in small datasets. The experimental results on the UATD, the Underwater Optical Target Detection Intelligent Algorithm Competition 2021 Dataset (URPC), and SCTD datasets show that DA-YOLOV7 exhibits outstanding performance, with mAP@0.5 scores reaching 89.4%, 89.9%, and 99.15%, respectively. In addition, the model maintains real-time speed while having superior accuracy and recall rates compared to existing mainstream target recognition models. These findings establish the superiority of DA-YOLOV7 in sonar image analysis tasks.

Shuffle-fusion pyramid network for bearing fault diagnosis under noisy environments

Abstract Recent advancements in deep learning have propelled the exploration of big data-driven fault diagnosis techniques. Nevertheless, traditional models often suffer from prohibitive computational demands, rendering them impractical for on-site deployment in rolling bearing fault diagnosis. To address this challenge, this paper introduces a novel lightweight fault diagnosis model with the pyramid architecture, named Shuffle-Fusion Pyramid Network (Shuffle-FPN). The model heralds several innovations: (1) A pyramid structure is designed to amalgamate fault signals across various scales, enlarging the network’s breadth and curtailing its depth. (2) Depth wise separable convolutions are adopted to streamline network parameters, thus achieving a lightweight model, and channel shuffling to ensure thorough information fusion across convolutional channels. (3) A global representation module is employed to offset the loss of global context, which accompanies increased convolutional depth. Collectively, these innovations enable Shuffle-FPN to extract nuanced fault features amidst noise and to operate on devices with limited memory, ensuring real-time fault diagnosis even in complex environments. Rigorous experiments on public datasets from Paderborn University (PU), supplemented by validations with our research group’s experiment data, reveal that Shuffle-FPN excels in fault identification under severe noise conditions and reduces memory footprint, which solidifies the superiority of Shuffle-FPN for practical fault diagnosis.

Automatically Extracting and Utilizing EEG Channel Importance Based on Graph Convolutional Network for Emotion Recognition.

Graph convolutional network (GCN) based on the brain network has been widely used for EEG emotion recognition. However, most studies train their models directly without considering network dimensionality reduction beforehand. In fact, some nodes and edges are invalid information or even interference information for the current task. It is necessary to reduce the network dimension and extract the core network. To address the problem of extracting and utilizing the core network, a core network extraction model (CWGCN) based on channel weighting and graph convolutional network and a graph convolutional network model (CCSR-GCN) based on channel convolution and style-based recalibration for emotion recognition have been proposed. The CWGCN model automatically extracts the core network and the channel importance parameter in a data-driven manner. The CCSR-GCN model innovatively uses the output information of the CWGCN model to identify the emotion state. The experimental results on SEED show that: 1) the core network extraction can help improve the performance of the GCN model; 2) the models of CWGCN and CCSR-GCN achieve better results than the currently popular methods. The idea and its implementation in this paper provide a novel and successful perspective for the application of GCN in brain network analysis of other specific tasks.

Bridging Convolutional Neural Networks and Transformers for Efficient Crack Detection in Concrete Building Structures.

Detecting cracks in building structures is an essential practice that ensures safety, promotes longevity, and maintains the economic value of the built environment. In the past, machine learning (ML) and deep learning (DL) techniques have been used to enhance classification accuracy. However, the conventional CNN (convolutional neural network) methods incur high computational costs owing to their extensive number of trainable parameters and tend to extract only high-dimensional shallow features that may not comprehensively represent crack characteristics. We proposed a novel convolution and composite attention transformer network (CCTNet) model to address these issues. CCTNet enhances crack identification by processing more input pixels and combining convolution channel attention with window-based self-attention mechanisms. This dual approach aims to leverage the localized feature extraction capabilities of CNNs with the global contextual understanding afforded by self-attention mechanisms. Additionally, we applied an improved cross-attention module within CCTNet to increase the interaction and integration of features across adjacent windows. The performance of CCTNet on the Historical Building Crack2019, SDTNET2018, and proposed DS3 has a precision of 98.60%, 98.93%, and 99.33%, respectively. Furthermore, the training validation loss of the proposed model is close to zero. In addition, the AUC (area under the curve) is 0.99 and 0.98 for the Historical Building Crack2019 and SDTNET2018, respectively. CCTNet not only outperforms existing methodologies but also sets a new standard for the accurate, efficient, and reliable detection of cracks in building structures.

Data enhanced lightweight network-based prediction of cold-rolled steel mechanical properties

The production of cold-rolled carbon steel is a complex nonlinear process, with chemical composition and process parameters being the key factors influencing its mechanical properties. Establishing the mapping relationship between key parameters and mechanical properties is the key to predicting the mechanical performance of rolled steel. Existing research has overlooked that chemical composition and process parameters are two different scales of data, lacking the exploration of potential features between these scales. This paper proposed a data enhancement-based expand-channel depth-separable convolutional neural network (MBE-EDSC) to predict the mechanical properties of cold-rolled steel. Multi-branch autoencoder was proposed to enhance the original data, which could be mined for potential features of the data. Meanwhile, the self-adaptive convolution channel attention mechanism was proposed to improve the sensitivity of the model to the features, and the channel extension layer to compensate for the problem of accuracy loss caused by deep convolution. Compared with the performance of the existing networks, the proposed network achieved 97.3 %, 98.5 %, and 99.2 % accuracy in predicting the yield strength, tensile strength, and elongation of cold-rolled carbon steel, respectively, and the R2 was improved by 4.9 %, 4.3 %, and 5.9 %, respectively.

A multi-scale multi-channel CNN introducing a channel-spatial attention mechanism hyperspectral remote sensing image classification method

ABSTRACT Aiming the problems that the classification performance of hyperspectral images in existing classification algorithms is highly dependent on spatial-spectral information and that detailed features are ignored in single convolutional channel feature extraction, resulting in poor generalization performance of the feature extraction model, a multi-scale multi-channel convolutional neural network (MMC-CNN) model is proposed in this paper. First, the data set is divided into two kinds of pixel module, and then different channels are used for feature extraction. A channel-space attention mechanism module is also introduced, and a multi-scale multichannel convolutional neural network (CSAM-MMC) model with the introduction of channel-space attention mechanism is proposed for deeper spatial-spectral feature extraction of hyperspectral image elements while reducing the redundancy of convolutional pooling parameters to achieve better HSI classification. To evaluate the effectiveness of the proposed model, experiments were conducted on Indian Pines, Pavia Center and KSC datasets respectively, and the overall classification accuracies of this paper’s MMC-CNN model in the HSI dataset were 97.23%, 98.50%, and 97.85%, thus verifying the model’s high feature extraction accuracy. The CSAM-MMC model in this paper further improves 0.13%, 0.35%, and 0.71% relative to the MMC-CNN model, which provides higher overall accuracies relative to other state-of-the-art algorithms.

Fault diagnosis of rolling bearings under varying speeds based on gray level co-occurrence matrix and DCCNN

Rolling bearings are widely used in various industries, including rail transit, aerospace, and wind power generation, playing a critical role. However, bearing failures can lead to serious consequences, impacting equipment operation and even causing safety accidents. Hence, the diagnosis of bearing faults is of utmost importance. However, the variable speed conditions experienced during bearing operation pose significant challenges to fault diagnosis. To overcome the limitations of traditional methods in diagnosing bearing faults under variable speed conditions, this paper proposes a fault diagnosis method based on the gray-level co-occurrence matrix (GLCM) and Dual Channel Convolutional Neural Network (DCCNN). The method introduces a two-dimensional grayscale matrix construction (2D-GMC) technique to extract grayscale texture features for fault diagnosis. Additionally, an unconventional kernel design method, based on grayscale image contrast, is proposed to reduce the complexity associated with traditional square kernels. A new DCCNN architecture is developed accordingly. Furthermore, the transfer learning concept is utilized to train the proposed DCCNN model using fault signals at specific rotational speeds. The method intercepts the variable speed into multi-speed short-time series, then constructs gray image under different speed to realize the rapid fault diagnosis of bearings under variable speed conditions.

Smartphone-based straw incorporation: An improved convolutional neural network

In conservation tillage, the effective incorporation of straw into soil is a critical factor for optimizing decomposition rates and addressing challenges related to straw accumulation. Consequently, this study introduces an innovative straw semantic segmentation model to extract straw mixed with soil from images, and proposes a method to measure the quality of straw and soil mixing based on this model. We conducted field experiments based on this method, analyzed the quality of straw incorporation at different depths, and finally integrated the model into a mobile application to facilitate the automatic, accurate, and rapid assessment of the quality of straw and soil mixing. The methodology of this study as an improved convolutional neural network entailed the fusion of deep separable convolution with a multi-scale feature extraction module, utilizing convolution kernels of varying sizes to capture the diverse features of straw. Further refinement included halving the first convolution channel and implementing a direct connection structure. Attention gates were strategically deployed to enhance salient features, resulting in superior accuracy compared to the original U-shaped network, fully convolutional network, and Otsu algorithm. Notably, the proposed model achieved enhanced performance while reducing the number of parameters and model complexity to one-third and one-half of the original U-shaped network, respectively. For a quantitative description of the rate and uniformity of straw incorporation, this study employed image processing technology and a grid counting method. The synergistic integration of smartphone imagery and deep learning expedited the delivery of rapid and reliable results, promising practical applicability in future straw incorporation practices. A demonstration video of the application is accessible at: https://doi.org/10.6084/m9.figshare.14480745.v3. The findings of this research contribute significantly to the advancement of conservation tillage by establishing a robust framework for the assessment and enhancement of straw incorporation operations’ efficiency.

Rehabilitation Assessment System for Stroke Patients Based on Fusion-Type Optoelectronic Plethysmography Device and Multi-Modality Fusion Model: Design and Validation.

This study aimed to propose a portable and intelligent rehabilitation evaluation system for digital stroke-patient rehabilitation assessment. Specifically, the study designed and developed a fusion device capable of emitting red, green, and infrared lights simultaneously for photoplethysmography (PPG) acquisition. Leveraging the different penetration depths and tissue reflection characteristics of these light wavelengths, the device can provide richer and more comprehensive physiological information. Furthermore, a Multi-Channel Convolutional Neural Network-Long Short-Term Memory-Attention (MCNN-LSTM-Attention) evaluation model was developed. This model, constructed based on multiple convolutional channels, facilitates the feature extraction and fusion of collected multi-modality data. Additionally, it incorporated an attention mechanism module capable of dynamically adjusting the importance weights of input information, thereby enhancing the accuracy of rehabilitation assessment. To validate the effectiveness of the proposed system, sixteen volunteers were recruited for clinical data collection and validation, comprising eight stroke patients and eight healthy subjects. Experimental results demonstrated the system's promising performance metrics (accuracy: 0.9125, precision: 0.8980, recall: 0.8970, F1 score: 0.8949, and loss function: 0.1261). This rehabilitation evaluation system holds the potential for stroke diagnosis and identification, laying a solid foundation for wearable-based stroke risk assessment and stroke rehabilitation assistance.

Pruning Deep Neural Network Models via Minimax Concave Penalty Regression

In this study, we propose a filter pruning method based on MCP (Minimax Concave Penalty) regression. The convolutional process is conceptualized as a linear regression procedure, and the regression coefficients serve as indicators to assess the redundancy of channels. In the realm of feature selection, the efficacy of sparse penalized regression gradually outperforms that of Lasso regression. Building upon this insight, MCP regression is introduced to screen convolutional channels, coupled with the coordinate descent method, to effectuate model compression. In single-layer pruning and global pruning analyses, the Top1 loss value associated with the MCP regression compression method is consistently smaller than that of the Lasso regression compression method across diverse models. Specifically, when the global pruning ratio is set to 0.3, the Top1 accuracy of the MCP regression compression method, in comparison with that of the Lasso regression compression method, exhibits improvements of 0.21% and 1.67% under the VGG19_Simple and VGG19 models, respectively. Similarly, for ResNet34, at two distinct pruning ratios, the Top1 accuracy demonstrates enhancements of 0.33% and 0.26%. Lastly, we compare and discuss the novel methods introduced in this study, considering both time and space resource consumption.

Fault severity recognition in axial piston pumps using attention-based adversarial discriminative domain adaptation neural network

Axial piston pumps are the ‘hearts’ of hydraulic systems whose fault recognition is necessary for the safety and reliability of hydraulic equipment. These pumps operate under different operating conditions and the fault recognition model trained at one operating point cannot be applicable at another operating point due to the problem of domain shifts. This paper proposes a transfer learning method for the fault severity recognition of axial piston pumps based on adversarial discriminative domain adaptation fused with a convolutional channel attention module. First, a convolutional neural network is pre-trained with labeled vibration data from the source domain, and a convolutional channel attention module is added to assign weights to different convolution kernels. Second, the trained source model is transferred to the target domain, and its parameters are updated by an adversarial training process between the labeled source data and the unlabeled target data. Finally, vibration data are collected from an axial piston pump at different fault levels under various operating conditions to validate the proposed method. Experimental results indicate that the proposed method achieves an average recognition accuracy of 98.3% and outperforms some other transfer learning methods by a large margin.

SSMA-YOLO: A Lightweight YOLO Model with Enhanced Feature Extraction and Fusion Capabilities for Drone-Aerial Ship Image Detection

Due to the unique distance and angles involved in satellite remote sensing, ships appear with a small pixel area in images, leading to insufficient feature representation. This results in suboptimal performance in ship detection, including potential misses and false detections. Moreover, the complexity of backgrounds in remote sensing images of ships and the clustering of vessels also adversely affect the accuracy of ship detection. Therefore, this paper proposes an optimized model named SSMA-YOLO, based on YOLOv8n. First, this paper introduces a newly designed SSC2f structure that incorporates spatial and channel convolution (SCConv) and spatial group-wise enhancement (SGE) attention mechanisms. This design reduces spatial and channel redundancies within the neural network, enhancing detection accuracy while simultaneously reducing the model’s parameter count. Second, the newly designed MC2f structure employs the multidimensional collaborative attention (MCA) mechanism to efficiently model spatial and channel features, enhancing recognition efficiency in complex backgrounds. Additionally, the asymptotic feature pyramid network (AFPN) structure was designed for progressively fusing multi-level features from the backbone layers, overcoming challenges posed by multi-scale variations. Experiments of the ships dataset show that the proposed model achieved a 4.4% increase in mAP compared to the state-of-the-art single-stage target detection YOLOv8n model while also reducing the number of parameters by 23%.

Ultrasound channel attention residual network for medical plane wave echo data-based average sound speed estimation

The average sound speed estimation is crucial for ultrasound imaging quality and diagnostic. In this article, the deep learning techniques were utilized and an innovative Ultrasound Channel Attention Residual Network (UCA-ResNet) was proposed. The UCA-ResNet incorporated a specially designed Ultrasound Channel Attention (UCA) block, which effectively enhanced relevant ultrasound channel features and convolutional channel features. For evaluation, the simulation, phantom, and in vivo experiments were conducted. In the simulation experiment, UCA-ResNet achieved remarkable results, with a mean absolute error (MAE) of 0.40 m/s, root mean square error (RMSE) of 1.25 m/s, standard deviation of error (SDE) of 1.25 m/s, and a one-time estimation time of 3.67 ms. Moreover, the phantom and in vivo experiments further validated the high accuracy and low computational cost of UCA-ResNet. The UCA-ResNet can accurately estimate the average sound speed using a single plane wave echo data while maintaining low computational cost. It has potential in enhancing medical ultrasound imaging quality and providing novel diagnostic insights. (Code and data will be available upon acceptance of the manuscript.)

Interpretable Siamese dual attention enhancement transfer compound diagnostic model for unbalanced samples

The intelligent transfer diagnosis model is used to address the issue of feature drift caused by the changing working conditions of rotating parts in engineering. However, few models can perform transfer diagnosis on multiple unbalanced samples of rotating parts simultaneously, and even fewer models can visually enhance the domain-invariant features, making them more interpretable. To address these issues, we propose a novel interpretable Siamese dual attention enhancement transfer compound diagnosis model for unbalanced samples. The model can diagnose multiple rotating parts simultaneously and consists of a channel feature attention enhancement (CFAE) network, a fragment feature attention enhancement (FFAE) network, and a Siamese feature fusion (SFF) network. The CFAE network enhances features of different convolutional channels, the FFAE network improves segment features in various frequency domains, and the SFF network extracts domain-invariant features of diverse rotating components under varying working conditions. The model is validated using bearing fault data collected under different loads and planetary gear fault data obtained at varying speeds. Its diagnostic accuracy remains above 96.4%, and the diagnostic variance is controlled within 1.0%. The model has good interpretability for imbalanced sample domain-invariant features, providing an effective tool for interpretable transfer diagnosis in this compound engineering situation.