Feature Representation Capability Research Articles

A lightweight dynamic dual-damped wavelet-based convolutional neural network for interpretable bearing fault diagnosis

Wavelet-based convolutional neural networks (CNNS) have attracted widespread attention because they can improve the interpretability of intelligent fault diagnosis methods. However, the fault feature representation capability of typical wavelet-based convolution kernel frameworks must be strengthened to improve the diagnostic accuracy of complex faults. In the meantime, the large number of network parameters leads to high computational costs. To address these issues, a lightweight wavelet-based dynamic CNN, which comprises a dual-damping wavelet-based dynamic CNN (DWDC) block and a discrete wavelet transformation (DWT) enhancement (DWTE) block, is put forward. In the DWDC block, a wavelet convolution layer is initially designed, where a dual-damping wavelet is used as the kernel function to improve the match of the convolution kernel with fault impulses. Subsequently, a dynamic convolution layer with multiple parallel small-size convolutional kernels is designed to screen the fault features instead of a multilayer network structure, thereby greatly reducing the number of network parameters. Finally, the DWTE block is constructed by combining the DWT and residual dense block, and it can mine more fault information from the previously extracted features. The experiments on the variable speed bearing dataset, locomotive bearing dataset with constant speed and the Case Western Reserve University dataset prove that the proposed approach outperforms five classical CNN models and six advanced wavelet-based CNN models. In addition, it can effectively solve the issue of data imbalance because of its powerful feature extraction capability.

Fire Smoke Target Detection Incorporating PBCA

Background: Fire incidents occur in complex scenarios, where the dynamic positions and varying scales of flames and smoke pose challenges for fire detection. To improve the stability, localization accuracy, and detection precision of small targets in fire detection, a fused PBCA method for fire and smoke object detection has been proposed in this paper, called FS-YOLOv8. Background: Fire incidents occur in complex scenarios, where the dynamic positions and varying scales of flames and smoke pose challenges for fire detection. To improve the stability, localization accuracy, and detection precision of small targets in fire detection, a fused PBCA method for fire and smoke object detection has been proposed in this paper, called FS-YOLOv8. Objective: The objective of this approach was to improve the detection accuracy of flames and smoke, enhance the robustness of the system, and strengthen the feature representation capability. It aimed to optimize the contribution of feature maps at different scales, allowing the network to capture inter-channel correlations while preserving precise localization information of the targets. Furthermore, it aimed to enhance the learning ability of small-scale flame and smoke objects. Method: Firstly, DCN (Deformable Convolutional Network) was integrated into the CSPDarknet backbone network to extract features from flame and smoke images. Subsequently, a module called PBCA was designed by combining BiFPN (Bidirectional Feature Pyramid Network) and coordinate attention. Finally, a small object detection layer, YOLO HEAD-4, was constructed. Results: The experimental results of our proposed FS-YOLOv8 method on a self-made dataset demonstrated improved detection accuracy compared to other conventional methods. Therefore, the FS-YOLOv8 method effectively enhanced the performance of object detection in fire and smoke scenarios. Conclusion: The FS-YOLOv8 method has been found to effectively improve the performance of object detection in fire and smoke scenarios, enhance the robustness of the system, strengthen the feature representation capability, and amplify the learning ability of small-scale flame and smoke objects. other: NO

Cross-Attention Enhanced Pyramid Multi-Scale Networks for Sensor-Based Human Activity Recognition.

Human Activity Recognition (HAR) has recently attracted widespread attention, with the effective application of this technology helping people in areas such as healthcare, smart homes, and gait analysis. Deep learning methods have shown remarkable performance in HAR. A pivotal challenge is the trade-off between recognition accuracy and computational efficiency, especially in resource-constrained mobile devices. This challenge necessitates the development of models that enhance feature representation capabilities without imposing additional computational burdens. Addressing this, we introduce a novel HAR model leveraging deep learning, ingeniously designed to navigate the accuracy-efficiency trade-off. The model comprises two innovative modules: 1) Pyramid Multi-scale Convolutional Network (PMCN), which is designed with a symmetric structure and is capable of obtaining a rich receptive field at a finer level through its multiscale representation capability; 2) Cross-Attention Mechanism, which establishes interrelationships among sensor dimensions, temporal dimensions, and channel dimensions, and effectively enhances useful information while suppressing irrelevant data. The proposed model is rigorously evaluated across four diverse datasets: UCI, WISDM, PAMAP2, and OPPORTUNITY. Additional ablation and comparative studies are conducted to comprehensively assess the performance of the model. Experimental results demonstrate that the proposed model achieves superior activity recognition accuracy while maintaining low computational overhead.

An efficient Fusion-Purification Network for Cervical pap-smear image classification

Background and Objectives:In cervical cell diagnostics, autonomous screening technology constitutes the foundation of automated diagnostic systems. Currently, numerous deep learning-based classification techniques have been successfully implemented in the analysis of cervical cell images, yielding favorable outcomes. Nevertheless, efficient discrimination of cervical cells continues to be challenging due to large intra-class and small inter-class variations. The key to dealing with this problem is to capture localized informative differences from cervical cell images and to represent discriminative features efficiently. Existing methods neglect the importance of global morphological information, resulting in inadequate feature representation capability. Methods:To address this limitation, we propose a novel cervical cell classification model that focuses on purified fusion information. Specifically, we first integrate the detailed texture information and morphological structure features, named cervical pathology information fusion. Second, in order to enhance the discrimination of cervical cell features and address the data redundancy and bias inherent after fusion, we design a cervical purification bottleneck module. This model strikes a balance between leveraging purified features and facilitating high-efficiency discrimination. Furthermore, we intend to unveil a more intricate cervical cell dataset: Cervical Cytopathology Image Dataset (CCID). Results:Extensive experiments on two real-world datasets show that our proposed model outperforms state-of-the-art cervical cell classification models. Conclusions:The results show that our method can well help pathologists to accurately evaluate cervical smears.

SwinSOD: Salient object detection using swin-transformer

The Transformer structure has achieved excellent performance in a wide range of applications in computer vision, and Swin-Transformer also shows strong feature representation capabilities. On this basis, we proposed a fusion model SwinSOD for RGB salient object detection. This model used a Swin-Transformer as the encoder to extract hierarchical features, was driven by a multi-head attention mechanism to bridge the gap between hierarchical features, progressively fused adjacent layer feature information under the guidance of global information, and refined the boundaries of saliency objects through the feedback information. Specifically, the Swin-Transformer encoder extracted multi-level features and then recalibrated the channels to optimize intra-layer channel features. The feature fusion module realized feature fusion between each layer under the guidance of global information. In order to clarify the fuzzy boundaries, the second stage feature fusion achieved edge refinement under the guidance of feedback information. The proposed model outperforms state-of-the-art models on five popular SOD datasets, demonstrating the advanced performance of this network. Code released: https://github.com/user-wu/SwinSOD.

Research on lightweight pavement disease detection model based on YOLOv7

Rapid urbanization has made road construction and maintenance imperative, but detecting road diseases has been time-consuming with limited accuracy. To overcome these challenges, we propose an efficient YOLOv7 road disease detection model. Our approach involves integrating MobilieNetV3 as the backbone feature extraction network to reduce the network’s parameters and computational requirements. Additionally, we introduce the BRA attention module into the spatial pyramid pooling module to eliminate redundant information and enhance the network’s feature representation capability. Moreover, we utilize the F-ReLU activation function in the backbone network, expanding the convolutional layers’ receptive field range. To optimize the model’s boundary loss, we employ the Wise-IoU loss function, which places more emphasis on the quality of ordinary samples and enhances the overall performance and generalization ability of the network. Experimental results demonstrate that our improved detection algorithm achieves a higher recall rate and mean average precision (mAP) on the public dataset (RDD) and the NJdata dataset in Nanjing’s urban area. Specifically, compared to YOLOv7, our model increases the recall rate and mAP on RDD by 3.3% and 2.6%, respectively. On the NJdata dataset, our model improves the recall rate and mAP by 1.9% and 1.3%, respectively. Furthermore, our model reduces parameter and computational requirements by 30% and 22.5%, respectively, striking a balance between detection accuracy and speed. In conclusion, our road disease detection model presents an effective solution to address the challenges associated with road disease detection in urban areas. It offers improved accuracy, efficiency, and generalization capabilities compared to existing models.

Multi-attention bidirectional contrastive learning method for unpaired image-to-image translation.

Unpaired image-to-image translation (I2IT) involves establishing an effective mapping between the source and target domains to enable cross-domain image transformation. Previous contrastive learning methods inadequately accounted for the variations in features between two domains and the interrelatedness of elements within the features. Consequently, this can result in challenges encompassing model instability and the blurring of image edge features. To this end, we propose a multi-attention bidirectional contrastive learning method for unpaired I2IT, referred to as MabCUT. We design separate embedding blocks for each domain based on depthwise separable convolutions and train them simultaneously from both the source and target domains. Then we utilize a pixel-level multi-attention extractor to query images from embedding blocks in order to select feature blocks with crucial information, thus preserving essential features from the source domain. To enhance the feature representation capability of the model, we incorporate depthwise separable convolutions for the generator. We conducted comprehensive evaluations using three datasets, demonstrating that our approach enhances the quality of unpaired I2IT while avoiding the issue of mode collapse-related image blurring.

PAS-GAN: A GAN based on the Pyramid Across-Scale module for visible-infrared image transformation

With the rapid development of computer vision, there is a recent trend for intelligent image understanding based on deep learning. RGB images and infrared images have complementary information in image capture tasks in complex environments due to their different imaging modalities. Therefore, the combination of the two plays an important role in improving video surveillance and target detection capabilities. However, large publicly available infrared image datasets are lacking and acquiring infrared images can be resource-intensive. The lack of samples can lead to a breakdown in the training of deep models. In this paper, to overcome this challenge, we construct a GAN-based visible-infrared image transformation model. The model uses existing visible data to generate infrared images by training an end-to-end generative network. We innovatively propose a lightweight PAS feature extraction module applied to the generator. It enriches the image detail representation of the feature domain from multiple dimensions and greatly improves the model feature representation capability. And the image gradient calculation is used to limit the direction of model optimization. We evaluated our model on three different publicly available datasets, evaluating the quality of the generated images in terms of both visual effects and objective numerical assessments. The experimental results show that the network exhibits excellent results in both qualitative and quantitative evaluation compared to the current state-of-the-art image generation methods. The lightweight feature extraction module also gives PAS-GAN a significant advantage in inference speed.

Fault detection of complicated processes based on an enhanced transformer network with graph attention mechanism

Recently, deep learning becomes increasingly popular in the industrial data analysis field due to its distinguished feature representation capability. As an emerging deep learning technology, transformer network has attracted extensive attention, but its application in industrial fault detection is still not sufficiently exploited. Furthermore, the traditional transformer network mainly focuses on the time series information of the data, while ignoring the spatial characteristic of the data. For this problem, an improved transformer network model with graph attention mechanism (GA-Tran) is proposed and used for complicated process fault detection. Considering the strong coupling property of process variables, the presented method performs joint spatial-temporal learning by mining the data information in views of both time-dimension and variable-dimension. On the one hand, a transformer encoder module equipped with a self-attention mechanism is utilized to capture long-term temporal dependencies, while multi-scale convolution is integrated to abstract short-term time dependencies. On the other hand, a graph attention network is introduced into the transformer network to meticulously analyze the spatial relationships among variables. Considering the topology of the industrial process variables, we use spectral clustering to infer prior topology information, and remove intelligently irrelevant spatial information. Different from the existing transformer models with the anomaly scores as the monitoring indicator, two monitoring statistics based on encoder features and reconstruction errors are constructed for process status monitoring. Studies on the Tennessee Eastman chemical process demonstrate the effectiveness of the proposed GA-Tran method.

SurfPro-NN: A 3D point cloud neural network for the scoring of protein–protein docking models based on surfaces features and protein language models

Protein–protein interactions (PPI) play a crucial role in numerous key biological processes, and the structure of protein complexes provides valuable clues for in-depth exploration of molecular-level biological processes. Protein–protein docking technology is widely used to simulate the spatial structure of proteins. However, there are still challenges in selecting candidate decoys that closely resemble the native structure from protein–protein docking simulations. In this study, we introduce a docking evaluation method based on three-dimensional point cloud neural networks named SurfPro-NN, which represents protein structures as point clouds and learns interaction information from protein interfaces by applying a point cloud neural network. With the continuous advancement of deep learning in the field of biology, a series of knowledge-rich pre-trained models have emerged. We incorporate protein surface representation models and language models into our approach, greatly enhancing feature representation capabilities and achieving superior performance in protein docking model scoring tasks. Through comprehensive testing on public datasets, we find that our method outperforms state-of-the-art deep learning approaches in protein–protein docking model scoring. Not only does it significantly improve performance, but it also greatly accelerates training speed. This study demonstrates the potential of our approach in addressing protein interaction assessment problems, providing strong support for future research and applications in the field of biology.

Spectral Graph Convolutional Neural Networks in the Context of Regularization Theory.

Graph convolutional neural networks (GCNN) have been widely used in graph learning and related applications. It has been identified that the filters in the state-of-the-art spectral graph convolutional networks (SGCNs) are essentially low-pass filters that enforce smoothness across the graph and use the functions of graph Laplacian as a tool that injects graph structure into the learning algorithm. There had been research findings that connect the smoothness functional in graphs, graph Laplacian, and regularization operators. We review the existing SGCNs in this context and propose a framework where the state-of-the-art filter designs can be deduced as special cases. We designed new filters that are associated with well-defined low-pass behavior and tested their performance on semisupervised node classification tasks. Their performance was found to be superior to that of the other state-of-the-art techniques. We further investigate the representation capability of low-pass features and make useful observations. In this context, we discuss a few points to further optimize the network, new strategies for designing SGCNs, their challenges, and some latest related developments. Based on our framework, we also deduce the connection between support vector kernels and SGCN filters.

A multi-task segmentation and classification network for remote ship hull inspection

In contrast to manual close-up ship hull inspection methods, Remote Inspection Technology (RIT) offers the potential to improve performance while minimising costs. Nevertheless, the effectiveness of the RIT is still subject to the experience and expertise of the inspectors. As a result, development has focused on applying automated data processing methods to RIT itself. However, in the hull inspection scenario, several challenges remain, including suboptimal hull imaging conditions, disparities in the distribution of defect categories, and significant variations in defect size. To overcome these challenges, we introduce a multi-task hull inspection network (MTHI-Net) that leverages the principles of multi-task learning to improve hull inspection accuracy. This network addresses two problems associated with RIT: image-level defect classification and pixel-level defect segmentation. Specifically, MTHI-Net exploits the advantages of spatial and channel self-attention mechanisms, a residual refinement module, and a feature fusion module to enhance the feature representation capability for defect classification and mitigate the probability of mis-segmentation. In addition, a lightweight MTHI-Net called MTHI-Net-Lite was developed. The experimental results show that the proposed networks fulfil the hull inspection task better than the baselines in both real underwater and in-dock RIT scenarios.

Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking

The global multi-object tracking (MOT) system can consider interaction, occlusion, and other ``visual blur'' scenarios to ensure effective object tracking in long videos. Among them, graph-based tracking-by-detection paradigms achieve surprising performance. However, their fully-connected nature poses storage space requirements that challenge algorithm handling long videos. Currently, commonly used methods are still generated trajectories by building one-forward associations across frames. Such matches produced under the guidance of first-order similarity information may not be optimal from a longer-time perspective. Moreover, they often lack an end-to-end scheme for correcting mismatches. This paper proposes the Composite Node Message Passing Network (CoNo-Link), a multi-scene generalized framework for modeling ultra-long frames information for association. CoNo-Link's solution is a low-storage overhead method for building constrained connected graphs. In addition to the previous method of treating objects as nodes, the network innovatively treats object trajectories as nodes for information interaction, improving the graph neural network's feature representation capability. Specifically, we formulate the graph-building problem as a top-k selection task for some reliable objects or trajectories. Our model can learn better predictions on longer-time scales by adding composite nodes. As a result, our method outperforms the state-of-the-art in several commonly used datasets.

DBL-Net: A dual-branch learning network with information from spatial and frequency domains for tumor segmentation and classification in breast ultrasound image

Breast cancer is a common malignancy, ranking as one of the most prevalent cancers among women globally. Improving treatment success rates for breast cancer are crucially dependent on early detection and accurate diagnosis. Ultrasound imaging, as a non-invasive and low-radiation imaging technique, plays a significant role in early breast cancer detection. However, due to the characteristics of fuzzy tumor boundaries and irregular shapes of the tumor, accurate breast tumor classification and segmentation remain challenging tasks. This paper introduces a novel approach for breast ultrasound image segmentation and classification tasks. Firstly, we study the potential value of frequency domain information in breast image analysis and propose the DBL-Net, a multi-task learning network for simultaneous tumor segmentation and tumor classification. Secondly, we customize a Spatial-Frequency Domain Feature Encoding module (S-FEM) to better encode and fuse spatial and frequency domain features of breast ultrasound images. Furthermore, we design a Representational Perception Enhancer (RPE) to improve the feature representation capabilities and prediction accuracy of the model. Finally, We have carried out a comprehensive series of tests on the BUS dataset, and the findings clearly indicate that our method outperforms the current state-of-the-art methods.

Multimodal Knowledge-enhanced Interactive Network with Mixed Contrastive Learning for Emotion Recognition in Conversation

Emotion Recognition in Conversations (ERC) aims to accurately identify the emotional labels of each utterance in a conversation, holding significant application value in human–computer interaction. Existing research suggests introducing commonsense knowledge (CSK) and multimodal information enhances model performance in ERC tasks. However, several challenges persist: (1) the neglect of complex psychological influences between utterances; (2) noise issues within modal information; (3) prediction challenges for emotion labels with few samples in different categories that exhibit semantic similarity but distinct emotional categories. To address the above problems, we propose a Multimodal Knowledge-enhanced Interactive Network with Mixed Contrastive Learning (MKIN-MCL). Firstly, we establish a knowledge aggregation graph to capture the dependencies of commonsense knowledge (CSK) between utterances during a conversation. We actively aggregate relevant knowledge information to enhance text features. Simultaneously, we apply feature filters for acoustic and visual modalities to eliminate noise and enhance feature quality. Furthermore, we implement an interactive attention module by stacking designed Cross-modal Interactive Transformers (CITs) to continuously explore the relevance between the interacting parties in their respective semantic spaces, thus improving the effectiveness of modality interaction while reducing noise generated during the interaction. Lastly, we employ the Mixed Contrastive Learning (MCL) strategy to enhance the model’s ability to handle few-shot labels. This strategy utilizes unsupervised contrastive learning to improve the representation capability of the multimodal fusion features and supervised contrastive learning to extract information from few-shot labels. Extensive experiments on two benchmark datasets, IEMOCAP and MELD, validate the effectiveness and superiority of the proposed model.

Research on surface defect detection model of steel strip based on MFFA‐YOLOv5

AbstractThe surface quality of steel strip is a critical indicator of the quality of hot‐rolled strip, so accurate inspection of its surface is essential. However, the complex texture of steel strip surface defects makes the detection challenging. Here, a multi‐scale feature fusion and attention based YOLOv5 (MFFA‐YOLOv5) model is proposed. Specifically, the bottom layer features are up‐sampled and fused not only with the middle layer features, but also with the top layer features, so that the model better captures the surface texture information of the steel strip. Secondly, an improved attention mechanism module is introduced to deal with the global and local information of the steel strip surface by introducing down‐sampling and up‐sampling paths based on Convolutional Block Attention Module (CBAM). Meanwhile, a self‐attention mechanism path is added to improve the capability of feature representation. Experimental results on the NEU‐DET dataset show that the MFFA‐YOLOv5 model significantly outperforms other state‐of‐the‐art methods.

Channel-wise attention enhanced and structural similarity constrained cycleGAN for effective synthetic CT generation from head and neck MRI images

BackgroundMagnetic resonance imaging (MRI) plays an increasingly important role in radiotherapy, enhancing the accuracy of target and organs at risk delineation, but the absence of electron density information limits its further clinical application. Therefore, the aim of this study is to develop and evaluate a novel unsupervised network (cycleSimulationGAN) for unpaired MR-to-CT synthesis.MethodsThe proposed cycleSimulationGAN in this work integrates contour consistency loss function and channel-wise attention mechanism to synthesize high-quality CT-like images. Specially, the proposed cycleSimulationGAN constrains the structural similarity between the synthetic and input images for better structural retention characteristics. Additionally, we propose to equip a novel channel-wise attention mechanism based on the traditional generator of GAN to enhance the feature representation capability of deep network and extract more effective features. The mean absolute error (MAE) of Hounsfield Units (HU), peak signal-to-noise ratio (PSNR), root-mean-square error (RMSE) and structural similarity index (SSIM) were calculated between synthetic CT (sCT) and ground truth (GT) CT images to quantify the overall sCT performance.ResultsOne hundred and sixty nasopharyngeal carcinoma (NPC) patients who underwent volumetric-modulated arc radiotherapy (VMAT) were enrolled in this study. The generated sCT of our method were more consistent with the GT compared with other methods in terms of visual inspection. The average MAE, RMSE, PSNR, and SSIM calculated over twenty patients were 61.88 ± 1.42, 116.85 ± 3.42, 36.23 ± 0.52 and 0.985 ± 0.002 for the proposed method. The four image quality assessment metrics were significantly improved by our approach compared to conventional cycleGAN, the proposed cycleSimulationGAN produces significantly better synthetic results except for SSIM in bone.ConclusionsWe developed a novel cycleSimulationGAN model that can effectively create sCT images, making them comparable to GT images, which could potentially benefit the MRI-based treatment planning.

Intelligent fault diagnosis for electro-hydrostatic actuator based on multisource information convolutional residual network

The electro-hydrostatic actuator (EHA), known for its advantages such as minimal throttling loss, high efficiency, and a significant volume-to-power ratio, has found extensive application in the fields of aeronautics and astronautics. However, ensuring the safety of aircraft that utilize EHAs requires efficient fault diagnosis due to the demanding operational conditions and prolonged usage. Traditional diagnostic approaches face challenges such as intricate fault modeling, complex multi-channel monitoring data, and a limited number of fault samples within the electro-hydraulic system. To overcome these challenges, we propose an intelligent diagnosis method based on a multi-source information convolutional residual network. Specifically, a multis-cale kernel is implemented to capture features at different scales, enhancing model expressiveness. The efficiency channel attention mechanism dynamically focuses on relevant channel features to improve feature learning ability. The residual network adaptively recalibrates features at each layer to facilitate fault feature learning. Additionally, the activate or not activation function is introduced to selectively activate shallow features, thereby improving the feature representation and generalization capability of the model. Experimental data from the EHA system validates the superiority of the proposed method, demonstrating a significant enhancement in the diagnostic accuracy of EHAs with limited samples.

Studies Advanced in Development and Application of Facial Expression Recognition

Facial expression is an important way to convey people's inner emotional changes, as the basis of emotional understanding and the prerequisite for computers to understand human emotions, expression recognition has attracted more and more research attention from academia and industry in recent years. Most of the early expression recognition methods relied on manual features such as texture, geometry, and contour. Thanks to the powerful feature representation capability of convolutional neural networks, deep learning-based face recognition technology has made breakthroughs in recognition accuracy and speed. This paper details the latest research progress in the field of face expression recognition, including the design ideas, key steps, advantages and disadvantages of representative methods. This paper also introduces the common face expression recognition datasets and quantitatively compares the results of different methods on these datasets. Finally, this paper discusses the problems in the face expression recognition research field and look forward to the future development.

DSCNet: lightweight and efficient self-supervised network via depthwise separable cross convolution blocks for speckle image matching.

Speckle structured light has become a research hotspot due to its ability to acquire target three-dimensional information with single image projection in recent years. To address the challenges of a low number of extracted speckle feature points, high mismatch rate and poor real-time performance in traditional algorithms, as well as the obstacle of requiring expensive annotation data in deep learning-based methods, a lightweight and efficient self-supervised convolutional neural network (CNN) is proposed to achieve high-precision and rapid matching of speckle images. First, to efficiently utilize the speckle projection information, a feature extraction backbone based on the depthwise separable cross convolution blocks is proposed. Second, in the feature detection module, a softargmax detection head is designed to refine the coordinates of speckle feature points to sub-pixel accuracy. In the feature description module, a coarse-to-fine module is presented to further refine matching accuracy. Third, we adopt strategies of transfer learning and self-supervised learning to improve the generalization and feature representation capabilities of the model. Data augmentation and real-time training techniques are used to improve the robustness of the model. The experimental results show that the proposed method achieves a mean matching accuracy of 91.62% for speckle feature points on the pilot's helmet, with mere 0.95% mismatch rate. The full model runs at 42ms for a speckle image pair on an RTX 3060.