Effective Feature Representation Research Articles

A duplex transform heterogeneous feature fusion network for road segmentation

Detecting roads in automatic driving environments poses a challenge due to issues such as boundary fuzziness, occlusion, and glare from light. We believe that two factors are instrumental in addressing these challenges and enhancing detection performance: global context dependency and effective feature representation that prioritizes important feature channels. To tackle these issues, we introduce DTRoadseg, a novel duplex Transformer-based heterogeneous feature fusion network designed for road segmentation. DTRoadseg leverages a duplex encoder architecture to extract heterogeneous features from both RGB images and point-cloud depth images. Subsequently, we introduce a multi-source Heterogeneous Feature Reinforcement Block (HFRB) for fusion of the encoded features, comprising a Heterogeneous Feature Fusion Module (HFFM) and a Reinforcement Fusion Module (RFM). The HFFM leverages the self-attention mechanisms of Transformers to achieve effective fusion through token interactions, while the RFM focuses on emphasizing informative features while downplaying less important ones, thereby reinforcing feature fusion. Finally, a Transformer decoder is utilized to produce the final semantic prediction. Furthermore, we employ a boundary loss function to optimize the segmentation structure area, reduce false detection areas, and improve model accuracy. Extensive experiments are carried out on the KITTI road dataset. The results demonstrate that, compared with state-of-the-art methods, DTRoadseg exhibits superior performance, achieving an average accuracy of 97.01%, a Recall of 96.35%, and runs at a speed of 0.09 s per picture.

DMRC-Net: Dual mechanism residual connection network for point cloud analysis

The purpose of point cloud analysis is to extract effective high-dimensional feature representations from point coordinates. However, existing methods only rely solely on global or local features. In this paper, we introduce the Dual Mechanism Residual Connection Network (DMRC-Net), a new end-to-end network architecture for more comprehensive representations. First, local features are encoded using graph convolution and fine-grained attention mechanisms, whereas global features are encoded using a multi-layer perceptron. Second, adaptive residual learning is designed to integrate two encoding results effectively. Furthermore, the structural information in local space is strengthened by collecting low-level geometric correlations. With these designs, DMRC-Net is able to capture both global and local details comprehensively. Experimental results show that our method performs well in point cloud classification, part segmentation, and semantic segmentation tasks.

Federated continual representation learning for evolutionary distributed intrusion detection in Industrial Internet of Things

As a promising paradigm, Federated Learning (FL)-based distributed intrusion detection offers potent protection for the network security of Industrial Internet of Things (IIoT) systems. Nonetheless, the practical deployment of IIoT systems occurs in a highly-complex and dynamic distributed environment. The ever-growing and dynamically-evolving cyber attacks will render the FL-based intrusion detection model inefficient, since FL cannot gracefully learn from the dynamic traffic data streams to identify new attacks. To address this issue, we propose the evolutionary distributed intrusion detection system based on federated continual representation learning, designed to continually capture effective feature representations of emerging attacks from dynamic traffic data streams. Specifically, we develop the supervised contrastive loss and the global information-aware regularization loss to alleviate the catastrophic forgetting on the previously observed attacks and mitigate the data heterogeneity across clients. Besides, we propose the prototype variance-based memory update strategy to ensure the effective memory replay data. Extensive experimental results demonstrate our proposed method outperforms the state-of-the-art methods by 13.3%–31.5% in terms of average accuracy on a real energy intrusion detection dataset.

Prediction of oxygen content in flue gas of power plant with stacked target-enhanced autoencoder and attention-based LSTM

Accurate and robust measurement of the oxygen content in the flue gas of power plant is significant to optimal the coal combustion process. Conventional hard measurement algorithms have low accuracy, short lifetime and high cost. Therefore, it is urgent to establish a precise and reliable soft measurement model. However, how to extract effective feature representations from complex process data is still the difficult and hot spot in the soft measurement field. The paper develops a novel soft measurement model, which integrates stacked target-enhanced autoencoder (STAE) with attention mechanism-based long short-term memory (ALSTM) in view of the above issues. The data cleaning strategy is firstly designed to filter the redundant data from the original data. Then STAE is developed to extract the useful features, which introduces target output into each autoencoder to ensure the feature representation is output-related and stacks several autoencoder units to learn high-level information. The extracted hierarchical features can not only represent the input data, but also have high correlation with the target output. A three-layer-based attention network is designed to allocate different weights on the different levels’ features. Finally, due to the dynamic characteristic of the problem, a LSTM-based prediction network is dexterously designed to learn the potential relation from extracted features to target output, i.e., oxygen content. Numerous real-word data is collected from a 300 MW power plant and extensive experiments are conducted. The maximum prediction errors of A and B are 0.42 and 0.41. The 95.1 % prediction errors of A side are within [−0.2, 0.2], and the 96.2 % prediction errors of B side are within [−0.2, 0.2].

Salient object detection in low-light RGB-T scene via spatial-frequency cues mining

Low-light conditions pose significant challenges to vision tasks, such as salient object detection (SOD), due to insufficient photons. Light-insensitive RGB-T SOD models mitigate the above problems to some extent, but they are limited in performance as they only focus on spatial feature fusion while ignoring the frequency discrepancy. To this end, we propose an RGB-T SOD model by mining spatial-frequency cues, called SFMNet, for low-light scenes. Our SFMNet consists of spatial-frequency feature exploration (SFFE) modules and spatial-frequency feature interaction (SFFI) modules. To be specific, the SFFE module aims to separate spatial-frequency features and adaptively extract high and low-frequency features. Moreover, the SFFI module integrates cross-modality and cross-domain information to capture effective feature representations. By deploying both modules in a top-down pathway, our method generates high-quality saliency predictions. Furthermore, we construct the first low-light RGB-T SOD dataset as a benchmark for evaluating performance. Extensive experiments demonstrate that our SFMNet can achieve higher accuracy than the existing models for low-light scenes.

FusPB-ESM2: Fusion model of ProtBERT and ESM-2 for cell-penetrating peptide prediction

Cell-penetrating peptides have attracted much attention for their ability to break through cell membrane barriers, which can improve drug bioavailability, reduce side effects, and promote the development of gene therapy. Traditional wet-lab prediction methods are time-consuming and costly, and computational methods provide a short-time and low-cost alternative. Still, the accuracy and reliability need to be further improved. To solve this problem, this study proposes a feature fusion-based prediction model, where the protein pre-trained language models ProtBERT and ESM-2 are used as feature extractors, and the extracted features from both are fused to obtain a more comprehensive and effective feature representation, which is then predicted by linear mapping. Validated by many experiments on public datasets, the method has an AUC value as high as 0.983 and shows high accuracy and reliability in cell-penetrating peptide prediction.

Cross-modal change detection flood extraction based on self-supervised contrastive pre-training

Abstract. Flood extraction is a critical issue in remote sensing analysis. Accurate flood extraction faces challenges such as complex scenes, image differences across modalities, and a shortage of labeled samples. Traditional supervised deep learning algorithms demonstrate promising prospects in flood extraction. They mostly rely on abundant labeled data. However, in practical applications, there is a scarcity of available labeled samples for flood change regions, leading to an expensive acquisition of such data for flood extraction. In contrast, there is a wealth of unlabeled data in remote sensing images. Self-supervised contrastive learning (SSCL) provides a solution, allowing learning from unlabeled data without explicit labels. Inspired by SSCL, we utilized the open-source CAU-Flood dataset and developed a framework for cross-modal change detection in flood extraction (CMCDFE). We employed the Barlow Twin (BT) SSCL algorithm to learn effective visual feature representations of flood change regions from unlabeled cross-modal bi-temporal remote sensing data. Subsequently, these well-initialized weight parameters were transferred to the task of flood extraction, achieving optimal accuracy. We introduced the improved CS-DeepLabV3+ network for extracting flood change regions from cross-modal bi-temporal remote sensing data, incorporating the CBAM dual attention mechanism. By demonstrating on the CAU-Flood dataset, we proved that fine-tuning with only a pre-trained encoder can surpass widely used ImageNet pre-training methods without additional data. This approach effectively addresses downstream cross-modal change detection flood extraction tasks.

Drug-target Interaction Prediction By Combining Transformer and Graph Neural Networks

Background: The prediction of drug-target interactions (DTIs) plays an essential role in drug discovery. Recently, deep learning methods have been widely applied in DTI prediction. However, most of the existing research does not fully utilize the molecular structures of drug compounds and the sequence structures of proteins, which makes these models unable to obtain precise and effective feature representations. Methods: In this study, we propose a novel deep learning framework combining transformer and graph neural networks for predicting DTIs. Our model utilizes graph convolutional neural networks to capture the global and local structure information of drugs, and convolutional neural networks are employed to capture the sequence feature of targets. In addition, the obtained drug and protein representations are input to multi-layer transformer encoders, respectively, to integrate their features and generate final representations. Results: The experiments on benchmark datasets demonstrated that our model outperforms previous graph-based and transformer-based methods, with 1.5% and 1.8% improvement in precision and 0.2% and 1.0% improvement in recall, respectively. The results indicate that the transformer encoders effectively extract feature information of both drug compounds and proteins. Conclusion: Overall, our proposed method validates the applicability of combining graph neural networks and transformer architecture in drug discovery, and due to the attention mechanisms, it can extract deep structure feature data of drugs and proteins.

A Continuous Non-Invasive Blood Pressure Prediction Method Based on Deep Sparse Residual U-Net Combined with Improved Squeeze and Excitation Skip Connections.

Arterial blood pressure (ABP) serves as a pivotal clinical metric in cardiovascular health assessments, with the precise forecasting of continuous blood pressure assuming a critical role in both preventing and treating cardiovascular diseases. This study proposes a novel continuous non-invasive blood pressure prediction model, DSRUnet, based on deep sparse residual U-net combined with improved SE skip connections, which aim to enhance the accuracy of using photoplethysmography (PPG) signals for continuous blood pressure prediction. The model first introduces a sparse residual connection approach for path contraction and expansion, facilitating richer information fusion and feature expansion to better capture subtle variations in the original PPG signals, thereby enhancing the network's representational capacity and predictive performance and mitigating potential degradation in the network performance. Furthermore, an enhanced SE-GRU module was embedded in the skip connections to model and weight global information using an attention mechanism, capturing the temporal features of the PPG pulse signals through GRU layers to improve the quality of the transferred feature information and reduce redundant feature learning. Finally, a deep supervision mechanism was incorporated into the decoder module to guide the lower-level network to learn effective feature representations, alleviating the problem of gradient vanishing and facilitating effective training of the network. The proposed DSRUnet model was trained and tested on the publicly available UCI-BP dataset, with the average absolute errors for predicting systolic blood pressure (SBP), diastolic blood pressure (DBP), and mean blood pressure (MBP) being 3.36 ± 6.61 mmHg, 2.35 ± 4.54 mmHg, and 2.21 ± 4.36 mmHg, respectively, meeting the standards set by the Association for the Advancement of Medical Instrumentation (AAMI), and achieving Grade A according to the British Hypertension Society (BHS) Standard for SBP and DBP predictions. Through ablation experiments and comparisons with other state-of-the-art methods, the effectiveness of DSRUnet in blood pressure prediction tasks, particularly for SBP, which generally yields poor prediction results, was significantly higher. The experimental results demonstrate that the DSRUnet model can accurately utilize PPG signals for real-time continuous blood pressure prediction and obtain high-quality and high-precision blood pressure prediction waveforms. Due to its non-invasiveness, continuity, and clinical relevance, the model may have significant implications for clinical applications in hospitals and research on wearable devices in daily life.

Transfer-Aware Graph U-Net with Cross-Level Interactions for PolSAR Image Semantic Segmentation

Although graph convolutional networks have found application in polarimetric synthetic aperture radar (PolSAR) image classification tasks, the available approaches cannot operate on multiple graphs, which hinders their potential to generalize effective feature representations across different datasets. To overcome this limitation and achieve robust PolSAR image classification, this paper proposes a novel end-to-end cross-level interaction graph U-Net (CLIGUNet), where weighted max-relative spatial convolution is proposed to enable simultaneous learning of latent features from batch input. Moreover, it integrates weighted adjacency matrices, derived from the symmetric revised Wishart distance, to encode polarimetric similarity into weighted max-relative spatial graph convolution. Employing end-to-end trainable residual transformers with multi-head attention, our proposed cross-level interactions enable the decoder to fuse multi-scale graph feature representations, enhancing effective features from various scales through a deep supervision strategy. Additionally, multi-scale dynamic graphs are introduced to expand the receptive field, enabling trainable adjacency matrices with refined connectivity relationships and edge weights within each resolution. Experiments undertaken on real PolSAR datasets show the superiority of our CLIGUNet with respect to state-of-the-art networks in classification accuracy and robustness in handling unknown imagery with similar land covers.

Self supervised learning based emotion recognition using physiological signals.

The significant role of emotional recognition in the field of human-machine interaction has garnered the attention of many researchers. Emotion recognition based on physiological signals can objectively reflect the most authentic emotional states of humans. However, existing labeled Electroencephalogram (EEG) datasets are often of small scale. In practical scenarios, a large number of unlabeled EEG signals are easier to obtain. Therefore, this paper adopts self-supervised learning methods to study emotion recognition based on EEG. Specifically, experiments employ three pre-defined tasks to define pseudo-labels and extract features from the inherent structure of the data. Experimental results indicate that self-supervised learning methods have the capability to learn effective feature representations for downstream tasks without any manual labels.

Trish: an efficient activation function for CNN models and analysis of its effectiveness with optimizers in diagnosing glaucoma

Glaucoma is an eye disease that spreads over time without showing any symptoms at an early age and can result in vision loss in advanced ages. The most critical issue in this disease is to detect the symptoms of the disease at an early age. Various researches are carried out on machine learning approaches that will provide support to the expert for this diagnosis. The activation function plays a pivotal role in deep learning models, as it introduces nonlinearity, enabling neural networks to learn complex patterns and relationships within data, thus facilitating accurate predictions and effective feature representations. In this study, it is focused on developing an activation function that can be used in CNN architectures using glaucoma disease datasets. The developed function (Trish) was compared with ReLU, LReLU, Mish, Swish, Smish, and Logish activation functions using SGD, Adam, RmsProp, AdaDelta, AdaGrad, Adamax, and Nadam optimizers in CNN architectures. Datasets consisting of retinal fundus images named ACRIMA and HRF were used within the scope of the experiments. These datasets are widely known and currently used in the literature. To strengthen the test validity, the proposed function was also tested on the CIFAR-10 dataset. As a result of the study, 97.22% validation accuracy performance was obtained. It should be stated that the acquired performance value is at a significant level for the detection of glaucoma.

TOP-ReID: Multi-Spectral Object Re-identification with Token Permutation

Multi-spectral object Re-identification (ReID) aims to retrieve specific objects by leveraging complementary information from different image spectra. It delivers great advantages over traditional single-spectral ReID in complex visual environment. However, the significant distribution gap among different image spectra poses great challenges for effective multi-spectral feature representations. In addition, most of current Transformer-based ReID methods only utilize the global feature of class tokens to achieve the holistic retrieval, ignoring the local discriminative ones. To address the above issues, we step further to utilize all the tokens of Transformers and propose a cyclic token permutation framework for multi-spectral object ReID, dubbled TOP-ReID. More specifically, we first deploy a multi-stream deep network based on vision Transformers to preserve distinct information from different image spectra. Then, we propose a Token Permutation Module (TPM) for cyclic multi-spectral feature aggregation. It not only facilitates the spatial feature alignment across different image spectra, but also allows the class token of each spectrum to perceive the local details of other spectra. Meanwhile, we propose a Complementary Reconstruction Module (CRM), which introduces dense token-level reconstruction constraints to reduce the distribution gap across different image spectra. With the above modules, our proposed framework can generate more discriminative multi-spectral features for robust object ReID. Extensive experiments on three ReID benchmarks (i.e., RGBNT201, RGBNT100 and MSVR310) verify the effectiveness of our methods. The code is available at https://github.com/924973292/TOP-ReID.

SEFRNet: An SAR Ship Target Detection Network With Effective Feature Representation

SEFRNet: An SAR Ship Target Detection Network With Effective Feature Representation

Prior based Pyramid Residual Clique Network for human body image super-resolution

Recent research in the analysis of human images, such as human parsing and pose estimation, usually requires input images to have a sufficiently high resolution. However, small images of people are commonly encountered in our daily lives, particularly in surveillance applications. This paper aims to ultra-resolve a tiny person image to its high-resolution counterpart by learning effective feature representations and exploiting useful human body prior knowledge. First, we propose the Residual Clique Block (RCB) to fully exploit compact feature representations for image Super-Resolution (SR). Second, a series of RCBs are cascaded in a coarse-to-fine manner to construct the Pyramid Residual Clique Network (PRCN), which simultaneously reconstructs multiple SR results (e.g. 2×, 4×, and 8×) in one feed-forward pass. Third, we utilize the human parsing map as the shape prior, and the high-frequency sub-bands of Uniform Discrete Curvelet Transform (UDCT) as the texture prior to enhance the details of reconstructed human body image. Experimental results demonstrate that our proposed method achieves state-of-the-art performance with superior visual quality and PSNR/SSIM scores. Moreover, we show that our results can considerably enhance the performance of human parsing and pose estimation tasks.

Identifying Antitubercular Peptides via Deep Forest Architecture with Effective Feature Representation.

Tuberculosis (TB) is a severe disease caused by Mycobacterium tuberculosis that poses a significant threat to human health. The emergence of drug-resistant strains has made the global fight against TB even more challenging. Antituberculosis peptides (ATPs) have shown promising results as a potential treatment for TB. However, conventional wet lab-based approaches to ATP discovery are time-consuming and costly and often fail to discover peptides with desired properties. To address these challenges, we propose a novel machine learning-based framework called ATPfinder that can significantly accelerate the discovery of ATP. Our approach integrates various efficient peptide descriptors and utilizes the deep forest algorithm to construct the model. This neural network-like cascading structure can effectively process and mine features without complex hyperparameter tuning. Our experimental results show that ATPfinder outperforms existing ATP prediction tools, achieving state-of-the-art performance with an accuracy of 89.3% and an MCC of 0.70. Moreover, our framework exhibits better robustness than baseline algorithms commonly used for other sequence analysis tasks. Additionally, the excellent interpretability of our model can assist researchers in understanding the critical features of ATP. Finally, we developed a downloadable desktop application to simplify the use of our framework for researchers. Therefore, ATPfinder can facilitate the discovery of peptide drugs and provide potential solutions for TB treatment. Our framework is freely available at https://github.com/lantianyao/ATPfinder/ (data sets and code) and https://awi.cuhk.edu.cn/dbAMP/ATPfinder.html (software).

Large-Scale Spatial–Temporal Identification of Urban Vacant Land and Informal Green Spaces Using Semantic Segmentation

The large-scale identification of urban vacant land (UVL) and informal green spaces (IGSs) using conventional identification methods is challenged by the high cost of time and resources as well as inconsistent outcomes. Moreover, the spatial–temporal changes of UVL and IGSs have received limited academic attention. We introduce a methodological framework for the large-scale automatic identification of UVL and IGSs in Hangzhou, China, based on semantic segmentation. We construct and release a large-scale dataset for UVL identification, containing five different UVL categories, and one of them is IGS, with 3096 patches for training and 128 patches for evaluation. We then train five different semantic segmentation networks using the dataset and utilise Segformer to predict UVL and IGSs within the whole urban area of Hangzhou. The presented segmentation model has a hierarchically structured Transformer encoder and a multilayer perceptron decoder, which incorporates local and global information to obtain effective feature representations. Extensive experiments have been conducted to evaluate the segmentation performance of the applied Segformer. Experimental results show the good identification performance of Segformer for UVL and IGSs. Results also verify that the proposed identification framework can be effectively used to analyse the spatial–temporal changes of UVL and IGSs in Hangzhou.

Crowd Counting Based on Multiscale Spatial Guided Perception Aggregation Network.

Crowd counting has received extensive attention in the field of computer vision, and methods based on deep convolutional neural networks (CNNs) have made great progress in this task. However, challenges such as scale variation, nonuniform distribution, complex background, and occlusion in crowded scenes hinder the performance of these networks in crowd counting. In order to overcome these challenges, this article proposes a multiscale spatial guidance perception aggregation network (MGANet) to achieve efficient and accurate crowd counting. MGANet consists of three parts: multiscale feature extraction network (MFEN), spatial guidance network (SGN), and attention fusion network (AFN). Specifically, to alleviate the scale variation problem in crowded scenes, MFEN is introduced to enhance the scale adaptability and effectively capture multiscale features in scenes with drastic scale variation. To address the challenges of nonuniform distribution and complex background in population, an SGN is proposed. The SGN includes two parts: the spatial context network (SCN) and the guidance perception network (GPN). SCN is used to capture the detailed semantic information between the multiscale feature positions extracted by MFEN, and improve the ability of deep structured information exploration. At the same time, the dependence relationship between the spatial remote context is established to enhance the receptive field. GPN is used to enhance the information exchange between channels and guide the network to select appropriate multiscale features and spatial context semantic features. AFN is used to adaptively measure the importance of the above different features, and obtain accurate and effective feature representations from them. In addition, this article proposes a novel region-adaptive loss function, which optimizes the regions with large recognition errors in the image, and alleviates the inconsistency between the training target and the evaluation metric. In order to evaluate the performance of the proposed method, extensive experiments were carried out on challenging benchmarks including ShanghaiTech Part A and Part B, UCF-CC-50, UCF-QNRF, and JHU-CROWD ++ . Experimental results show that the proposed method has good performance on all four datasets. Especially on ShanghaiTech Part A and Part B, CUCF-QNRF, and JHU-CROWD ++ datasets, compared with the state-of-the-art methods, our proposed method achieves superior recognition performance and better robustness.

Reconstructed Graph Neural Network With Knowledge Distillation for Lightweight Anomaly Detection.

The proliferation of Internet-of-Things (IoT) technologies in modern smart society enables massive data exchange for offering intelligent services. It becomes essential to ensure secure communications while exchanging highly sensitive IoT data efficiently, which leads to high demands for lightweight models or algorithms with limited computation capability provided by individual IoT devices. In this study, a graph representation learning model, which seamlessly incorporates graph neural network (GNN) and knowledge distillation (KD) techniques, named reconstructed graph with global-local distillation (RG-GLD), is designed to realize the lightweight anomaly detection across IoT communication networks. In particular, a new graph network reconstruction strategy, which treats data communications as nodes in a directed graph while edges are then connected according to two specifically defined rules, is devised and applied to facilitate the graph representation learning in secure and efficient IoT communications. Both the structural and traffic features are then extracted from the graph data and flow data respectively, based on the graph attention network (GAT) and multilayer perceptron (MLP) techniques. These can benefit the GNN-based KD process in accordance with the more effective feature fusion and representation, considering both structural and data levels across the dynamic IoT networks. Furthermore, a lightweight local subgraph preservation mechanism improved by the graph attention mechanism and downsampling scheme to better utilize the topological information, and a so-called global information alignment defined based on the self-attention mechanism to effectively preserve the global information, are developed and incorporated in a refined graph attention based KD scheme. Compared with four different baseline methods, experiments and evaluations conducted based on two public datasets demonstrate the usefulness and effectiveness of our proposed model in improving the efficiency of knowledge transfer with higher classification accuracy but lower computational load, which can be deployed for lightweight anomaly detection in sustainable IoT computing environments.

HybAVPnet: a Novel Hybrid Network Architecture for Antiviral Peptides Prediction.

Viruses pose a great threat to human production and life, thus the research and development of antiviral drugs is urgently needed. Antiviral peptides play an important role in drug design and development. Compared with the time-consuming and laborious wet chemical experiment methods, it is critical to use computational methods to predict antiviral peptides accurately and rapidly. However, due to limited data, accurate prediction of antiviral peptides is still challenging and extracting effective feature representations from sequences is crucial for creating accurate models. This study introduces a novel two-step approach, named HybAVPnet, to predict antiviral peptides with a hybrid network architecture based on neural networks and traditional machine learning methods. We adopted a stacking-like structure to capture both the long-term dependencies and local evolution information to achieve a comprehensive and diverse prediction using the predicted labels and probabilities. Using an ensemble technique with the different kinds of features can reduce the variance without increasing the bias. The experimental result shows HybAVPnet can achieve better and more robust performance compared with the state-of-the-art methods, which makes it useful for the research and development of antiviral drugs. Meanwhile, it can also be extended to other peptide recognition problems because of its generalization ability.