Feature Representation Method Research Articles

A Novel Source Code Representation Approach Based on Multi-Head Attention

Code classification and code clone detection are crucial for understanding and maintaining large software systems. Although deep learning surpasses traditional techniques in capturing the features of source code, existing models suffer from low processing power and high complexity. We propose a novel source code representation method based on the multi-head attention mechanism (SCRMHA). SCRMHA captures the vector representation of entire code segments, enabling it to focus on different positions of the input sequence, capture richer semantic information, and simultaneously process different aspects and relationships of the sequence. Moreover, it can calculate multiple attention heads in parallel, speeding up the computational process. We evaluate SCRMHA on both the standard dataset and an actual industrial dataset, and analyze the differences between these two datasets. Experiment results in code classification and clone detection tasks show that SCRMHA consumes less time and reduces complexity by about one-third compared with traditional source code feature representation methods. The results demonstrate that SCRMHA reduces the computational complexity and time consumption of the model while maintaining accuracy.

CoCo-ST: Comparing and Contrasting Spatial Transcriptomics data sets using graph contrastive learning.

Traditional feature dimension reduction methods have been widely used to uncover biological patterns or structures within individual spatial transcriptomics data. However, these methods are designed to yield feature representations that emphasize patterns or structures with dominant high variance, such as the normal tissue spatial pattern in a precancer setting. Consequently, they may inadvertently overlook patterns of interest that are potentially masked by these high-variance structures. Herein we present our graph contrastive feature representation method called CoCo-ST (Comparing and Contrasting Spatial Transcriptomics) to overcome this limitation. By incorporating a background data set representing normal tissue, this approach enhances the identification of interesting patterns in a target data set representing precancerous tissue. Simultaneously, it mitigates the influence of dominant common patterns shared by the background and target data sets. This enables discerning biologically relevant features crucial for capturing tissue-specific patterns, a capability we showcased through the analysis of serial mouse precancerous lung tissue samples.

Research on Computer Artificial Intelligence Algorithm to Optimize Physical Training Intensity and Recovery Cycle Model

This paper established an association model between high-load training and cardiac damage to minimize sports injuries. It is a key condition to promote the development of modern sports. This project intends to use 8 biochemical parameters related to high-load training to build a mathematical model of high-load training-myocardial damage. A mathematical model for high load training and cardiac damage was established. Firstly, based on sparse feature representation method, the biochemical mechanism model of high-intensity training was constructed. The energy supply law of intramuscular energy supply and the dynamic law of glycolysis energy supply during exercise were extracted. Eight biochemical parameters were obtained after high intensity training. To construct an accurate association between high-intensity training-heart damage by measuring the correlation between eight indicators of performance. The validity of the model is verified by simulation.

Heart Sound Signals Classification with Image Conversion Employed

The number of patients with cardiovascular diseases worldwide is increasing rapidly, while medical resources are increasingly scarce. Heart sound classification, as the most direct means of discovering cardiovascular diseases, is attracting the attention of researchers around the world. Although great progress has been made in heart sound classification in recent years, most of them are based on traditional statistical feature methods and temporal dimension features. These traditional temporal dimension feature representation and classification methods cannot achieve good classification accuracy. This paper proposes a new partition attention module and Fusionghost module, and the entire network framework is named PANet. Without segmentation of the heart sound signal, the heart sound signal is converted into a bispectrum and input into the proposed framework for feature extraction and classification tasks. The network makes full use of multi-scale feature extraction and feature map fusion, improving the network feature extraction ability. This paper conducts a comprehensive study of the performance of different network parameters and different module numbers, and compares the performance with the most advanced algorithms currently available. Experiments have shown that for two classification problems (normal or abnormal), the classification accuracy rate on the 2016 PhysioNet/CinC Challenge database reached 97.89%, the sensitivity was 96.96%, and the specificity was 98.85%.

A Curb-Detection Network with a Tri-Plane BEV Encoder Module for Autonomous Delivery Vehicles

Curb detection tasks play a crucial role in the perception of the autonomous driving environment for logistics vehicles. With the popularity of multi-modal sensors under the BEV (Bird’s Eye View) paradigm, curb detection tasks are increasingly being integrated into multi-task perception networks, achieving robust detection results. This paper modifies and integrates the tri-plane spatial feature representation method of the EG3D network from the field of 3D reconstruction into a BEV-based multi-modal sensor detection network, including LiDAR, pinhole cameras, and fisheye cameras. The system collects a total of 24,350 frames of data under real road conditions for experimentation, proving the effectiveness of the proposed method.

BiPSTP: Sequence feature encoding method for identifying different RNA modifications with bidirectional position-specific trinucleotides propensities

RNA modification, a post-transcriptional regulatory mechanism, significantly influences RNA biogenesis and function. The accurate identification of modification sites is paramount for investigating their biological implications. Methods for encoding RNA sequence into numerical data play a crucial role in developing robust models for predicting modification sites. However, existing techniques suffer from limitations, including inadequate information representation, challenges in effectively integrating positional and sequential information, and the generation of irrelevant or redundant features when combining multiple approaches. These deficiencies hinder the effectiveness of machine learning models in addressing the performance challenges associated with predicting RNA modification sites. Here we introduce a novel RNA sequence feature representation method, named BiPSTP, which utilizes bidirectional trinucleotide position-specific propensities. We employ the parameter ξ to denote the interval between the current nucleotide and its adjacent forward or backward dinucleotide, enabling the extraction of positional and sequential information from RNA sequences. Leveraging the BiPSTP method, we have developed the prediction model mRNAPred using support vector machine (SVM) classifier to identify multiple types of RNA modification sites. We evaluate the performance of our BiPSTP method and mRNAPred model across 12 distinct RNA modification types. Our experimental results demonstrate the superiority of the mRNAPred model compared to state-of-art models in the domain of RNA modification sites identification. Importantly, our BiPSTP method enhances the robustness and generalization performance of prediction models. Notably, it can be applied to feature extraction from DNA sequences to predict other biological modification sites.

Multimodal representation learning for tourism recommendation with two-tower architecture.

Personalized recommendation plays an important role in many online service fields. In the field of tourism recommendation, tourist attractions contain rich context and content information. These implicit features include not only text, but also images and videos. In order to make better use of these features, researchers usually introduce richer feature information or more efficient feature representation methods, but the unrestricted introduction of a large amount of feature information will undoubtedly reduce the performance of the recommendation system. We propose a novel heterogeneous multimodal representation learning method for tourism recommendation. The proposed model is based on two-tower architecture, in which the item tower handles multimodal latent features: Bidirectional Long Short-Term Memory (Bi-LSTM) is used to extract the text features of items, and an External Attention Transformer (EANet) is used to extract image features of items, and connect these feature vectors with item IDs to enrich the feature representation of items. In order to increase the expressiveness of the model, we introduce a deep fully connected stack layer to fuse multimodal feature vectors and capture the hidden relationship between them. The model is tested on the three different datasets, our model is better than the baseline models in NDCG and precision.

A Multi-classification Accessment Framework for Reproducible Evaluation of Multimodal Learning in Alzheimer's Disease.

Multimodal learning is widely used in automated early diagnosis of Alzheimer's disease. However, the current studies are based on an assumption that different modalities can provide more complementary information to help classify the samples from the public dataset Alzheimer's Disease Neuroimaging Initiative (ADNI). In addition, the combination of modalities and different tasks are external factors that affect the performance of multimodal learning. Above all, we summrise three main problems in the early diagnosis of Alzheimer's disease: (i) unimodal vs multimodal; (ii) different combinations of modalities; (iii) classification of different tasks. In this paper, to experimentally verify these three problems, a novel and reproducible multi-classification framework for Alzheimer's disease early automatic diagnosis is proposed to evaluate and verify the above issues. The multi-classification framework contains four layers, two types of feature representation methods, and two types of models to verify these three issues. At the same time, our framework is extensible, that is, it is compatible with new modalities generated by new technologies. Following that, a series of experiments based on the ADNI-1 dataset are conducted and some possible explanations for the early diagnosis of Alzheimer's disease are obtained through multimodal learning. Experimental results show that SNP has the highest accuracy rate of 57.09% in the early diagnosis of Alzheimer's disease. In the modality combination, the addition of Single Nucleotide Polymorphism modality improves the multi-modal machine learning performance by 3% to 7%. Furthermore, we analyse and discuss the most related Region of Interest and Single Nucleotide Polymorphism features of different modalities.

A Novel Feature Representation Method Based on Distances Between Statistical Matrices of Acoustic Emission Waveforms

A Novel Feature Representation Method Based on Distances Between Statistical Matrices of Acoustic Emission Waveforms

An adaptive cepstrum feature representation method with variable frame length and variable filter banks for acoustic emission signals

Acoustic emission technology is widely employed in defect detection, structural health monitoring, fault diagnosis, and other applications due to its benefits of non-contact, real-time, sensitivity, and flexibility. To implement efficient and timely intelligent monitoring, it is crucial to promptly and precisely gather the time-varying properties in the non-stationary acoustic emission signals. The current work proposes an adaptive cepstrum feature representation method that combines variable frame length and variable filter bank. The variable frame length is implemented based on the spectral distribution correlation, and the optimal frame length and frame rate are selected by iterative operations. The variable filter bank, on the other hand, characterizes the local measure features of variable frame length signals by setting dynamic weight distributions. The proposed adaptive cepstrum algorithm can be used as an auxiliary tool to initially analyze complex non-stationary signals, and thus effectively identify the time-varying and frequency components in acoustic emission signals. The superiority of the proposed method has been confirmed on simulation data. Furthermore, the practical performance of the proposed method is evaluated and illustrated based on the burst signal from laser shock peening processing and the mixed signal from pipeline leakage. The proposed method can be extended to other broadband signals and can be used for feature pre-processing before deep learning network modeling to improve recognition accuracy due to its high time–frequency resolution.

Sparsity-guided Discriminative Feature Encoding for Robust Keypoint Detection

Existing handcrafted keypoint detectors typically focus on designing specific local structures manually while ignoring whether they have enough flexibility to explore diverse visual patterns in an image. Despite the advancement of learning-based approaches in the past few years, most of them still rely on the availability of the outputs of handcrafted detectors as a part of training. In fact, such dependence limits their ability to discover various visual information. Recently, semi-handcrafted methods based on sparse coding have emerged as a promising paradigm to alleviate the above issue. However, the visual relationships between feature points have not been considered in the encoding stage, which may weaken the discriminative capability of feature representations for keypoint recognition. To tackle this problem, we propose a novel sparsity-guided discriminative feature representation (SDFR) method that attempts to explore the intrinsic correlations of keypoint candidates, thus ensuring the validity of characterizing distinctive and diverse structural information. Specifically, we first incorporate an affinity constraint into the feature representation objective, which jointly encodes all the patches in an image while highlighting the similarities and differences between them. Meanwhile, a smoother sparsity regularization with the Frobenius norm is leveraged to further preserve the similarity relationships of patch representations. Due to the differentiable property of this sparsity, SDFR is computationally feasible and effective for representing dense patches. Finally, we treat the SDFR model as multiple optimization sub-problems and introduce an iterative solver. During comprehensive evaluations on five challenging benchmarks, the proposed method achieves favorable performances compared with the state of the art in the literature.

Hyperspectral image band selection method combining the variation degree of salient features

ABSTRACTBand selection (BS) work for hyperspectral remote sensing images (HRSIs) has attracted increasing attention from researchers. In the HRSI, the salient features of each band always include the more important discriminative features. However, most existing BS methods always have difficulty in simultaneously performing local-global consistency analysis on salient features in bands of HRSIs. In addition, these methods also ignore the consideration of the diversity of band information to some extent when selecting a band subset. To tackle these problems, a novel unsupervised hyperspectral BS method combining the variation degree of salient features (UBSF) is proposed in this article. UBSF first promotes the singular value decomposition method to analyse the salient features in bands and develops a new low-dimensional local-global salient feature representation (NLR) method for discriminative feature representations of the bands. The NLR method combines the variation degree of the features of the significant representative patches of each band in their corresponding significant feature subspaces to achieve a local-global consistency analysis of salient features of each band of HRSIs. Next, UBSF proposes a new two-stage strategy (TSS) to preserve the diversity of BS results as much as possible. In TSS, we propose a novel similarity criterion, which can make the highly similar band subsets with near information amount in the overall bands adaptively compressed according to the obtained similarity results of bands further. Then, the band subset with the most information is selected as the final BS results in the obtained reduced-dimensional HRSI. Finally, the validity of the UBSF method is verified based on three measured HRSI datasets. The experimental results show that the proposed UBSF is superior to other state-of-the-art BS methods.

Fake news detection: Taxonomy and comparative study

The proliferation of social networks has presented a significant challenge in combating the pervasive issue of fake news within modern societies. Due to the large amount of information and news produced daily in text, audio, and video, the validation and verification of this information have become crucial tasks. Leveraging advancements in artificial intelligence, distinguishing between fake news and factual information through automatic fake news detection systems has become more feasible. Automatic fake news detection has been explored from diverse perspectives, employing various feature extraction and classification models. Nonetheless, empirical evaluations, categorization, and comparisons of existing techniques for handling this problem remain limited. In this paper, we revisit the definitions and perspectives of fake news and propose an updated taxonomy for the field based on multiple criteria: (1) Type of features used in fake news detection; (2) Fake news detection perspectives; (3) Feature representation methods; and (4) Classification approaches. Moreover, we conduct an extensive empirical study to evaluate several feature representation techniques and classification approaches based on accuracy and computational cost. Our experimental results demonstrate that the optimal feature extraction techniques vary depending on the characteristics of the dataset. Notably, context-dependent models based on transformer models consistently exhibit superior performance. Additionally, employing transformer models as feature extraction methods, rather than solely fine-tuning the network for the downstream task, improves overall performance. Through extensive error analysis, we identify that a combination of feature representation methods and classification algorithms, including classical ones, offer complementary aspects and should be considered for achieving better generalization performance while maintaining a relatively low computational cost. For further details, including source codes, figures, and datasets, please refer to our project’s GitHub repository: [https://github.com/FFarhangian/Fake-news-detection-Comparative-Study].

Similarity and Complementarity Attention-Based Graph Neural Networks for Mashup-Oriented Cloud API Recommendation

Mashups, which combine various web application programming interfaces (APIs) to implement some complex requirements, have grown to be a popular technique for developing service-oriented software. However, recommending suitable cloud APIs for mashup creation is challenging due to the rapidly increasing number of comparable APIs. Many existing mashup-oriented cloud API recommendations focus on functional similarity and ignore functional complementarity, which significantly impacts the accuracy of the recommendation results. Therefore, this paper proposed a feature representation and recommendation method for cloud APIs that fuses both similarity and complementarity. A heterogeneous information network of the cloud API ecosystem was constructed, and the neighbors, based on metapaths, were aggregated using a self-attention mechanism to generate the features of similarity and complementarity for the cloud APIs. Then, the mashup-related attention was utilized to fuse the two features, taking into consideration the varying preferences of different mashups towards similarity and complementarity features of cloud APIs. This fusion resulted in features that align cloud APIs with mashup requirements, which were employed to predict the probability of the mashup invoking a particular candidate cloud API. The proposed method was evaluated on a real dataset, and the results showed that it outperforms the baseline method and enhances the performance of mashup-oriented cloud API recommendations.

A survey of cross-modal pre-training

Pre-training allows autonomous learning of samples inherent semantics from unlabeled data in a large scale, and then obtains a task-independent general representation. Furthermore, the generalization of model encoding across different tasks can be improved. Inspired by early high-performance pre-trained models for a single modality of text or images, scholars began to explore image-text cross-modal pre-training methods, aiming to achieve the ability of understanding both visual information and textual semantics separately and achieve effective alignment between the two. This paper aims to summarize the development of image-text pre-training. Specifically, it summarizes the feature extraction and representation methods, pre-training tasks and extended downstream tasks used by various mainstream models. This paper also introduces and compares the mainstream models into two types of architectures: single-stream and dual-stream. Through the analysis, it can be concluded that the embedding representation determines the difficulty and performance of the subsequent fusion process, and the two kinds of architectures lead to different levels of fusion capabilities.

A Novel Feature Representation Method Based on Similarity Between Statistical Distributions of Acoustic Emission Waveforms

A Novel Feature Representation Method Based on Similarity Between Statistical Distributions of Acoustic Emission Waveforms

Manifold-based sparse representation for opinion mining

What the consumer thinks about an organization's products, services, and events is a crucial performance indicator for businesses. The brief opinion pieces were quickly published on websites and social media platforms and have been analyzed by machine learning methods. The classical text feature representation methods suffer from high dimensionality, sparsity, noisy, irrelevant and redundant information. This paper focuses on how to enhance feature representation for opinion mining. Some nonlinear feature selection methods based on manifold assumption have been exploited to resolve these problems. The inherent manifold configuration was commonly ascertained through a nearest neighbor graph, whereby the neighbors in the current techniques may exhibit diverse polarities. To alleviate this burden, it is proposed to exploit both manifold assumption and sparse property as prior knowledge for opinion representation to learn intrinsic structure from data. First, the graph representation of user reviews based on the mentioned prior knowledge is learned. Then, the spectral properties of the learned graph are exploited to present data in a new feature space. The proposed algorithm is applied to four various common input features on two benchmark datasets, the Internet Movie Database (IMDB) and the Amazon review dataset. Our experiments reveal that the proposed algorithm yields considerable enhancements in terms of F-measure, accuracy, and other standard performance measures compared to the combination of state-of-the-art features with various classifiers. The highest classification accuracies of 99.15 and 91.97 are obtained in the proposed method on IMDB and Amazon using a linear SVM classifier, respectively. The impact of the parameters of the proposed algorithm is also investigated in this paper. The incorporation of a sparse manifold-based representation has led to noteworthy advancements beyond the baseline, and this success serves to validate the underlying assumptions.

Temporal-frequency-phase feature classification using 3D-convolutional neural networks for motor imagery and movement.

Recently, convolutional neural networks (CNNs) have been widely applied in brain-computer interface (BCI) based on electroencephalogram (EEG) signals. Due to the subject-specific nature of EEG signal patterns and the multi-dimensionality of EEG features, it is necessary to employ appropriate feature representation methods to enhance the decoding accuracy of EEG. In this study, we proposed a method for representing EEG temporal, frequency, and phase features, aiming to preserve the multi-domain information of EEG signals. Specifically, we generated EEG temporal segments using a sliding window strategy. Then, temporal, frequency, and phase features were extracted from different temporal segments and stacked into 3D feature maps, namely temporal-frequency-phase features (TFPF). Furthermore, we designed a compact 3D-CNN model to extract these multi-domain features efficiently. Considering the inter-individual variability in EEG data, we conducted individual testing for each subject. The proposed model achieved an average accuracy of 89.86, 78.85, and 63.55% for 2-class, 3-class, and 4-class motor imagery (MI) classification tasks, respectively, on the PhysioNet dataset. On the GigaDB dataset, the average accuracy for 2-class MI classification was 91.91%. For the comparison between MI and real movement (ME) tasks, the average accuracy for the 2-class were 87.66 and 80.13% on the PhysioNet and GigaDB datasets, respectively. Overall, the method presented in this paper have obtained good results in MI/ME tasks and have a good application prospect in the development of BCI systems based on MI/ME.

Linear adversarial vector modeling based intrusion detection using feature subset selection and representation methods

Linear adversarial vector modeling based intrusion detection using feature subset selection and representation methods

Multiple temporal scale aggregation graph convolutional network for skeleton-based action recognition

Skeleton-based human action recognition is attracting increasing attention and is widely applied in virtual reality, human–computer interaction system and other cases. Nevertheless, the performance of recent methods on actions with similar appearance features is still barely satisfactory due to their inherent weakness in modeling the discriminative temporal dynamics. Besides, previous methods have limitation in mining the global information of action. To this end, a multiple temporal scale aggregation graph convolutional network is proposed. Firstly, taking advantage of the varying temporal resolutions offered by different layers in the graph convolutional network, we develop a multiple temporal scale aggregation module to extract discriminative temporal feature. Secondly, a new skeleton feature representation method termed as relative joint across frames is proposed, which provides more global structure clue than the absolute coordinates. Furthermore, we propose a five-stream structure that comprehensively models complementary features and eventually achieves a significant performance boost. In the empirical experiments, our method shows an improvement of 2.38% and 4.08% compared to the baseline method on the cross-subject evaluation benchmark of NTU-RGB+D 60 and NTU-RGB+D 120, respectively.