Final Representation Research Articles

A Semantically Driven Hybrid Network for Unsupervised Entity Alignment

The major challenge in the task of entity alignment (EA) lies in the heterogeneity of the knowledge graph. The traditional solution to EA is to first map entities to the same space via knowledge embedding and then calculate the similarity between entities from different knowledge graphs. However, these methods mainly rely on manually labeled seeds of EA, which limits their applicability. Some researchers have begun using pseudo-labels rather than seeds for unsupervised EA. However, directly using pseudo-labels causes new problems, such as noise in the pseudo-labels. In this article, we propose a model called the Semantically Driven Hybrid Network (SDHN) to reduce the impact of noise in the pseudo-labels on the performance of EA models. The SDHN consists of two modules: a Teacher–Student Network (TSN) and a Rotation and Penalty (RAP) module. The TSN module reduces the impact of noise in two ways: (1) The TSN’s teacher network guides its student network to construct pseudo-labels based on semantic information instead of directly creating pseudo-labels. (2) It adaptively fuses semantic information into student networks to improve the final representation of entity embedding. Finally, the TSN enhances the performance of models of entity alignment via the RAP module. The results of experiments on multiple benchmark datasets showed that the SDHN outperforms state-of-the-art models.

Enhancing review-based user representation on learned social graph for recommendation

In recent years, review-based methods have been widely used to learn user representations because reviews contain abundant information. However, few users would like to write reviews for items. The data sparsity problem in reviews has become an emerging challenge in the field. Meanwhile, other approaches resort to social information based on graph neural networks (GNNs) to augment user representation. However, the efficacy of these approaches is always jeopardized because social graphs are not available in most real-world scenarios. Therefore, we propose a new Enhancing Review-based User Representation Model on Learned Social Graph for Recommendation, named ERUR. Specifically, we first introduce a review encoder to model review-based user/item representations. Second, we design a graph learning network to learn social relations between users according to the review-based user representation. Third, a graph neural network is developed to augment the final user representation under the supervision of a generative adversarial network. It integrates user reviews and social relations to enrich the final user representation for recommendation and further alleviate the data sparsity problem in reviews. Finally, we conduct experiments on seven datasets to demonstrate the effectiveness of the ERUR model in user representation learning compared to the SOTA recommendation models.

Transformer-Based Domain-Specific Representation for Unsupervised Domain Adaptive Vehicle Re-Identification

Fully-supervised vehicle re-identification (re-ID) methods are faced with performance degradation when applied to new image domains. Therefore, developing unsupervised domain adaptation (UDA) to transfer the knowledge from learned source domain to new unlabeled target domain becomes an indispensable task. It is challenging because different domains have various image appearances, such as different backgrounds, illuminations and resolutions, especially when cameras have different viewpoints. To tackle this domain gap issue, a novel Transformer-based Domain-Specific Representation learning network (TDSR) is proposed to dynamically focus on corresponding detailed hints for each domain. Specifically, with the source and target domain being trained simultaneously, a domain encoding module is proposed to introduce domain information into the network. The original features of source and target domains are enriched with these domain encodings first, and then sequentially processed by a Transformer encoder to model contextual information and a decoder to summarize the encoded features into the final domain-specific feature representations. Moreover, we propose a Contrastive Clustering Loss (CCL) to directly optimize the distribution of features at cluster level. Instances are overall pulled closer to the prototype of the same identity, and pushed farther from the prototypes of different identities. It helps compact the clusters in the latent space and improve the discriminative capability of the network, leading to more accurate pseudo-label assignment in TDSR. Our method outperforms the state-of-the-art UDA methods on vehicle re-ID benchmark datasets VeRi and VehicleID on both real-world to real-world and synthetic to real-world settings.

KGNMDA: A Knowledge Graph Neural Network Method for Predicting Microbe-Disease Associations.

Accumulated studies discovered that various microbes in human bodies were closely related to complex human diseases and could provide new insight into drug development. Multiple computational methods were constructed to predict microbes that were potentially associated with diseases. However, most previous methods were based on single characteristics of microbes or diseases, that lacked important biological information related to microorganisms or diseases. Therefore, we constructed a knowledge graph centered on microorganisms and diseases from several existed databases to provide knowledgeable information for microbes and diseases. Then, we adopted a graph neural network method to learn representations of microbes and diseases from the constructed knowledge graph. After that, we introduced the Gaussian kernel similarity features of microbes and diseases to generate final representations of microbes and diseases. At last, we proposed a score function on final representations of microbes and diseases to predict scores of microbe-disease associations. Comprehensive experiments on the Human Microbe-Disease Association Database (HMDAD) dataset had demonstrated that our approach outperformed baseline methods. Furthermore, we implemented case studies on two important diseases (asthma and inflammatory bowel disease), the result demonstrated that our proposed model was effective in revealing the relationship between diseases and microbes. The source code of our model and the data were available on https://github.com/ChangzhiJiang/KGNMDA_master.

GA-ENs: A novel drug–target interactions prediction method by incorporating prior Knowledge Graph into dual Wasserstein Generative Adversarial Network with gradient penalty

Bipartite graph-based drug–target interactions (DTIs) prediction methods are commonly limited by the sparse structure of the graph, resulting in acquiring suboptimal node feature representations. In addition, these defective node features will also interfere with the representation quality of corresponding edge features. To alleviate the sparsity of bipartite graph and get better node representation, according to the prior Knowledge Graph (KG), we developed a novel prediction model based on Variational Graph Auto-Encoder (VGAE) combined with our proposed dual Wasserstein Generative Adversarial Network with gradient penalty strategy (dual-WGAN-GP) for generating edge information and augmenting their representations. Specifically, GA-ENs first utilized dual-WGAN-GP to fill possible edges by a prior KG containing various molecular associations knowledge when constructing a bipartite graph of known DTIs. Moreover, we utilized the KG transfer probability matrix to redefine the drug–drug and target–target similarity matrix, thus constructing the final graph adjacent matrix. Combining graph adjacent matrix with node features, we learn node representations by VGAE and augment them by utilizing dual-WGAN-GP again, thus obtaining final edge representations. Finally, a fully connected network with three layers was designed to predict potential DTIs. Extensive experiment results show that GA-ENs has excellent performance for DTIs prediction and could be a supplement tool for practical DTIs biological screening.

CKDAN: Content and keystroke dual attention networks with pre-trained models for continuous authentication

With the rapid development of digitalization, information security becomes more and more important. User authentication is a significant line of defense. During the past few years, many research works concentrating on user authentication have been published. Traditional authentication techniques (password, ID card-based authentication, or other biometrics like fingerprint or iris) basically fall into the category of one-time authentication. The disadvantage of these methods is that once an unknown user completes an authentication through forgery, the system would continue to operate without any resistance, thus putting the entire system at risk. Recently, due to the prevalence of deep learning techniques, some researchers have applied learning-based models to user authentication. However, almost all existing methods only focus on keystroke dynamics while ignoring the ”text” entered by users during keystrokes. In this paper, we propose contents and keystroke dual attention networks with pre-trained models for continuous authentication. To the best of our knowledge, our paper is the first to address user-inputted ”text” during keystrokes as an important asset beyond traditional characteristic keystroke dynamics. Specifically, we use the well-known pre-trained RoBERTa model to capture textual features. Then, we pass textual features and conventional features through our proposed dual attention networks. Our networks fuse these features and acquire final representations. Experiments show that the CKDAN model achieves state-of-the-art performance on two datasets, Clarkson II keystroke dataset and Buffalo dataset, outperforming all baseline methods.

ProposalVLAD with Proposal-Intra Exploring for Temporal Action Proposal Generation

Temporal action proposal generation aims to localize temporal segments of human activities in videos. Current boundary-based proposal generation methods can generate proposals with precise boundary but often suffer from the inferior quality of confidence scores used for proposal retrieving. In this article, we propose an effective and end-to-end action proposal generation method, named ProposalVLAD, with Proposal-Intra Exploring Network (PVPI-Net). We first propose a ProposalVLAD module to dynamically generate global features of the entire video, then we combine the global features and proposal local features to generate the final feature representations for all candidate proposals. Then, we design a novel Proposal-Intra Loss function (PI-Loss) to generate more reliable proposal confidence scores. Extensive experiments on large-scale and challenging datasets demonstrate the effectiveness of our proposed method. Experimental results show that our PVPI-Net achieves significant improvements on two benchmark datasets (i.e., THUMOS’14 and ActivityNet-1.3) and sets new records for temporal action detection task.

Graph convolutional network combining node similarity association and layer attention for personalized recommendation

Although current graph convolutional network (GCN) has achieved competitive performance in personalized recommendation systems, most of existing GCN based recommendation methods mainly rely on user–item interaction data and the fixed association weights of the nodes at different layers, which greatly limit them to further effectively learn the final embedding representation of nodes when interaction data is scarce. This paper proposes a GCN based recommendation model combining node similarity association and layer attention mechanism (NSAGCN) for predicting user–item interactions in personalized recommendation. The proposed NSAGCN model integrates the similarity associations of the same type nodes into a heterogeneous network based on the bipartite graph of user–item interaction to enrich semantic information of the original sparse interaction graph and more effectively learn the node embedding features. In addition, a layer attention strategy is used to aggregate the embeddings from different graph convolutional layers with various association weights according to the proximity to the target nodes. Extensive experiments on three public benchmark datasets (ML-100K, ML-1M, and Book-Crossing) show that the proposed NSAGCN model outperforms state-of-the-art models by an average improvement of 8.58%, 6.91%, and 6.33% in Recall, Precision, and Normalized Discounted Cumulative Gain (NDCG), respectively.

SDTF-Net: Static and dynamic time–frequency network for Speech Emotion Recognition

Speech Emotion Recognition (SER) is an important but challenging research topic in Human–Computer interaction (HCI). As a crucial component influencing the model’s performance, in SER, feature extraction needs to comprehensively consider the static local states, the dynamic change tendency, and the time–frequency characteristics of speech signals to generate meaningful emotional representations and further improve the SER performance. To address this aim, we propose the Static and Dynamic Time–frequency Network called SDTF-Net. To reduce the interference between different features, in SDTF-Net, we introduce two parallel modules, i.e., Time–Frequency Convolutional Neural Network (TF-CNN) and Time–Frequency Transformer (TF-Trans), to generate the static and dynamic emotional features based on the time–frequency information, respectively. Specifically, in the TF-Trans module, we design a three-step strategy to learn the relatively complicated dynamic feature, comprising two parallel Bi-direction Multi-scale Dynamic Activation (BMDA) blocks, two parallel transformer (Trans) blocks, and a time–frequency fusion (TF-Cross) block. Subsequently, we propose an SD-Cross module to effectively fuse the learned static and dynamic features, generating the final meaningful emotional representations. Given the problem of data sparsity, moreover, we employ the data augmentation method called SpecAugment to improve the learning ability of SDTF-Net. From the experimental results on IEMOCAP and MSP-IMPROV datasets, we observe that SDTF-Net outperforms the state-of-the-art methods. Furthermore, extensive ablation experiments further indicate the rationality of the SDTF-Net structure and the necessity of considering dynamic and static features based on time–frequency information comprehensively in SER.

Learning asymmetric embedding for attributed networks via convolutional neural network

Recently network embedding has gained increasing attention due to its advantages in facilitating network computation tasks such as link prediction, node classification and node clustering. The objective of network embedding is to represent network nodes in a low-dimensional vector space while retaining as much information as possible from the original network including structural, relational, and semantic information. However, asymmetric nature of directed networks poses many challenges as how to best preserve edge directions during the embedding process. Here, we propose a novel deep asymmetric attributed network embedding model based on the convolutional graph neural network, called AAGCN. The main idea is to maximally preserve the asymmetric proximity and asymmetric similarity of directed attributed networks. AAGCN introduces two neighbourhood feature aggregation schemes to separately aggregate the features of a node with the features of its in- and out- neighbours. Then, it learns two embedding vectors for each node, one source embedding vector and one target embedding vector. The final representations are the results of concatenating source and target embedding vectors. We test the performance of AAGCN on four real-world networks for network reconstruction, link prediction, node classification and visualization downstream tasks and investigate the impact of hyperparameters of the proposed method on the performance of the tasks. The experimental results show the superiority of AAGCN against state-of-the-art embedding methods.

A double attention graph network for link prediction on temporal graph

Many data in the real world can be abstracted into the format of dynamic graph, and predicting the connection relationship between nodes in graph is a problem that is often faced in the researching. With the development of deep learning techniques, Graph Neural Networks(GNNs) have been widely used to solve graph-related problems and achieved great results. However, most of the works focus on static graph, or applied the method of static graph to coarse grained discrete time graph after slice dynamic graph into snapshot. As a kind of data that incorporates all the time process information, temporal graph contains a variety of information that worth to be explored. Several works have been done for temporal graph data, but there are still shortcomings. We believe that when researching the evolution of dynamic graph, the influence of the surrounding environment on each node in local time and space is decisive for the properties of the node, which has not been considered in the previous works. Therefore, we propose a novel general model: Double Attention Temporal Graph Network(DATGN). Through an activity based sampling algorithm, the significant nodes in the local time-structure space that associated with each node are sampled and generated as sequence, then the global information of node is aggregated through a global attention network. After which the information of the sampled sequences is aggregated by a local attention network that does not depend on edge relations, and the final representation vector of the node is obtained to make the prediction. In the experiment, we selected three data sets from different knowledge domains and compared them with the current state-of-art models. In the transductive and inductive link prediction tasks, DATGN both achieved the best results in terms of accuracy and operational efficiency than other baseline models, and we discussed the reasons of this improvement. The experiment results demonstrate that the local spatial–temporal network layer can capture the evolutionary pattern of the time sequence and improve the accuracy of link prediction.

Improving drug-drug interactions prediction with interpretability via meta-path-based information fusion.

Drug-drug interactions (DDIs) are compound effects when patients take two or more drugs at the same time, which may weaken the efficacy of drugs or cause unexpected side effects. Thus, accurately predicting DDIs is of great significance for the drug development and the drug safety surveillance. Although many methods have been proposed for the task, the biological knowledge related to DDIs is not fully utilized and the complex semantics among drug-related biological entities are not effectively captured in existing methods, leading to suboptimal performance. Moreover, the lack of interpretability for the predicted results also limits the wide application of existing methods for DDIs prediction. In this study, we propose a novel framework for predicting DDIs with interpretability. Specifically, we construct a heterogeneous information network (HIN) by explicitly utilizing the biological knowledge related to the procedure of inducing DDIs. To capture the complex semantics in HIN, a meta-path-based information fusion mechanism is proposed to learn high-quality representations of drugs. In addition, an attention mechanism is designed to combine semantic information obtained from meta-paths with different lengths to obtain final representations of drugs for DDIs prediction. Comprehensive experiments are conducted on 2410 approved drugs, and the results of predictive performance comparison show that our proposed framework outperforms selected representative baselines on the task of DDIs prediction. The results of ablation study and cold-start scenario indicate that the meta-path-based information fusion mechanism red is beneficial for capturing the complex semantics among drug-related biological entities. Moreover, the results of case study demonstrate that the designed attention mechanism is able to provide partial interpretability for the predicted DDIs. Therefore, the proposed method will be a feasible solution to the task of predicting DDIs.

BFCN: A Novel Classification Method of Encrypted Traffic Based on BERT and CNN

With the speedy advancement of encryption technology and the exponential increase in applications, network traffic classification has become an increasingly important research topic. Existing methods for classifying encrypted traffic have certain limitations. For example, traditional approaches such as machine learning rely heavily on feature engineering, deep learning approaches are susceptible to the amount and distribution of labeled data, and pretrained models focus merely on the global traffic features while ignoring local features. To solve the above problem, we propose a BERT-based byte-level feature convolutional network (BFCN) model consisting of two novel modules. The first is a packet encoder module, in which we use the BERT pretrained encrypted traffic classification model to capture global traffic features through its attention mechanism; the second is a CNN module, which captures byte-level local features in the traffic through convolutional operations. The packet-level and byte-level features are concatenated as the traffic’s final representation, which can better represent encrypted traffic. Our approach achieves state-of-the-art performance on the publicly available ISCX-VPN dataset for the traffic service and application identification task, achieving F1 scores of 99.11% and 99.41%, respectively, on these two tasks. The experimental results demonstrate that our method further improves the performance of encrypted traffic classification.

A brief review of contrastive learning applied to astrophysics

Abstract Reliable tools to extract patterns from high-dimensionality spaces are becoming more necessary as astronomical data sets increase both in volume and complexity. Contrastive Learning is a self-supervised machine learning algorithm that extracts informative measurements from multidimensional data sets, which has become increasingly popular in the computer vision and Machine Learning communities in recent years. To do so, it maximizes the agreement between the information extracted from augmented versions of the same input data, making the final representation invariant to the applied transformations. Contrastive Learning is particularly useful in astronomy for removing known instrumental effects and for performing supervised classifications and regressions with a limited amount of available labels, showing a promising avenue towards Foundation Models. This short review paper briefly summarizes the main concepts behind contrastive learning and reviews the first promising applications to astronomy. We include some practical recommendations on which applications are particularly attractive for contrastive learning.

Deep Learning-Based Multiscale Pyramid Sieve and Analysis Module in Image Segmentation

Image segmentation is an essential part of remote sensing preprocessing. This paper proposes a deep learning-based multiscale pyramid sieve and analysis module to solve the problems associated with multiscale object coexistence in the complex semantic environments of multispectral remote sensing images. The image information distribution features, which change with scales in different convolutional feature map pyramids, were investigated, and the relationship between informative scales and the variation function curve was examined. Then, the proposed sieve and analysis module was designed according to the results. The proposed module is described as follows. (1) First, a feature map pyramid is built via a convolution calculation. (2) Sieve informative feature layers from the feature map pyramid with variation analysis. (3) Fuse informative feature layers. (4) The final feature representation is input into subsequent calculations for image segmentation. Experimental results demonstrate that, compared to the control group, the precision of the experimental group improved by 3.57%-5.89%, and the intraclass conformity improved by 2.74%-5.58%. In addition, the intraclass chaos decreased by 1.5%-7.9%. The experimental results demonstrate that the proposed multiscale pyramid sieve and analysis module can separate informative feature layers and improve the hierarchical segmentation accuracy of remote sensing multispectral images.

Sentence opinion mining model for fusing target entities in official government documents

<abstract> <p>When drafting official government documents, it is necessary to firmly grasp the main idea and ensure that any positions stated within the text are consistent with those in previous documents. In combination with the field's demands, By taking advantage of suitable text-mining techniques to harvest opinions from sentences in official government documents, the efficiency of official government document writers can be significantly increased. Most existing opinion mining approaches employ text classification methods to directly mine the sentential text of official government documents while disregarding the influence of the objects described within the documents (i.e., the target entities) on the sentence opinion categories. To address these issues, this study proposes a sentence opinion mining model that fuses the target entities within documents. Based on the Bi-directional long short-term (BiLSTM) and attention mechanisms, the model fully considers the attention given by a official government document's target entity to different words within the corresponding sentence text, as well as the dependency between words of the sentence. The model subsequently fuses two by using feature vector fusion to obtain the final semantic representation of the text, which is then classified using a fully connected network and softmax function. Experimental results based on a dataset of official government documents show that the model significantly outperforms baseline models such as Text-convolutional neural network (TextCNN), recurrent neural network (RNN), and BiLSTM.</p> </abstract>

Hypergraph representation of multimodal brain networks for patients with end-stage renal disease associated with mild cognitive impairment

The structure and function of brain networks (BN) may be altered in patients with end-stage renal disease (ESRD). However, there are relatively few attentions on ESRD associated with mild cognitive impairment (ESRDaMCI). Most studies focus on the pairwise relationships between brain regions, without taking into account the complementary information of functional connectivity (FC) and structural connectivity (SC). To address the problem, a hypergraph representation method is proposed to construct a multimodal BN for ESRDaMCI. First, the activity of nodes is determined by connection features extracted from functional magnetic resonance imaging (fMRI) (i.e., FC), and the presence of edges is determined by physical connections of nerve fibers extracted from diffusion kurtosis imaging (DKI) (i.e., SC). Then, the connection features are generated through bilinear pooling and transformed into an optimization model. Next, a hypergraph is constructed according to the generated node representation and connection features, and the node degree and edge degree of the hypergraph are calculated to obtain the hypergraph manifold regularization (HMR) term. The HMR and L1 norm regularization terms are introduced into the optimization model to achieve the final hypergraph representation of multimodal BN (HRMBN). Experimental results show that the classification performance of HRMBN is significantly better than that of several state-of-the-art multimodal BN construction methods. Its best classification accuracy is 91.0891%, at least 4.3452% higher than that of other methods, verifying the effectiveness of our method. The HRMBN not only achieves better results in ESRDaMCI classification, but also identifies the discriminative brain regions of ESRDaMCI, which provides a reference for the auxiliary diagnosis of ESRD.

Predicting Drug-Target Interactions Over Heterogeneous Information Network.

Identifying Drug-Target Interactions (DTIs) is a critical step in studying pathogenesis and drug development. Due to the fact that conventional experimental methods usually suffer from high costs and low efficiency, various computational methods have been proposed to detect potential DTIs by extracting features from the biological information of drugs and their target proteins. Though effective, most of them fall short of considering the topological structure of the DTI network, which provides a global view to discover novel DTIs. In this paper, a network-based computational method, namely LG-DTI, is proposed to accurately predict DTIs over a heterogeneous information network. For drugs and target proteins, LG-DTI first learns not only their local representations from drug molecular structures and protein sequences, but also their global representations by using a semi-supervised heterogeneous network embedding method. These two kinds of representations consist of the final representations of drugs and target proteins, which are then incorporated into a Random Forest classifier to complete the task of DTI prediction. The performance of LG-DTI has been evaluated on two independent datasets and also compared with several state-of-the-art methods. Experimental results show the superior performance of LG-DTI. Moreover, our case study indicates that LG-DTI can be a valuable tool for identifying novel DTIs.

Recommendation model based on intention decomposition and heterogeneous information fusion.

In order to solve the problem of timeliness of user and item interaction intention and the noise caused by heterogeneous information fusion, a recommendation model based on intention decomposition and heterogeneous information fusion (IDHIF) is proposed. First, the intention of the recently interacting items and the users of the recently interacting candidate items is decomposed, and the short feature representation of users and items is mined through long-short term memory and attention mechanism. Then, based on the method of heterogeneous information fusion, the interactive features of users and items are mined on the user-item interaction graph, the social features of users are mined on the social graph, and the content features of the item are mined on the knowledge graph. Different feature vectors are projected into the same feature space through heterogeneous information fusion, and the long feature representation of users and items is obtained through splicing and multi-layer perceptron. The final representation of users and items is obtained by combining short feature representation and long feature representation. Compared with the baseline model, the AUC on the Last.FM and Movielens-1M datasets increased by 1.83 and 4.03 percentage points, respectively, the F1 increased by 1.28 and 1.58 percentage points, and the Recall@20 increased by 3.96 and 2.90 percentage points. The model proposed in this paper can better model the features of users and items, thus enriching the vector representation of users and items, and improving the recommendation efficiency.

One-stage Visual Relationship Referring with Transformers and Adaptive Message Passing.

There exist a variety of visual relationships among entities in an image. Given a relationship query ⟨subject, predicate, object⟩, the task of visual relationship referring (VRR) aims to disambiguate instances of the same entity category and simultaneously localize the subject and object entities in an image. Previous works of VRR can be generally categorized into one-stage and multi-stage methods. The former ones directly localize a pair of entities from the image but they suffer from low prediction accuracy, while the latter ones perform better but they are indirect to localize only a couple of entities by pre-generating a rich amount of candidate proposals. In this paper, we formulate the task of VRR as an end-to-end bounding box regression problem and propose a novel one-stage approach, called VRR-TAMP, by effectively integrating Transformers and an adaptive message passing mechanism. First, visual relationship queries and images are respectively encoded to generate the basic modality-specific embeddings, which are then fed into a cross-modal Transformer encoder to produce the joint representation. Second, to obtain the specific representation of each entity, we introduce an adaptive message passing mechanism and design an entity-specific information distiller SR-GMP, which refers to a gated message passing (GMP) module that works on the joint representation learned from a single learnable token. The GMP module adaptively distills the final representation of an entity by incorporating the contextual cues regarding the predicate and the other entity. Experiments on VRD and Visual Genome datasets demonstrate that our approach significantly outperforms its one-stage competitors and achieves competitive results with the state-of-the-art multi-stage methods.