Pair Representation Research Articles

DrugDAGT: a dual-attention graph transformer with contrastive learning improves drug-drug interaction prediction

BackgroundDrug-drug interactions (DDIs) can result in unexpected pharmacological outcomes, including adverse drug events, which are crucial for drug discovery. Graph neural networks have substantially advanced our ability to model molecular representations; however, the precise identification of key local structures and the capture of long-distance structural correlations for better DDI prediction and interpretation remain significant challenges.ResultsHere, we present DrugDAGT, a dual-attention graph transformer framework with contrastive learning for predicting multiple DDI types. The dual-attention graph transformer incorporates attention mechanisms at both the bond and atomic levels, thereby enabling the integration of short and long-range dependencies within drug molecules to pinpoint key local structures essential for DDI discovery. Moreover, DrugDAGT further implements graph contrastive learning to maximize the similarity of representations across different views for better discrimination of molecular structures. Experiments in both warm-start and cold-start scenarios demonstrate that DrugDAGT outperforms state-of-the-art baseline models, achieving superior overall performance. Furthermore, visualization of the learned representations of drug pairs and the attention map provides interpretable insights instead of black-box results.ConclusionsDrugDAGT provides an effective tool for accurately predicting multiple DDI types by identifying key local chemical structures, offering valuable insights for prescribing medications, and guiding drug development. All data and code of our DrugDAGT can be found at https://github.com/codejiajia/DrugDAGT.

The Observed T Cell Receptor Space database enables paired-chain repertoire mining, coherence analysis, and language modeling

T cell activation is governed through Tcell receptors (TCRs), heterodimers of two sequence-variable chains (often an α and β chain) that synergistically recognize antigen fragments presented on cell surfaces. Despite this, there only exist repositories dedicated to collecting single-chain, not paired-chain, TCR sequence data. We addressed this gap by creating the Observed TCR Space (OTS) database, a source of consistently processed and annotated, full-length, paired-chain TCR sequences. Currently, OTS contains 5.35 million redundant (1.63 million non-redundant), predominantly human sequences from across 50 studies and at least 75 individuals. Using OTS, we identify pairing biases, public TCRs, and distinct chain coherence patterns relative to antibodies. We also release a paired-chain TCR language model, providing paired embedding representations and a method for residue in-filling conditional on the partner chain. OTS will be updated as a central community resource and is freely downloadable and available as a web application.

Finding the most potent compounds using active learning on molecular pairs.

Active learning allows algorithms to steer iterative experimentation to accelerate and de-risk molecular optimizations, but actively trained models might still exhibit poor performance during early project stages where the training data is limited and model exploitation might lead to analog identification with limited scaffold diversity. Here, we present ActiveDelta, an adaptive approach that leverages paired molecular representations to predict improvements from the current best training compound to prioritize further data acquisition. We apply the ActiveDelta concept to both graph-based deep (Chemprop) and tree-based (XGBoost) models during exploitative active learning for 99 Ki benchmarking datasets. We show that both ActiveDelta implementations excel at identifying more potent inhibitors compared to the standard exploitative active learning implementations of Chemprop, XGBoost, and Random Forest. The ActiveDelta approach is also able to identify more chemically diverse inhibitors in terms of their Murcko scaffolds. Finally, deep models such as Chemprop trained on data selected through ActiveDelta approaches can more accurately identify inhibitors in test data created through simulated time-splits. Overall, this study highlights the large potential for molecular pairing approaches to further improve popular active learning strategies in low data regimes by enabling faster and more accurate identification of more diverse molecular hits against critical drug targets.

Fast Discriminant Analysis With Adaptive Reconstruction Structure Preserving.

Neighborhood reconstruction methods have been widely applied to feature engineering. Existing reconstruction-based discriminant analysis methods normally project high-dimensional data into a low-dimensional space while preserving the reconstruction relationships among samples. However, there are three limitations: 1) the reconstruction coefficients are learned based on the collaborative representation of all sample pairs, which requires the training time to be the cube of the number of samples; 2) these coefficients are learned in the original space, ignoring the interference of the noise and redundant features; and 3) there is a reconstruction relationship between heterogeneous samples; this will enlarge the similarity of heterogeneous samples in the subspace. In this article, we propose a fast and adaptive discriminant neighborhood projection model to tackle the above drawbacks. First, the local manifold structure is captured by bipartite graphs in which each sample is reconstructed by anchor points derived from the same class as that sample; this can avoid the reconstruction between heterogeneous samples. Second, the number of anchor points is far less than the number of samples; this strategy can reduce the time complexity substantially. Third, anchor points and reconstruction coefficients of bipartite graphs are updated adaptively in the process of dimensionality reduction, which can enhance the quality of bipartite graphs and extract discriminative features simultaneously. An iterative algorithm is designed to solve this model. Extensive results on toy data and benchmark datasets show the effectiveness and superiority of our model.

BiRNN-DDI: A Drug-Drug Interaction Event Type Prediction Model Based on Bidirectional Recurrent Neural Network and Graph2Seq Representation.

Research on drug-drug interaction (DDI) prediction, particularly in identifying DDI event types, is crucial for understanding adverse drug reactions and drug combinations. This work introduces a Bidirectional Recurrent Neural Network model for DDI event type prediction (BiRNN-DDI), which simultaneously considers structural relationships and contextual information. Our BiRNN-DDI model constructs drug feature graphs to mine structural relationships. For contextual information, it transforms drug graphs into sequences and employs a two-channel structure, integrating BiRNN, to obtain contextual representations of drug-drug pairs. The model's effectiveness is demonstrated through comparisons with state-of-the-art models on two DDI event-type benchmarks. Extensive experimental results reveal that BiRNN-DDI surpasses other models in accuracy, AUPR, AUC, F1 score, Precision, and Recall metrics on both small and large datasets. Additionally, our model exhibits a lower parameter space, indicating more efficient learning of drug feature representations and prediction of potential DDI event types.

GSRF-DTI: a framework for drug-target interaction prediction based on a drug-target pair network and representation learning on a large graph

BackgroundIdentification of potential drug-target interactions (DTIs) with high accuracy is a key step in drug discovery and repositioning, especially concerning specific drug targets. Traditional experimental methods for identifying the DTIs are arduous, time-intensive, and financially burdensome. In addition, robust computational methods have been developed for predicting the DTIs and are widely applied in drug discovery research. However, advancing more precise algorithms for predicting DTIs is essential to meet the stringent standards demanded by drug discovery.ResultsWe proposed a novel method called GSRF-DTI, which integrates networks with a deep learning algorithm to identify DTIs. Firstly, GSRF-DTI learned the embedding representation of drugs and targets by integrating multiple drug association information and target association information, respectively. Then, GSRF-DTI considered the influence of drug-target pair (DTP) association on DTI prediction to construct a drug-target pair network (DTP-NET). Next, we utilized GraphSAGE on DTP-NET to learn the potential features of the network and applied random forest (RF) to predict the DTIs. Furthermore, we conducted ablation experiments to validate the necessity of integrating different types of network features for identifying DTIs. It is worth noting that GSRF-DTI proposed three novel DTIs.ConclusionsGSRF-DTI not only considered the influence of the interaction relationship between drug and target but also considered the impact of DTP association relationship on DTI prediction. We initially use GraphSAGE to aggregate the neighbor information of nodes for better identification. Experimental analysis on Luo’s dataset and the newly constructed dataset revealed that the GSRF-DTI framework outperformed several state-of-the-art methods significantly.

Span-level bidirectional retention scheme for aspect sentiment triplet extraction

The objective of the Aspect Sentiment Triplet Extraction (ASTE) task is to identify triplets of (aspect, opinion, sentiment) from user-generated reviews. The current study does not extensively integrate the interaction between word pairs and aspect-opinion pairs during the learning process at the granularity of sentence analysis. Furthermore, the bidirectional inference for the triplet, along with the parallel computing approach for long-span texts, also fail to achieve efficient unification. We introduce a new perspective: Span-level Bidirectional Retention Scheme(SBRS) for Aspect Sentiment Triplet Extraction model. The model comprises two pathways. The first pathway involves extracting effective aspect-opinion pair outcomes via two progressive submodules that operate on words and word pairs at varying scales. Building on the first pathway, the second pathway senses the interaction information of word pairs through bidirectional recursion and combines an efficient parallel computing approach. This combination allows the model to utilize three features – context, semantics, and relationship – to accurately identify the sentimental orientation. Thus, the two pathways facilitate the learning of relation-aware representations of word pairs. We carried out experiments on two public datasets, showing an average enhancement of 3.34% and 1.72% in F1 scores compared to the most recent baselines models, and multiple experiments from diverse angles proved the model’s superiority.

A Fusion Pretrained Approach for Identifying the Cause of Sarcasm Remarks

Sarcastic remarks often appear in social media and e-commerce platforms to express almost exclusively negative emotions and opinions on certain instances, such as dissatisfaction with a purchased product or service. Thus, the detection of sarcasm allows merchants to timely resolve users’ complaints. However, detecting sarcastic remarks is difficult because of its common form of using counterfactual statements. The few studies that are dedicated to detecting sarcasm largely ignore what sparks these sarcastic remarks, which could be because of an empty promise of a merchant’s product description. This study formulates a novel problem of sarcasm cause detection that leverages domain information, dialogue context information, and sarcasm sentences by proposing a pretrained language model-based approach equipped with a novel hybrid multihead fusion-attention mechanism that combines self-attention, target-attention, and a feed-forward neural network. The domain information and the dialogue context information are then interactively fused to obtain the domain-specific dialogue context representation, and bidirectionally enhanced sarcasm-cause pair representations are generated for detecting sarcasm spark. Experimental results on real-world data sets demonstrate the efficacy of the proposed model. The findings of this study contribute to the literature on sarcasm cause detection and provide business value to relevant stakeholders and consumers. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: This work was partially supported by the National Natural Science Foundation of China [Grants 72293575, 62071467, and 62141608] and the Research Grant Council of the Hong Kong Special Administrative Region, China [Grants 11500322 and 11500421]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0285 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0285 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Assessing the Cross-linguistic Utility of Abstract Meaning Representation

Abstract Semantic representations capture the meaning of a text. Abstract Meaning Representation (AMR), a type of semantic representation, focuses on predicate-argument structure and abstracts away from surface form. Though AMR was developed initially for English, it has now been adapted to a multitude of languages in the form of non-English annotation schemas, cross-lingual text-to-AMR parsing, and AMR-to-(non-English) text generation. We advance prior work on cross-lingual AMR by thoroughly investigating the amount, types, and causes of differences that appear in AMRs of different languages. Further, we compare how AMR captures meaning in cross-lingual pairs versus strings, and show that AMR graphs are able to draw out fine-grained differences between parallel sentences. We explore three primary research questions: (1) What are the types and causes of differences in parallel AMRs? (2) How can we measure the amount of difference between AMR pairs in different languages? (3) Given that AMR structure is affected by language and exhibits cross-lingual differences, how do cross-lingual AMR pairs compare to string-based representations of cross-lingual sentence pairs? We find that the source language itself does have a measurable impact on AMR structure, and that translation divergences and annotator choices also lead to differences in cross-lingual AMR pairs. We explore the implications of this finding throughout our study, concluding that, although AMR is useful to capture meaning across languages, evaluations need to take into account source language influences if they are to paint an accurate picture of system output, and meaning generally.

Prediction of Drug-Target Affinity Using Attention Neural Network.

Studying drug-target interactions (DTIs) is the foundational and crucial phase in drug discovery. Biochemical experiments, while being the most reliable method for determining drug-target affinity (DTA), are time-consuming and costly, making it challenging to meet the current demands for swift and efficient drug development. Consequently, computational DTA prediction methods have emerged as indispensable tools for this research. In this article, we propose a novel deep learning algorithm named GRA-DTA, for DTA prediction. Specifically, we introduce Bidirectional Gated Recurrent Unit (BiGRU) combined with a soft attention mechanism to learn target representations. We employ Graph Sample and Aggregate (GraphSAGE) to learn drug representation, especially to distinguish the different features of drug and target representations and their dimensional contributions. We merge drug and target representations by an attention neural network (ANN) to learn drug-target pair representations, which are fed into fully connected layers to yield predictive DTA. The experimental results showed that GRA-DTA achieved mean squared error of 0.142 and 0.225 and concordance index reached 0.897 and 0.890 on the benchmark datasets KIBA and Davis, respectively, surpassing the most state-of-the-art DTA prediction algorithms.

Aggregation of melanoma tumour site within Western Australian families

BackgroundEvidence is emerging that melanoma has distinct aetiologic pathways and subtypes, characterized by factors like anatomic site of the tumour. To explore genetic influences on anatomic subtypes, we examined the extent to which melanomas in first-degree relatives shared the same body site of occurrence. MethodsPopulation-level linked data was used to identify the study population of over 1.5 million individuals born in Western Australia between 1945 and 2014, and their first-degree relatives. There were 1009 pairs of invasive tumours from 677 family pairs, each categorised by anatomic site. Greater than expected representation of site-concordant pairs would suggest the presence of genetic factors that predispose individuals to site-specific melanoma. ResultsComparing observed versus expected totals, we observed a modest increase in site concordance for invasive head/neck and truncal tumours (P=0.02). A corresponding analysis including in situ tumours showed a similar concordance (P=0.05). No further evidence of concordance was observed when stratified by sex. ConclusionIn conclusion, modest evidence of aggregation was observed but with inconsistent patterns between sites. Results suggest that further investigation into the familial aggregation of melanoma by tumour site is warranted, with the inclusion of genetic data in order to disentangle the relative contributions of genetic and environmental factors.

Boundary-aware information maximization for self-supervised medical image segmentation

Self-supervised representation learning can boost the performance of a pre-trained network on downstream tasks for which labeled data is limited. A popular method based on this paradigm, known as contrastive learning, works by constructing sets of positive and negative pairs from the data, and then pulling closer the representations of positive pairs while pushing apart those of negative pairs. Although contrastive learning has been shown to improve performance in various classification tasks, its application to image segmentation has been more limited. This stems in part from the difficulty of defining positive and negative pairs for dense feature maps without having access to pixel-wise annotations.In this work, we propose a novel self-supervised pre-training method that overcomes the challenges of contrastive learning in image segmentation. Our method leverages Information Invariant Clustering (IIC) as an unsupervised task to learn a local representation of images in the decoder of a segmentation network, but addresses three important drawbacks of this approach: (i) the difficulty of optimizing the loss based on mutual information maximization; (ii) the lack of clustering consistency for different random transformations of the same image; (iii) the poor correspondence of clusters obtained by IIC with region boundaries in the image. Toward this goal, we first introduce a regularized mutual information maximization objective that encourages the learned clusters to be balanced and consistent across different image transformations. We also propose a boundary-aware loss based on cross-correlation, which helps the learned clusters to be more representative of important regions in the image. Compared to contrastive learning applied in dense features, our method does not require computing positive and negative pairs and also enhances interpretability through the visualization of learned clusters.Comprehensive experiments involving four different medical image segmentation tasks reveal the high effectiveness of our self-supervised representation learning method. Our results show the proposed method to outperform by a large margin several state-of-the-art self-supervised and semi-supervised approaches for segmentation, reaching a performance close to full supervision with only a few labeled examples.

A New Mechanism for Eliminating Implicit Conflict in Graph Contrastive Learning

Graph contrastive learning (GCL) has attracted considerable attention because it can self-supervisedly extract low-dimensional representation of graph data. InfoNCE-based loss function is widely used in graph contrastive learning, which pulls the representations of positive pairs close to each other and pulls the representations of negative pairs away from each other. Recent works mainly focus on designing new augmentation methods or sampling strategies. However, we argue that the widely used InfoNCE-based methods may contain an implicit conflict which seriously confuses models when learning from negative pairs. This conflict is engendered by the encoder's message-passing mechanism and the InfoNCE loss function. As a result, the learned representations between negative samples cannot be far away from each other, compromising the model performance. To our best knowledge, this is the first time to report and analysis this conflict of GCL. To address this problem, we propose a simple but effective method called Partial ignored Graph Contrastive Learning (PiGCL). Specifically, PiGCL first dynamically captures the conflicts during training by detecting the gradient of representation similarities. It then enables the loss function to ignore the conflict, allowing the encoder to adaptively learn the ignored information without self-supervised samples. Extensive experiments demonstrate the effectiveness of our method.

ScGREAT: Transformer-based deep-language model for gene regulatory network inference from single-cell transcriptomics

Gene regulatory networks (GRNs) involve complex and multi-layer regulatory interactions between regulators and their target genes. Precise knowledge of GRNs is important in understanding cellular processes and molecular functions. Recent breakthroughs in single-cell sequencing technology made it possible to infer GRNs at single-cell level. Existing methods, however, are limited by expensive computations, and sometimes simplistic assumptions. To overcome these obstacles, we propose scGREAT, a framework to infer GRN using gene embeddings and transformer from single-cell transcriptomics. scGREAT starts by constructing gene expression and gene biotext dictionaries from scRNA-seq data and gene text information. The representation of TF gene pairs is learned through optimizing embedding space by transformer-based engine. Results illustrated scGREAT outperformed other contemporary methods on benchmarks. Besides, gene representations from scGREAT provide valuable gene regulation insights, and external validation on spatial transcriptomics illuminated the mechanism behind scGREAT annotation. Moreover, scGREAT identified several TF target regulations corroborated in studies.

Weighted and Choquet $$L^p$$ distance representation of comparative dissimilarity relations on fuzzy description profiles

AbstractWe consider comparative dissimilarity relations on pairs on fuzzy description profiles, the latter providing a fuzzy set-based representation of pairs of objects. Such a relation expresses the idea of “no more dissimilar than” and is used by a decision maker when performing a case-based decision task under vague information. We first limit ourselves to those relations admitting a weighted $$\varvec{L}^p$$ L p distance representation, for which we provide an axiomatic characterization in case the relation is complete, transitive and defined on the entire space of pairs of fuzzy description profiles. Next, we switch to the more general class of comparative dissimilarity relations representable by a Choquet $$\varvec{L}^p$$ L p distance, parameterized by a completely alternating normalized capacity.

Relation-aware graph structure embedding with co-contrastive learning for drug–drug interaction prediction

Relation-aware graph structure embedding (RaGSE) is promising for predicting multi-relational drug–drug interactions (DDIs). Most existing methods based on RaGSE commence by constructing a multi-relational DDI graph. They then learn the RaGSEs for drugs by aggregating their neighbor’s features under different relations. Nevertheless, new drugs lack neighbors in the DDI graph. This limitation hinders the ability of these methods to effectively learn RaGSEs for new drugs, resulting in suboptimal performance when evaluating DDIs that involve new drugs. To alleviate this issue, we propose a novel DDI prediction method based on relation-aware graph structure embedding with co-contrastive learning, RaGSECo. The proposed RaGSECo constructs two heterogeneous graphs: a multi-relational DDI graph and a multi-attribute drug–drug similarity (DDS) graph. The two graphs are used respectively for learning and propagating the RaGSEs of drugs, aiming to ensure all drugs, including new ones, can possess effective RaGSEs. Additionally, we present a novel co-contrastive learning module to learn feature representations for drug pairs (DPs). This module learns DP representations from two distinct views (interaction and similarity views) and encourages these views to supervise each other collaboratively to obtain more discriminative DP representations. We evaluate the effectiveness of our RaGSECo on three different tasks using two real datasets. Experimental outcomes unequivocally indicate that our RaGSECo surpasses several state-of-the-art methods in performance.

Noise Imitation Based Adversarial Training for Robust Multimodal Sentiment Analysis

As an inevitable phenomenon in real-world applications, data imperfection has emerged as one of the most critical challenges for multimodal sentiment analysis. However, existing approaches tend to overly focus on a specific type of imperfection, leading to performance degradation in real-world scenarios where multiple types of noise exist simultaneously. In this work, we formulate the imperfection with the modality feature missing at the training period and propose the noise intimation based adversarial training framework to improve the robustness against various potential imperfections at the inference period. Specifically, the proposed method first uses temporal feature erasing as the augmentation for noisy instances construction and exploits the modality interactions through the self-attention mechanism to learn multimodal representation for original-noisy instance pairs. Then, based on paired intermediate representation, a novel adversarial training strategy with semantic reconstruction supervision is proposed to learn unified joint representation between noisy and perfect data. For experiments, the proposed method is first verified with the modality feature missing, the same type of imperfection as the training period, and shows impressive performance. Moreover, we show that our approach is capable of achieving outstanding results for other types of imperfection, including modality missing, automation speech recognition error and attacks on text, highlighting the generalizability of our model. Finally, we conduct case studies on general additive distribution, which introduce background noise and blur into raw video clips, further revealing the capability of our proposed method for real-world applications.

Address the Unseen Relationships: Attribute Correlations in Text Attribute Person Search.

Text attribute person search aims to identify the particular pedestrian by textual attribute information. Compared to person re-identification tasks which requires imagery samples as its query, text attribute person search is more useful under the circumstance where only witness is available. Most existing text attribute person search methods focus on improving the matching correlation and alignments by learning better representations of person-attribute instance pairs, with few consideration of the latent correlations between attributes. In this work, we propose a graph convolutional network (GCN) and pseudo-label-based text attribute person search method. Concretely, the model directly constructs the attribute correlations by label co-occurrence probability, in which the nodes are represented by attribute embedding and edges are by the filtered correlation matrix of attribute labels. In order to obtain better representations, we combine the cross-attention module (CAM) and the GCN. Furthermore, to address the unseen attribute relationships, we update the edge information through the instances through testing set with high predicted probability thus to better adapt the attribute distribution. Extensive experiments illustrate that our model outperforms the existing state-of-the-art methods on publicly available person search benchmarks: Market-1501 and PETA.

A Dual-Modality Complex-Valued Fusion Method for Predicting Side Effects of Drug-Drug Interactions Based on Graph Neural Network.

Predicting potential side effects of drug-drug interactions (DDIs), which is a major concern in clinical treatment, can increase therapeutic efficacy. In recent studies, how to use the multi-modal drug features is important for DDI prediction. Thus, it remains a challenge to explore an efficient computational method to achieve the feature fusion cross- and intra-modality. In this paper, we propose a dual-modality complex-valued fusion method (DMCF-DDI) for predicting the side effects of DDIs, using the form and properties of complex-vector to enhance the representations of DDIs. Firstly, DMCF-DDI applies two Graph Convolutional Network (GCN) encoders to learn molecular structure and topological features from fingerprint and knowledge graphs, respectively. Secondly, an asymmetric skip connection (ASC) uses distinct semantic-level features to construct the complex-valued drug pair representations (DPRs). Then, the complex-vector multiplication is used as a fusion operator to obtain the fine-grained DPRs. Finally, we calculate the prediction probability of DDIs by Hermitian inner product in the complex space. Compared with other methods, DMCF-DDI achieves superior performance in all situations using a fusion operator with the lowest parameter numbers. For the case study, we select six diseases and common side effects in clinical treatment to verify identification ability of our model. We also prove the advantage of ASC and complex-valued fusion can achieve to align the cross-modal fused positive DPRs through a comprehensive analysis on the phase-modulus distribution histogram of DPRs. In the end, we explain the reason for alignment based on the similarity of features and node neighbors.

TransVAE-DTA: Transformer and variational autoencoder network for drug-target binding affinity prediction

Background and objectiveRecent studies have emphasized the significance of computational in silico drug-target binding affinity (DTA) prediction in the field of drug discovery and drug repurposing. However, existing DTA prediction approaches suffer from two major deficiencies that impede their progress. Firstly, while most methods primarily focus on the feature representations of drug-target binding affinity pairs, they fail to consider the long-distance relationships of proteins. Furthermore, many deep learning-based DTA predictors simply model the interaction of drug-target pairs through concatenation, which hampers the ability to enhance prediction performance. MethodsTo address these issues, this study proposes a novel framework named TransVAE-DTA, which combines the transformer and variational autoencoder (VAE). Inspired by the early success of VAEs, we aim to further investigate the feasibility of VAEs for drug structure encoding, while utilizing the transformer architecture for target feature representation. Additionally, an adaptive attention pooling (AAP) module is designed to fuse the drug and target encoded features. Notably, TransVAE-DTA is proven to maximize the lower bound of the joint likelihood of drug, target, and their DTAs. ResultsExperimental results demonstrate the superiority of TransVAE-DTA in drug-target binding affinity prediction assignments on two public Davis and KIBA datasets. ConclusionsIn this research, the developed TransVAE-DTA opens a new avenue for engineering drug-target interactions.