Structural Embedding Research Articles

CLOOME: contrastive learning unlocks bioimaging databases for queries with chemical structures.

The field of bioimage analysis is currently impacted by a profound transformation, driven by the advancements in imaging technologies and artificial intelligence. The emergence of multi-modal AI systems could allow extracting and utilizing knowledge from bioimaging databases based on information from other data modalities. We leverage the multi-modal contrastive learning paradigm, which enables the embedding of both bioimages and chemical structures into a unified space by means of bioimage and molecular structure encoders. This common embedding space unlocks the possibility of querying bioimaging databases with chemical structures that induce different phenotypic effects. Concretely, in this work we show that a retrieval system based on multi-modal contrastive learning is capable of identifying the correct bioimage corresponding to a given chemical structure from a database of ~2000 candidate images with a top-1 accuracy >70 times higher than a random baseline. Additionally, the bioimage encoder demonstrates remarkable transferability to various further prediction tasks within the domain of drug discovery, such as activity prediction, molecule classification, and mechanism of action identification. Thus, our approach not only addresses the current limitations of bioimaging databases but also paves the way towards foundation models for microscopy images.

Attribute network joint embedding based on global attention

The attribute network not only has a complex topology, but its nodes also contain rich attribute information. Attribute network embedding methods extract both network topology and node attribute information to learn low-dimensional embedding of large attribute networks, which are of great importance for network analysis. In this paper, we propose attribute network joint embedding based on global attention (GAJE). First, GAJE uses the structure information of the attribute network to obtain the structure embedding vectors of nodes. Second, we propose a global attention method to obtain the attribute embedding vectors of nodes. This method captures the relationships of different attributes within and between nodes through convolutional neural networks and attention mechanisms, respectively. Finally, GAJE concatenates the structure embedding vectors and the attribute embedding vectors to obtain the final joint embedding vectors which simultaneously reflects the network structure and attributes information. For link prediction and node classification, we compared GAJE with nine well-known network embedding methods in four real datasets. The experimental results show that the algorithm has good attribute network embedding effects. Our tensorflow implementation of the GAJE is available at https://github.com/Andrewsama/GAJE-master.

Deep joint adversarial learning for anomaly detection on attribute networks

Attribute network anomaly detection has attracted growing interest in recent years, which aims to separate the points whose behavior is clearly different from others. The complex interactions between the network structure and node attributes result in difficulty in detecting anomalous nodes on attribute networks. To alleviate the above mentioned issue, in this paper, we design a deep joint adversarial learning representation framework (JAANE) for attribute network anomaly detection, by capturing the consistency and complementarity between network structure and node attributes. Specifically, JAANE utilizes a weight-sharing encoder to learn the attribute embedding and structure embedding in a shared latent space. Then, the feature fusion module fuses the learned attribute embedding and structure embedding into the fused node embedding to capture the consistency and complementarity between them. Finally, the fused node embedding is regularized via adversarial learning, and the anomaly nodes outside the regularized hypersphere space can be effectively detected. The experiment results on the real-world datasets indicate that the proposed JAANE performs better than other state-of-the-art, which demonstrates the effectiveness of the proposed method. The source code and data are released in https://haoyfan.github.io/.

ContactLib-ATT: A Structure-Based Search Engine for Homologous Proteins.

General-purpose protein structure embedding can be used for many important protein biology tasks, such as protein design, drug design and binding affinity prediction. Recent researches have shown that attention-based encoder layers are more suitable to learn high-level features. Based on this key observation, we propose a two-level general-purpose protein structure embedding neural network, called ContactLib-ATT. On local embedding level, a biologically more meaningful contact context is introduced. On global embedding level, attention-based encoder layers are employed for better global representation learning. Our general-purpose protein structure embedding framework is trained and tested on the SCOP40 2.07 dataset. As a result, ContactLib-ATT achieves a SCOP superfamily classification accuracy of 82.4% (i.e., 6.7% higher than state-of-the-art method). On the same dataset, ContactLib-ATT is used to simulate a structure-based search engine for remote homologous proteins, and our top-10 candidate list contains at least one remote homolog with a probability of 91.9%.

Multimodal learning of noncoding variant effects using genome sequence and chromatin structure.

A growing amount of noncoding genetic variants, including single-nucleotide polymorphisms, are found to be associated with complex human traits and diseases. Their mechanistic interpretation is relatively limited and can use the help from computational prediction of their effects on epigenetic profiles. However, current models often focus on local, 1D genome sequence determinants and disregard global, 3D chromatin structure that critically affects epigenetic events. We find that noncoding variants of unexpected high similarity in epigenetic profiles, with regards to their relatively low similarity in local sequences, can be largely attributed to their proximity in chromatin structure. Accordingly, we have developed a multimodal deep learning scheme that incorporates both data of 1D genome sequence and 3D chromatin structure for predicting noncoding variant effects. Specifically, we have integrated convolutional and recurrent neural networks for sequence embedding and graph neural networks for structure embedding despite the resolution gap between the two types of data, while utilizing recent DNA language models. Numerical results show that our models outperform competing sequence-only models in predicting epigenetic profiles and their use of long-range interactions complement sequence-only models in extracting regulatory motifs. They prove to be excellent predictors for noncoding variant effects in gene expression and pathogenicity, whether in unsupervised "zero-shot" learning or supervised "few-shot" learning. Codes and data can be accessed at https://github.com/Shen-Lab/ncVarPred-1D3D and https://zenodo.org/record/7975777.

Embedding Principle for Rings and Abelian Groups

Summary The article concerns about formalizing a certain lemma on embedding of algebraic structures in the Mizar system, claiming that if a ring A is embedded in a ring B then there exists a ring C which is isomorphic to B and includes A as a subring. This construction applies to algebraic structures such as Abelian groups and rings.

Structure Learning Via Meta-Hyperedge for Dynamic Rumor Detection

Online social networks have greatly facilitated our lives but have also propagated the spreading of rumours. Traditional works mostly find rumors from content, but content can be strategically manipulated to evade such detection, making these methods brittle. To improve the accuracy and robustness of rumor detection, we propose to integrate and exploit the content, propagation structure, and temporal relations because information in the networks always spreads dynamically with significant structures. In this paper, we propose a novel rumor detection framework in online temporal networks via structure learning. Specifically, to exploit the propagation structure, we propose a novel hyperedge walking strategy on a meta-hyperedge graph to learn the representations of sub-structures in the networks. Then a hyperedge expansion method is proposed to generate more global structural features. The expanded hyperedges are more hierarchical, making the learned structural embeddings more expressive. To make full use of content, we design a hypergraph learning model using hyperedge expansion to fuse node content with structural features and generate comprehensive representations for the entire graph. To exploit temporal relations, we design a masked temporal attention unit for learning the evolving patterns of the network. Extensive evaluations with six state-of-the-art baselines on two real-world datasets demonstrate the superiority of our solution.

An efficient circRNA-miRNA interaction prediction model by combining biological text mining and wavelet diffusion-based sparse network structure embedding

MotivationAccumulating clinical evidence shows that circular RNA (circRNA) plays an important regulatory role in the occurrence and development of human diseases, which is expected to provide a new perspective for the diagnosis and treatment of related diseases. Using computational methods can provide high probability preselection for wet experiments to save resources. However, due to the lack of neighborhood structure in sparse biological networks, the model based on network embedding and graph embedding is difficult to achieve ideal results. ResultsIn this paper, we propose BioDGW-CMI, which combines biological text mining and wavelet diffusion-based sparse network structure embedding to predict circRNA-miRNA interaction (CMI). In detail, BioDGW-CMI first uses the Bidirectional Encoder Representations from Transformers (BERT) for biological text mining to mine hidden features in RNA sequences, then constructs a CMI network, obtains the topological structure embedding of nodes in the network through heat wavelet diffusion patterns. Next, the Denoising autoencoder organically combines the structural features and Gaussian kernel similarity, finally, the feature is sent to lightGBM for training and prediction. BioDGW-CMI achieves the highest prediction performance in all three datasets in the field of CMI prediction. In the case study, all the 8 pairs of CMI based on circ-ITCH were successfully predicted. AvailabilityThe data and source code can be found at https://github.com/1axin/BioDGW-CMI-model.

Blind spots in IPE: contract law and the structural embedding of transnational capitalism

This paper focuses on the ‘blind spots in IPE’ recently addressed in related Special Issues of Review of International Political Economy and New Political Economy. It identifies a blind spot of law in IPE, tracing the problem to a blind spot in the discipline of international relations (IR) generated by tendencies in dominant theories to consider international law to be super-structural, epiphenomenal, and not worthy of political analysis, and to associations of international law with idealism. The analysis notes an inverse blind spot in international law (IL), wherein legal scholars tend to regard IL as autonomous of politics and the political economy of ‘who gets what, when, how’. These blind spots contribute to the neglect of how legal forms function to mystify the political economy of IL, thereby advancing transnational capitalism and inequality. The empirical focus is on transnational contract law, dispute settlement, and value and production chain contracting. These laws function as Capital’s law and central pillars of the transnational politico-legal order by reaching deep inside states to restructure domestic spheres according to new constitutionalism and the demands of transnational capital. Drawing on critical political economy, the paper develops a historical materialist analysis to denaturalize and demystify Capital’s law. This is a necessary move for scholars interested in the progressive and emancipatory potential of IL.

Do the paper’s connections to existing work disclose its citation impact? A study based on graph representation learning

Influential scientific papers tend to be primarily based on combinations of prior works. However, assessing the potential impact of a new scientific paper remains a challenging task. In this article, we introduce an innovative framework to investigate the relationship between the embedding of citation networks and a paper’s future citation counts, based on the graph representation learning approach. First, we employ three Nobel Prize-winning topic papers from the Web of Science as our data source. Through data preprocessing and direct citation network modelling, we train the struc2vec model to obtain embeddings of papers’ citation network structure. Then, we perform visualisation and analysis on two types of networks. One is the direct-citation network, in which we identify four patterns of linkage between newly published papers and existing knowledge, and the other is the co-citation network, where we measure three structural variation indicators of new papers based on existing research findings. Finally, a statistical test is used to examine the predictive potentials of network embeddings. The results demonstrate that the structural features captured by the graph representation learning model can be used to predict a paper’s citation counts and impact. This article innovatively combines cluster analysis, visual analysis and statistical analysis to gain insights into the relationship between the hard-to-explain structural embeddings of newly published papers in a citation network and their future citations.

TI-MVD: A temporal interaction-enhanced model for malware variants detection

Advanced malware variants attacks have been posing catastrophes to the cyber ecosystem. However, existing malware variants detection methods are feeble for detecting the advanced malware variants due to the twofold flaws. First, most detection methods focus on analyzing the isolated features instead of investigating the meaningful contextual interactions between fine-grained malware entities, resulting in poor performance. Second, the existing graph-based detection approaches are incapable of leveraging the temporal dependence information between execution behaviors to capture the malicious evolutionary patterns and incur expensive time costs when traversing vast invalid paths. To overcome these limitations, this paper proposes TI-MVD, a temporal interaction-enhanced malware variants detection framework. TI-MVD models the fine-grained malware objects with a temporal heterogeneous graph, which can simultaneously leverage the temporal and structural embedding features to detect malware variants. Concretely, a novel end-to-end interaction-enhanced embedding approach is proposed to learn the structural embedding, which is capable of incorporating explicit and implicit interactive information between node pairs to boost detection effectiveness. Meanwhile, a strong-correlated clique method exploiting two coupled GRUs is presented to handle the temporal interactions in a parallel manner, which can drastically reduce the time cost of temporal embedding. Experimental results on four real-world datasets demonstrate that our proposed TI-MVD outperforms the state-of-the-art methods by a large margin.

Graph clustering network with structure embedding enhanced

Recently, deep clustering utilizing Graph Neural Networks has shown good performance in the graph clustering. However, the structure information of graph was underused in existing deep clustering methods. Particularly, the lack of concern on mining different types structure information simultaneously. To tackle with the problem, this paper proposes a Graph Clustering Network with Structure Embedding Enhanced (GC-SEE) which extracts nodes importance-based and attributes importance-based structure information via a feature attention fusion graph convolution module and a graph attention encoder module respectively. Additionally, it captures different orders-based structure information through multi-scale feature fusion. Finally, a self-supervised learning module has been designed to integrate different types structure information and guide the updates of the GC-SEE. The comprehensive experiments on benchmark datasets commonly used demonstrate the superiority of the GC-SEE. The results showcase the effectiveness of the GC-SEE in exploiting multiple types of structure for deep clustering.

Integrating hybrid piezoelectric actuators with a single fiber Bragg grating sensor for online monitoring of structures

The application of fiber Bragg grating (FBG) sensors for detecting ultrasonic guided waves (GW) in a hybrid system with piezoelectric transducers (PZT) is a promising area for structural integrity analysis due to FBG’s potential for embedding in structures. However, the directional sensitivity of FBG sensors presents a challenge for signal processing and the direct application of damage mapping methods designed for omnidirectional PZT sensors is not feasible. The novelty of this study lies in the development of a proof-of-study damage mapping method that uses an improved cosine distance (CD) formulation to detect damage paths while accounting for the directional sensitivity of FBG sensors. Additionally, a sectorial elliptical code (SEC) based on CD was used to localize damages in a hybrid FBG-PZT network. This approach was validated with numerical simulations, demonstrating its effectiveness in accurately detecting and localizing damages in structures. The damage detection with the CD method located all the damage paths which help to identify the probable damage region. The SEC method localized damages not only in the inner sensor zone but also in the outer zone. The CD method successfully predicted the probable damage zones overcoming the directional sensitivity issue. The error predicted with the SEC method was as low as 0.7 cm in the experimental and 0.1 cm in numerical studies.

Detecting bots in social-networks using node and structural embeddings

Users on social networks such as Twitter interact with each other without much knowledge of the real-identity behind the accounts they interact with. This anonymity has created a perfect environment for bot accounts to influence the network by mimicking real-user behaviour. Although not all bot accounts have malicious intent, identifying bot accounts in general is an important and difficult task. In the literature there are three distinct types of feature sets one could use for building machine learning models for classifying bot accounts. These feature-sets are: user profile metadata, natural language features (NLP) extracted from user tweets and finally features extracted from the the underlying social network. Profile metadata and NLP features are typically explored in detail in the bot-detection literature. At the same time less attention has been given to the predictive power of features that can be extracted from the underlying network structure. To fill this gap we explore and compare two classes of embedding algorithms that can be used to take advantage of information that network structure provides. The first class are classical embedding techniques, which focus on learning proximity information. The second class are structural embedding algorithms, which capture the local structure of node neighbourhood. We show that features created using structural embeddings have higher predictive power when it comes to bot detection. This supports the hypothesis that the local social network formed around bot accounts on Twitter contains valuable information that can be used to identify bot accounts.

A Unipolar Quantum Dot Diode Structure for Advanced Quantum Light Sources.

Triggered, indistinguishable single photons are crucial in various quantum photonic implementations. Here, we realize a novel n+-i-n++ diode structure embedding semiconductor quantum dots: the gated device enables spectral tuning of the transitions and deterministic control of the charged states. Blinking-free single-photon emission and high two-photon indistinguishability are observed. The line width's temporal evolution is investigated across over 6 orders of magnitude time scales, combining photon-correlation Fourier spectroscopy, high-resolution photoluminescence spectroscopy, and two-photon interference (visibility of VTPI,2ns = (85.8 ± 2.2)% and VTPI,9ns = (78.3 ± 3.0)%). Most of the dots show no spectral broadening beyond ∼9 ns time scales, and the photons' line width ((420 ± 30) MHz) deviates from the Fourier-transform limit by a factor of 1.68. The combined techniques verify that most dephasing mechanisms occur at time scales ≤2 ns, despite their modest impact. The presence of n-doping implies higher carrier mobility, enhancing the device's appeal for high-speed tunable, high-performance quantum light sources.

Neighbor group structure preserving based consensus graph learning for incomplete multi-view clustering

In the area of clustering, multi-view clustering has drawn a lot of research attention by making full use of information from different views. In many practical applications, collecting complete multi-view data without missing views is sometimes expensive and impossible. Therefore, the study in incomplete multi-view clustering has become a trend in the field of clustering analysis. Graph learning-based approach is one of the most effective tools. The essence of graph learning is how to construct the affinity graph or similarity matrix, whose elements depict the similarity of the corresponding sample pairs. In this paper, we propose a new method, called Neighbor Group Structure Preserving-based Consensus Graph Learning (NGSP_CGL), to learn a high-quality consensus graph for incomplete multi-view clustering. Different from the existing graph learning-based works which only focus on the relationship between isolated sample pairs, NGSP_CGL seeks to explore the neighbor group structure corresponding to the nearest neighbor sets of sample pairs and designs a novel but simple nearest neighbor group structure embedding constraint so as to enhance the quality of consensus graph. The experimental results on several datasets demonstrate the effectiveness of NGSP_CGL.

Incorporating logic rules with textual representations for interpretable knowledge graph reasoning

Reasoning on knowledge graphs (KGs) is significant for downstream applications, such as question answering and information extraction. On the basis of using factual triples in KGs, learning logic rules has been explored to handle KG reasoning in an interpretable pattern. Some rule learning methods simply apply thresholds to reduce the exponential complexity caused by the vast candidate rules derived from KGs. Others are designed to prune candidates on the basis of embedding facts in the KG. However, these methods only consider the factual triples, which mainly contain structural information, and ignore sufficient semantic information in real-world KGs. This situation weakens the methods in searching for high-quality rules. To this end, we propose a joint method by incorporating logic rules with textual representations for interpretable KG reasoning, named LoTus. Firstly, LoTus obtains the structural embeddings with only factual triples, and the textual embeddings with descriptions of entities, which are later integrated into the representations of entities and relations. Secondly, top candidate rules are selected by a proposed joint pruning strategy incorporating structural and textual representations in KGs. Finally, LoTus captures high-quality rules and removes low-quality ones by a filtering process with metrics of rules. It reduces the time complexity on KG reasoning by learning rules and provides interpretability simultaneously. Extensive experiments on different datasets show that LoTus obtains competitive performance comparing to baselines on KG reasoning metrics Hits@1, Hits@10 and mean reciprocal rank (MRR), and achieves significant superiority among the recent rule learning methods.

Ontology alignment with semantic and structural embeddings

Ontology alignment is essential for data integration and interoperability across multiple applications across diverse disciplines. In recent decades, significant advancements have been made in the development of advanced methods and systems for ontology alignment. Empirical results have suggested that ontological semantics can be effectively employed to enhance the alignment process. Besides, structural information is crucial for ontology alignment as it reflects the relations among adjacent concepts in the ontology. Previous works are mainly based on external lexicon and predefined rules based on ontological structure. Recently, deep learning has imposed positive impacts on ontology alignment and obtained substantial improvement.This paper proposes a new method based on ontology embedding incorporating the semantic and structural features. It utilizes the distance between the embedding of two ontological concepts to be aligned as the criterion for alignment. The proposed method is used to align two widely used food ontologies and three Chinese food classification ontologies. The experimental results show that our method enhances the performance compared to several state-of-the-art alignment systems, demonstrating the importance of learning semantic representation and structural representation. Furthermore, the proposed method is evaluated on several different tracks of the Ontology Alignment Evaluation Initiative (OAEI), and experimental results show that our method outperforms other baselines in effectiveness. The data and code can be obtained from: https://github.com/haozhigang1111/Ontology-Alignment.git.

Geometry-Aware Planar Embedding of Treelike Structures

The growing complexity of spatial and structural information in 3D data makes data inspection and visualization a challenging task. We describe a method to create a planar embedding of 3D treelike structures using their skeleton representations. Our method maintains the original geometry, without overlaps, to the best extent possible, allowing exploration of the topology within a single view. We present a novel camera view generation method which maximizes the visible geometric attributes (segment shape and relative placement between segments). Camera views are created for individual segments and are used to determine local bending angles at each node by projecting them to 2D. The final embedding is generated by minimizing an energy function (the weights of which are user adjustable) based on branch length and the 2D angles, while avoiding intersections. The user can also interactively modify segment placement within the 2D embedding, and the overall embedding will update accordingly. A global to local interactive exploration is provided using hierarchical camera views that are created for subtrees within the structure. We evaluate our method both qualitatively and quantitatively and demonstrate our results by constructing planar visualizations of line data (traced neurons) and volume data (CT vascular and bronchial data).

Hard Sample Aware Network for Contrastive Deep Graph Clustering

Contrastive deep graph clustering, which aims to divide nodes into disjoint groups via contrastive mechanisms, is a challenging research spot. Among the recent works, hard sample mining-based algorithms have achieved great attention for their promising performance. However, we find that the existing hard sample mining methods have two problems as follows. 1) In the hardness measurement, the important structural information is overlooked for similarity calculation, degrading the representativeness of the selected hard negative samples. 2) Previous works merely focus on the hard negative sample pairs while neglecting the hard positive sample pairs. Nevertheless, samples within the same cluster but with low similarity should also be carefully learned. To solve the problems, we propose a novel contrastive deep graph clustering method dubbed Hard Sample Aware Network (HSAN) by introducing a comprehensive similarity measure criterion and a general dynamic sample weighing strategy. Concretely, in our algorithm, the similarities between samples are calculated by considering both the attribute embeddings and the structure embeddings, better revealing sample relationships and assisting hardness measurement. Moreover, under the guidance of the carefully collected high-confidence clustering information, our proposed weight modulating function will first recognize the positive and negative samples and then dynamically up-weight the hard sample pairs while down-weighting the easy ones. In this way, our method can mine not only the hard negative samples but also the hard positive sample, thus improving the discriminative capability of the samples further. Extensive experiments and analyses demonstrate the superiority and effectiveness of our proposed method. The source code of HSAN is shared at https://github.com/yueliu1999/HSAN and a collection (papers, codes and, datasets) of deep graph clustering is shared at https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering on Github.