Embedding Framework Research Articles

Benzimidazole-based covalent organic framework embedding single-atom Pt sites for visible-light-driven photocatalytic hydrogen evolution

Visible-light-driven photocatalytic hydrogen evolution reaction (HER) over a semiconductor provides an effective avenue to produce renewable clean energy and alleviate energy and environmental crises. However, the HER efficiency is still limited by the sluggish electron transfer process. Herein, a highly active covalent organic framework (COF) was constructed from the unusual benzimidazole monomer in a microwave-assisted solvothermal pathway. With single-atom Pt sites as cocatalyst, the catalyst exhibited an HER rate up to 115 mmol g–1 h–1 and a high turnover frequency of 4475.1 h–1 under visible-light irradiation. The above performance relied on the combination of benzimidazole moieties and COF framework, which, on the one hand, stabilized photogenerated electrons to prolong the electron lifetime, and on the other hand provided a strong host-guest interaction that resulted in the creation of single-atom Pt sites and the acceleration of the electron-transfer to the active sites for proton reduction. This work demonstrates the perspective of electron stabilization and interfacial charge transfer avenue construction in the HER process, which can be reached by a molecular-level design of COF-based organic semiconductors by using structural and functional diverse asymmetric building blocks.

A confidence-aware and path-enhanced convolutional neural network embedding framework on noisy knowledge graph

In recent years, Knowledge Graphs (KGs) have been widely used in applications such as search engines and Q&A systems. During construction, errors were inevitably introduced and KGs may contain many incorrect facts. However, most KG representation learning methods assume that all triple facts are correct, making KG noise detection a challenge. Although some knowledge representation methods are based on a confidence-aware framework, they are usually limited by the translation assumption in TransE and confidence calculation methods. To address the above problems, we propose a more accurate and effective confidence-aware and path-enhanced convolutional neural network knowledge embedding framework (CPConvKE). It makes use of structural, entity type, and rule information for knowledge graph noise detection and knowledge representation learning simultaneously. Specifically, for representation learning, we introduce a gating-based path embedding method to filter out noise while learning entity and relation embeddings. For the confidence evaluation, we propose a triple confidence estimator that uses entity type and rule information as prior probability and defines posterior probability based on the embedding. We evaluate the effectiveness of our model on the knowledge graph noise detection, completion, and triple classification tasks. The experimental results show that our confidence-aware model achieves significant and consistent improvements on all tasks, which confirms the ability of CPConvKE to detect noise and learn clean embeddings in a noisy knowledge graph.

DeHIN: A Decentralized Framework for Embedding Large-Scale Heterogeneous Information Networks

Modeling heterogeneity by extraction and exploitation of high-order information from heterogeneous information networks (HINs) has been attracting immense research attention in recent times. Such heterogeneous network embedding (HNE) methods effectively harness the heterogeneity of small-scale HINs. However, in the real world, the size of HINs grow exponentially with the continuous introduction of new nodes and different types of links, making it a billion-scale network. Learning node embeddings on such HINs creates a performance bottleneck for existing HNE methods that are commonly centralized, i.e., complete data and the model are both on a single machine. To address large-scale HNE tasks with strong efficiency and effectiveness guarantee, we present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Decentralized Embedding Framework for Heterogeneous Information Network</i> (DeHIN) in this paper. In DeHIN, we generate a distributed parallel pipeline that utilizes hypergraphs in order to infuse parallelization into the HNE task. DeHIN presents a context preserving partition mechanism that innovatively formulates a large HIN as a hypergraph, whose hyperedges connect semantically similar nodes. Our framework then adopts a decentralized strategy to efficiently partition HINs by adopting a tree-like pipeline. Then, each resulting subnetwork is assigned to a distributed worker, which employs the deep information maximization theorem to locally learn node embeddings from the partition it receives. We further devise a novel embedding alignment scheme to precisely project independently learned node embeddings from all subnetworks onto a common vector space, thus allowing for downstream tasks like link prediction and node classification. As shown from our experimental results, DeHIN significantly improves the efficiency and accuracy of existing HNE models as well as outperforms the large-scale graph embedding frameworks by efficiently scaling up to large-scale HINs.

Network Embedding via Deep Prediction Model

Network-structured data becomes ubiquitous in daily life and is growing at a rapid pace. It presents great challenges to feature engineering due to the high non-linearity and sparsity of the data. The local and global structure of the real-world networks can be reflected by dynamical transfer behaviors among nodes. This paper proposes a network embedding framework to capture the transfer behaviors on structured networks via deep prediction models. We first design a degree-weight biased random walk model to capture the transfer behaviors on the network. Then a deep network embedding method is introduced to preserve the transfer possibilities among the nodes. A network structure embedding layer is added into conventional deep prediction models, including Long Short-Term Memory Network and Recurrent Neural Network, to utilize the sequence prediction ability. To keep the local network neighborhood, we further perform a Laplacian supervised space optimization on the embedding feature representations. Experimental studies are conducted on various datasets including social networks, citation networks, biomedical network, collaboration network and language network. The results show that the learned representations can be effectively used as features in a variety of tasks, such as clustering, visualization, classification, reconstruction and link recovery, and achieve promising performance compared with state-of-the-arts.

Resource allocation for dataflow applications in FANETs using anypath routing

Management of network resources in advanced IoT applications is a challenging topic due to their distributed nature from the Edge to the Cloud, and the heavy demand of real-time data from many sources to take action in the deployment. FANETs (Flying Ad-hoc Networks) are a clear example of heterogeneous multi-modal use cases, which require strict quality in the network communications, as well as the coordination of the computing capabilities, in order to operate correctly the final service. In this paper, we present a Virtual Network Embedding (VNE) framework designed for the allocation of dataflow applications, composed of nano-services that produce or consume data, in a wireless infrastructure, such as an airborne network. To address the problem, an anypath-based heuristic algorithm that considers the quality demand of the communication between nano-services is proposed, coined as Quality-Revenue Paired Anypath Dataflow VNE (QRPAD-VNE). We also provide a simulation environment for the evaluation of its performance according to the virtual network (VN) request load in the system. Finally, we show the suitability of a multi-parameter framework in conjunction with anypath routing in order to have better performance results that guarantee minimum quality in the wireless communications.

Distributed Graph Embedding with Information-Oriented Random Walks

Graph embedding maps graph nodes to low-dimensional vectors, and is widely adopted in machine learning tasks. The increasing availability of billion-edge graphs underscores the importance of learning efficient and effective embeddings on large graphs, such as link prediction on Twitter with over one billion edges. Most existing graph embedding methods fall short of reaching high data scalability. In this paper, we present a general-purpose, distributed, information-centric random walk-based graph embedding framework, DistGER, which can scale to embed billion-edge graphs. DistGER incrementally computes information-centric random walks. It further leverages a multi-proximity-aware, streaming, parallel graph partitioning strategy, simultaneously achieving high local partition quality and excellent workload balancing across machines. DistGER also improves the distributed Skip-Gram learning model to generate node embeddings by optimizing the access locality, CPU throughput, and synchronization efficiency. Experiments on real-world graphs demonstrate that compared to state-of-the-art distributed graph embedding frameworks, including KnightKing, DistDGL, and Pytorch-BigGraph, DistGER exhibits 2.33×--129× acceleration, 45% reduction in cross-machines communication, and &gt;10% effectiveness improvement in downstream tasks.

Crowdsourcing Truth Inference via Reliability-Driven Multi-View Graph Embedding

Crowdsourcing truth inference aims to assign a correct answer to each task from candidate answers that are provided by crowdsourced workers. A common approach is to generate workers’ reliabilities to represent the quality of answers. Although crowdsourced triples can be converted into various crowdsourced relationships, the available related methods are not effective in capturing these relationships to alleviate the harm to inference that is caused by conflicting answers. In this research, we propose aReliability-drivenMulti-viewGraphEmbedding framework forTruthinference (TiReMGE), which explores multiple crowdsourced relationships by organically integrating worker reliabilities into a graph space that is constructed from crowdsourced triples. Specifically, to create an interactive environment, we propose a reliability-driven initialization criterion for initializing vectors of tasks and workers as interactive carriers of reliabilities. From the perspective of multiple crowdsourced relationships, a multi-view graph embedding framework is proposed for reliability information interaction on a task-worker graph, which encodes latent crowdsourced relationships into vectors of workers and tasks for reliability update and truth inference. A heritable reliability updating method based on the Lagrange multiplier method is proposed to obtain reliabilities that match the quality of workers for interaction by a novel constraint law. Our ultimate goal is to minimize the Euclidean distance between the encoded task vector and the answer that is provided by a worker with high reliability. Extensive experimental results on nine real-world datasets demonstrate that TiReMGE significantly outperforms the nine state-of-the-art baselines.

Multimodal interaction aware embedding for location-based social networks

Location-based social networks (LBSNs) have greatly promoted the development of the field of human mobility mining. However, the sparsity, multimodality and heterogeneity nature of the user check-in data remains a great concern for learning high-quality user or other entity representations, especially in the downstream application tasks, such as point-of-interest (POI) recommendation. Most existing methods focus on user preference modeling based on sequential POI tags without exploring the interaction between different modalities (e.g., user-user interactions, user-timestamp interactions, user-POI interactions, etc.). To this end, we introduce a multimodal interaction aware embedding framework to generate reliable entity embeddings on the heterogeneous socio-spatial network. At its core, first, multi-modal interaction sub-graph sampling techniques are designed to capture the heterogeneous contexts; then, a self-supervised contrastive learning technique is leveraged to extract intra-modality and inter-modality interactions in a light way. We conduct experiments on the next-POI recommendation tasks based on three real-world datasets. Experimental results demonstrate the superiority of our model over the state-of-the-art embedding learning algorithms.

Multi-View Multi-Task Campaign Embedding for Cold-Start Conversion Rate Forecasting

In online advertising, it is critical for advertisers to forecast conversion rate (CVR) of campaigns. Previous work on campaign forecasting concentrates on the time-series analysis which depend on the availability of a length of history. However, these approaches become inadequate for cold-start campaigns which lack for the observation of past. In this work, we attempt to mitigate this challenge by learning an unsupervised and composite campaign embedding to capture multi-view semantic relationships on campaign information, and consequently forecasting the cold-start campaigns using the nearest neighbor campaigns. Specifically, we propose a novel embedding framework which simultaneously extracts and fuses heterogeneous knowledge from multiple views of campaign data in a multi-task learning fashion, to learn the semantic relationship of ad message, conversion rule, and audience targeting. We develop a hierarchical attention mechanism to refine the embedding model at two levels - an intra-view attention to improve context aggregation, and an inter-task attention to balance task importance. Finally, we adopt the k-NN regression model to predict the CVR based on the neighboring campaigns in the embedding space which encodes the multi-view campaign proximity. We conduct extensive experiments on a real-world advertising campaign dataset. The results demonstrate the effectiveness of the proposed embedding method for CVR forecasting in cold-start scenarios.

HWNet v3: a joint embedding framework for recognition and retrieval of handwritten text

HWNet v3: a joint embedding framework for recognition and retrieval of handwritten text

Advanced audio coding steganography algorithm with distortion minimization model based on audio beat

Currently, most advanced audio coding (AAC) steganography methods are content-non-adaptive without considering the characteristics of audio, and there are several limitations in imperceptibility and steganalysis. In this paper, we use an audio feature beat as the anchor point to identify the cover elements, group the quantized modified discrete cosine transform (QMDCT) coefficients in the small value area, and finally use the syndrome-trellis codes (STCs) framework for content-adaptive embedding to obtain the minimum distortion. In the STCs framework, we comprehensively consider auditory and data distortions. Experimental results demonstrate that the proposed steganography algorithm has a 10% improvement over the compared algorithms in terms of imperceptibility and steganalysis, and it can accurately extract secret information in face of frame loss and misalignment.

HyperDNE: Enhanced hypergraph neural network for dynamic network embedding

Representation learning provides an attractive opportunity to model the evolution of dynamic networks. However, the existing methods have two limitations: (1) most graph neural network-based methods fail to utilize the high-order proximity of nodes that captures the important properties of a network topology; (2) evolutionary dynamics-based methods are much fine-grained in modeling time information but neglect the coherence of dynamic networks, which leads to the model being susceptible to subtle noise. In this paper, we propose an enhanced hypergraph neural network framework for dynamic network embedding (HyperDNE) to tackle these issues. Specifically, we innovatively design a sequential hypergraph with dual-stream output to explore the group properties of nodes and edges, and a line graph neural network is added as an auxiliary enhancement scheme to further aggregate social influence from the degree of social convergence. Then, we compute the final embedding through attentions along the node and hyperedge levels to fuse multi-level variations in the network structure. The experimental results on six real networks demonstrate significant gains for HyperDNE over several state-of-the-art network embedding baselines. The dataset and source code of HyperDNE are publicly available at https://github.com/qhgz2013/HyperDNE.

An efficient protocol for excited states of large biochromophores.

Efficient energy transport in photosynthetic antenna is a long-standing source of inspiration for artificial light harvesting materials. However, characterizing the excited states of the constituent chromophores poses a considerable challenge to mainstream quantum chemical and semiempirical excited state methods due to their size and complexity and the accuracy required to describe small but functionally important changes in their properties. In this paper, we explore an alternative approach to calculating the excited states of large biochromophores, exemplified by a specific method for calculating the Qy transition of bacteriochlorophyll a, which we name Chl-xTB. Using a diagonally dominant approximation to the Casida equation and a bespoke parameterization scheme, Chl-xTB can match time-dependent density functional theory's accuracy and semiempirical speed for calculating the potential energy surfaces and absorption spectra of chlorophylls. We demonstrate that Chl-xTB (and other prospective realizations of our protocol) can be integrated into multiscale models, including concurrent excitonic and point-charge embedding frameworks, enabling the analysis of biochromophore networks in a native environment. We exploit this capability to probe the low-frequency spectral densities of excitonic energies and interchromophore interactions in the light harvesting antenna protein LH2 (light harvesting complex 2). The impact of low-frequency protein motion on interchromophore coupling and exciton transport has routinely been ignored due to the prohibitive costs of including it in simulations. Our results provide a more rigorous basis for continued use of this approximation by demonstrating that exciton transition energies are unaffected by low-frequency vibrational coupling to exciton interaction energies.

Adaptive Reference-Related Graph Embedding for Hyperspectral Anomaly Detection

Graph embedding provides an effective way to reveal intrinsic feature of high-dimensional data on the foundation of preserving topological properties. Under the framework of graph embedding, the hyperspectral image can be represented by a weighted graph, where pixels and similarities among them are treated as vertices and edge weights, respectively. In this article, an adaptive reference-related graph embedding (ARGE) method is proposed to efficaciously obtain the low-dimensional feature and improve computational efficiency. The ARGE method is composed of two primary processes. The key to connecting these two processes is reference vertices set, which is the abstraction of graph topological features. Firstly, the reference vertices are adaptively selected through a three-step adaptive reference set selection (ARSS) algorithm. Secondly, the original high-dimensional graph is embedded as a low-dimensional graph through preserving the reference-related structure. Specifically, the pairwise similarities between vertices and reference vertices are preserved in embedding space. In addition, a new hybrid dissimilarity measure of Rao distance and spectral information divergence (RD-SID) is designed to depict the spectral difference between pixels. To evaluate the effectiveness of the proposed method, the obtained low-dimensional feature is fed into the anomaly detector to detect anomalous pixels. The experimental results on five real and one synthetic hyperspectral datasets demonstrate the superiority of the proposed ARGE method over the compared feature extraction methods.

Dense Video Captioning With Early Linguistic Information Fusion

Dense captioning methods generally detect events in videos first and then generate captions for the individual events. Events are localized solely based on the visual cues while ignoring the associated linguistic information and context. Whereas end-to-end learning may implicitly take guidance from language, these methods still fall short of the power of explicit modeling. In this paper, we propose a Visual-Semantic Embedding (ViSE) Framework that models the word(s)-context distributional properties over the entire semantic space and computes weights for all the n-grams such that higher weights are assigned to the more informative n-grams. The weights are accounted for in learning distributed representations of all the captions to construct a semantic space. To perform the contextualization of visual information and the constructed semantic space in a supervised manner, we design Visual-Semantic Joint Modeling Network (VSJM-Net). The learned ViSE embeddings are then temporally encoded with a Hierarchical Descriptor Transformer (HDT). For caption generation, we exploit a transformer architecture to decode the input embeddings into natural language descriptions. Experiments on the large-scale ActivityNet Captions dataset and YouCook-II dataset demonstrate the efficacy of our method.

A Scalable Unsupervised Feature Selection With Orthogonal Graph Representation for Hyperspectral Images

Feature selection is essential in various fields of science and engineering, from remote sensing to computer vision. Reducing data dimensionality by removing redundant features and selecting the most informative ones improves machine learning algorithms’ performance, especially in supervised classification tasks, while lowering storage needs. Graph-embedding techniques have recently been found efficient for feature selection since they preserve the geometric structure of the original feature space while embedding data into a low-dimensional subspace. However, the main drawback is the high computational cost of solving an eigenvalue decomposition problem, especially for large-scale problems. This paper addresses this issue by combining the graph embedding framework and representation theory for a novel feature selection method. Inspired by the high dimensional model representation, the feature transformation is assumed to be a linear combination of a set of univariate orthogonal functions carried out in the graph embedding framework. As a result, an explicit embedding function is created, which can be utilised to embed out-of-samples into low-dimensional space and provide a feature relevance score. The significant contribution of the proposed method is to divide an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional generalised eigenvalue problem into <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> small-sized eigenvalue problems. With this property, the computational complexity of the graph embedding is significantly reduced, resulting in a scalable feature selection method, which could be easily parallelized too. The performance of the proposed method is compared favourably to its counterparts in high-dimensional hyperspectral image processing in terms of classification accuracy, feature stability, and computational time.

A Knowledge Graph Embedding Framework With Triple Semantics

The knowledge graph embedding model aims to use low-dimensional real-valued vectors to represent the entities and relations in the triples, where operations such as link prediction and triple classification can be performed based on these representations. However, existing embedding models only consider the structural embedding of triples, while ignoring the semantic information of triples. This paper proposes a knowledge graph embedding learning framework combined with triple semantic information (KGSE). KGSE comprehensively considers the structural embedding and semantic embedding of triples, where semantic embedding is used as a supplement to improve the quality of embedding. Specifically, KGSE uses the improved TransD model to obtain the structural embedding of triples, and employs the deep convolutional neural model combined with an attention mechanism to obtain the semantic embedding of triples. In addition, a novel energy function is designed to jointly train the above two embeddings. Experimental results show that the proposed framework improves significantly compared with Trans-based models and other baseline models in link prediction and triple classification tasks, which verifies the effectiveness of the proposed framework.

Embedding and Trajectories of Temporal Networks

Temporal network data are increasingly available in various domains, and often represent highly complex systems with intricate structural and temporal evolutions. Due to the difficulty of processing such complex data, it may be useful to coarse grain temporal network data into a numeric trajectory embedded in a low-dimensional space. We refer to such a procedure as temporal network embedding, which is distinct from procedures that aim at embedding individual nodes. Temporal network embedding is a challenging task because we often have access only to discrete time-stamped events between node pairs, and, in general, the events occur with irregular intervals, making the construction of the network at a given time a nontrivial question already. We propose a method to generate trajectories of temporal networks embedded in a low-dimensional space given a sequence of time-stamped events as input. We realize this goal by combining the landmark multidimensional scaling, which is an out-of-sample extension of the well-known multidimensional scaling method, and the framework of tie-decay temporal networks. This combination enables us to obtain a continuous-time trajectory describing the evolution of temporal networks. We then study mathematical properties of the proposed temporal network embedding framework. Finally, we showcase the method with empirical data of social contacts to find temporal organization of contact events and loss of them over a single day and across different days.

Knowledge Graph Representation Learning Based on Automatic Network Search for Link Prediction

Link prediction, also known as Knowledge Graph Completion (KGC), is the common task in Knowledge Graphs (KGs) to predict missing connections between entities. Most existing methods focus on designing shallow, scalable models, which have less expressive than deep, multi-layer models. Furthermore, most operations like addition, matrix multiplications or factorization are handcrafted based on a few known relation patterns in several well-known datasets, such as FB15k, WN18, etc. However, due to the diversity and complex nature of real-world data distribution, it is inherently difficult to preset all latent patterns. To address this issue, we propose KGE-ANS, a novel knowledge graph embedding framework for general link prediction tasks using automatic network search. KGE-ANS can learn a deep, multi-layer effective architecture to adapt to different datasets through neural architecture search. In addition, the general search space we designed is tailored for KG tasks. We perform extensive experiments on benchmark datasets and the dataset constructed in this paper. The results show that our KGE-ANS outperforms several state-of-the-art methods, especially on these datasets with complex relation patterns.

A unified network embedding algorithm for multi-type similarity measures

Traditional network embedding aims to learn representations by capturing a predefined vertex-to-vertex similarity measure. However, in practice, there are different types of similarity measures (e.g., connectivity and structural similarity), which are appropriate for different downstream applications. Meanwhile, it is hard to select the “best” similarity measure that can mostly benefit the application, considering the required domain knowledge of both application scenario and network science. It sometimes requires to cooperate these similarity measures with each other for achieving better performance. Therefore, automatically integrate multiple types of similarity measures into a uniform network embedding framework is critical to obtain effective vertex representations for a downstream application. In this paper, we address the above problem in social networks, and propose a semi-supervised representation learning algorithm. The general idea of our approach is to impose social influence, which occurs when one’s opinions, emotions, or behaviors are affected by others in a social network. Particularly, we build the connection between a user’s representation vector and the probability of her being influenced by another user to have a particular label (e.g., fraud, personal interest, etc.). We conduct efficient experiments based on six real-world datasets and find a clear improvement of our approach comparing with several state-of-the-art baselines.