Proposed Embedding Method Research Articles

Transferable non-invasive modal fusion-transformer (NIMFT) for end-to-end hand gesture recognition.

Objective.Recent studies have shown that integrating inertial measurement unit (IMU) signals with surface electromyographic (sEMG) can greatly improve hand gesture recognition (HGR) performance in applications such as prosthetic control and rehabilitation training. However, current deep learning models for multimodal HGR encounter difficulties in invasive modal fusion, complex feature extraction from heterogeneous signals, and limited inter-subject model generalization. To address these challenges, this study aims to develop an end-to-end and inter-subject transferable model that utilizes non-invasively fused sEMG and acceleration (ACC) data.Approach.The proposed non-invasive modal fusion-transformer (NIMFT) model utilizes 1D-convolutional neural networks-based patch embedding for local information extraction and employs a multi-head cross-attention (MCA) mechanism to non-invasively integrate sEMG and ACC signals, stabilizing the variability induced by sEMG. The proposed architecture undergoes detailed ablation studies after hyperparameter tuning. Transfer learning is employed by fine-tuning a pre-trained model on new subject and a comparative analysis is performed between the fine-tuning and subject-specific model. Additionally, the performance of NIMFT is compared to state-of-the-art fusion models.Main results.The NIMFT model achieved recognition accuracies of 93.91%, 91.02%, and 95.56% on the three action sets in the Ninapro DB2 dataset. The proposed embedding method and MCA outperformed the traditional invasive modal fusion transformer by 2.01% (embedding) and 1.23% (fusion), respectively. In comparison to subject-specific models, the fine-tuning model exhibited the highest average accuracy improvement of 2.26%, achieving a final accuracy of 96.13%. Moreover, the NIMFT model demonstrated superiority in terms of accuracy, recall, precision, and F1-score compared to the latest modal fusion models with similar model scale.Significance.The NIMFT is a novel end-to-end HGR model, utilizes a non-invasive MCA mechanism to integrate long-range intermodal information effectively. Compared to recent modal fusion models, it demonstrates superior performance in inter-subject experiments and offers higher training efficiency and accuracy levels through transfer learning than subject-specific approaches.

Topological feature generation for link prediction in biological networks.

Graph or network embedding is a powerful method for extracting missing or potential information from interactions between nodes in biological networks. Graph embedding methods learn representations of nodes and interactions in a graph with low-dimensional vectors, which facilitates research to predict potential interactions in networks. However, most graph embedding methods suffer from high computational costs in the form of high computational complexity of the embedding methods and learning times of the classifier, as well as the high dimensionality of complex biological networks. To address these challenges, in this study, we use the Chopper algorithm as an alternative approach to graph embedding, which accelerates the iterative processes and thus reduces the running time of the iterative algorithms for three different (nervous system, blood, heart) undirected protein-protein interaction (PPI) networks. Due to the high dimensionality of the matrix obtained after the embedding process, the data are transformed into a smaller representation by applying feature regularization techniques. We evaluated the performance of the proposed method by comparing it with state-of-the-art methods. Extensive experiments demonstrate that the proposed approach reduces the learning time of the classifier and performs better in link prediction. We have also shown that the proposed embedding method is faster than state-of-the-art methods on three different PPI datasets.

Multi-View Multi-Task Campaign Embedding for Cold-Start Conversion Rate Forecasting

In online advertising, it is critical for advertisers to forecast conversion rate (CVR) of campaigns. Previous work on campaign forecasting concentrates on the time-series analysis which depend on the availability of a length of history. However, these approaches become inadequate for cold-start campaigns which lack for the observation of past. In this work, we attempt to mitigate this challenge by learning an unsupervised and composite campaign embedding to capture multi-view semantic relationships on campaign information, and consequently forecasting the cold-start campaigns using the nearest neighbor campaigns. Specifically, we propose a novel embedding framework which simultaneously extracts and fuses heterogeneous knowledge from multiple views of campaign data in a multi-task learning fashion, to learn the semantic relationship of ad message, conversion rule, and audience targeting. We develop a hierarchical attention mechanism to refine the embedding model at two levels - an intra-view attention to improve context aggregation, and an inter-task attention to balance task importance. Finally, we adopt the k-NN regression model to predict the CVR based on the neighboring campaigns in the embedding space which encodes the multi-view campaign proximity. We conduct extensive experiments on a real-world advertising campaign dataset. The results demonstrate the effectiveness of the proposed embedding method for CVR forecasting in cold-start scenarios.

Distributed Network Embedding: A Distributed Control Approach for Undirected Networks

In this paper, a new network embedding method is proposed for an effective network representation of vertices. While the existing embedding methods have embedded a set of vertices via network representation learning in a fully centralized way, we present a novel distributed network embedding (DNE) method without representation learning. By coupling first-order proximity and second-order proximity with distance embedding constraints, a distributed and explicit embedding protocol steers a set of vertices in a vector space to preserve local and global structural information of networks. The proposed method does not require representation learning process, and is distributed, explicit, and straightforward. Therefore, we expect our approach to be fast, cost-effective, especially suitable for dynamic networks, and robust for sparse networks. Lastly, numerical experiments will briefly validate the proposed embedding method using a well-known Zachary's karate club network.

Expanded graph embedding for joint network alignment and link prediction

Link prediction in social networks has been an active field of study in recent years fueled by the rapid growth of many social networks. Many link prediction methods are harmed by users’ intention of avoiding being traced across networks. They may provide inaccurate information or overlook a great deal of information in multiple networks. This problem was overcome by developing methods for predicting links in a network based on known links in another network. Node alignment between the two networks significantly improves the efficiency of those methods. This research proposes a new embedding method to improve link prediction and node alignment results. The proposed embedding method is based on the Expanded Graph, which is our new novel network that has edges from both networks in addition to edges across the networks. Matrix factorization on the Finite Step Transition and Laplacian similarity matrices of the Expanded Graph has been used to obtain the embeddings for the nodes. Using the proposed embedding techniques, we jointly run network alignment and link prediction tasks iteratively to let them optimize each other’s results. We performed extensive experiments on many datasets to examine the proposed method. We achieved significant improvements in link prediction precision, which was 50% better than the peer’s method, and in recall, which was 500% better in some datasets. We also scale down the processing time of the solution to be more applicable to big social networks. We conclude that computed embedding in this type of problem is more suitable than learning the embedding since it shortens the processing time and gives better results.

NodeSim: node similarity based network embedding for diverse link prediction

In real-world complex networks, understanding the dynamics of their evolution has been of great interest to the scientific community. Predicting non-existent but probable links is an essential task of social network analysis as the addition or removal of the links over time leads to the network evolution. In a network, links can be categorized as intra-community links if both end nodes of the link belong to the same community, otherwise inter-community links. The existing link-prediction methods have mainly focused on achieving high accuracy for intra-community link prediction. In this work, we propose a network embedding method, called NodeSim, which captures both similarities between the nodes and the community structure while learning the low-dimensional representation of the network. The embedding is learned using the proposed NodeSim random walk, which efficiently explores the diverse neighborhood while keeping the more similar nodes closer in the context of the node. We verify the efficacy of the proposed embedding method over state-of-the-art methods using diverse link prediction. We propose a machine learning model for link prediction that considers both the nodes’ embedding and their community information to predict the link between two given nodes. Extensive experimental results on several real-world networks demonstrate the effectiveness of the proposed method for both inter and intra-community link prediction.

Automatic Conversion of Event Data to Event Logs Using CNN and Event Density Embedding

In process mining, converting event data to event logs is related to the quality of analysis results. In general, to convert event data into event logs, it is necessary to identify process entities, such as the case identifier, activity label, activity originator, and activity timestamp, from the data fields in the event data, as well as other optional attributes. Up to now, the event log conversion process has been attempted by relying on an expert’s intuition or an analyst’s experience. However, the conversion is a challenging procedure without sufficient prior knowledge of process mining. To automate the conversion process, an event log–converting algorithm based on the convolutional neural network (CNN) was developed with a new embedding method called Event Density Embedding (EDE). To verify the performance of the proposed embedding method and the automatic event log conversion framework, a comparative experiment was performed using nine pieces of real-world event data. The experiments show that our method is 5–20% higher conversion accuracy than the other methods. It is expected that business experts will be able to easily apply the method to process mining technology by utilizing system-derived event data.

Quantile–Quantile Embedding for distribution transformation and manifold embedding with ability to choose the embedding distribution

We propose a new embedding method, named Quantile–Quantile Embedding (QQE), for distribution transformation and manifold embedding with the ability to choose the embedding distribution. QQE, which uses the concept of quantile–quantile plot from visual statistical tests, can transform the distribution of data to any theoretical desired distribution or empirical reference sample. Moreover, QQE gives the user a choice of embedding distribution in embedding the manifold of data into the low dimensional embedding space. It can also be used for modifying the embedding distribution of other dimensionality reduction methods, such as PCA, t-SNE, and deep metric learning, for better representation or visualization of data. We propose QQE in both unsupervised and supervised forms. QQE can also transform a distribution to either an exact reference distribution or its shape. We show that QQE allows for better discrimination of classes in some cases. Our experiments on different synthetic and image datasets show the effectiveness of the proposed embedding method.

Inductive semi-supervised learning with Graph Convolution based regression

This brief paper introduces a framework for supervised and semi-supervised learning by estimating a non-linear embedding that incorporates Spectral Graph Convolutions structure. The proposed algorithm exploits data-driven graphs in two ways. First, it integrates data smoothness over graphs. Second, its regression loss function jointly uses the data and its graph in the sense that the regressor model sees convolved data samples. The resulting framework can solve the problem of over-fitting on local neighborhood structures for image data having varied natures like outdoor scenes, faces and man-made objects. The proposed Graph Convolution based Semi-supervised Embedding (GCSE) not only provides a new perspective to non-linear embedding research but also induces the standpoint on Spectral Graph Convolutions methods. A series of experiments are conducted on four image datasets in order to compare the proposed method with some state-of-art semi-supervised methods. This evaluation demonstrates the effectiveness of the proposed embedding method.

T-EDGE: Temporal WEighted MultiDiGraph Embedding for Ethereum Transaction Network Analysis

Recently, graph embedding techniques have been widely used in the analysis of various networks, but most of the existing embedding methods omit the network dynamics and the multiplicity of edges, so it is difficult to accurately describe the detailed characteristics of the transaction networks. Ethereum is a blockchain-based platform supporting smart contracts. The open nature of blockchain makes the transaction data on Ethereum completely public, and also brings unprecedented opportunities for the transaction network analysis. By taking the realistic rules and features of transaction networks into consideration, we first model the Ethereum transaction network as a Temporal Weighted Multidigraph (TWMDG), where each node is a unique Ethereum account and each edge represents a transaction weighted by amount and assigned with timestamp. Then we define the problem of Temporal Weighted Multidigraph Embedding (T-EDGE) by incorporating both temporal and weighted information of the edges, the purpose being to capture more comprehensive properties of dynamic transaction networks. To evaluate the effectiveness of the proposed embedding method, we conduct experiments of node classification on real-world transaction data collected from Ethereum. Experimental results demonstrate that T-EDGE outperforms baseline embedding methods, indicating that time-dependent walks and multiplicity characteristic of edges are informative and essential for time-sensitive transaction networks.

Semi-supervised elastic manifold embedding with deep learning architecture

Graph-based embedding aims to reduce the dimension of high dimensional data and to extract relevant features for learning tasks. In this letter, we propose an Elastic graph-based embedding with deep architecture which deeply explores the structural information of the data. We introduce a flexible deep learning that can overcome the limitations and weaknesses of single-layer learning models. The proposed deep architecture incorporates the geometrical manifold structure of the data. The resulting framework can be used for semi-supervised and supervised settings. Besides, the resulting optimization problems can be solved efficiently. We apply the algorithm on five public image datasets including scene, face and object datasets. These experiments demonstrate the effectiveness of the proposed embedding method, and also show that the proposed method compares favorably with many competing state-of-the-art graph-based methods.

Dynamics-Preserving Graph Embedding for Community Mining and Network Immunization

In recent years, the graph embedding approach has drawn a lot of attention in the field of network representation and analytics, the purpose of which is to automatically encode network elements into a low-dimensional vector space by preserving certain structural properties. On this basis, downstream machine learning methods can be implemented to solve static network analytic tasks, for example, node clustering based on community-preserving embeddings. However, by focusing only on structural properties, it would be difficult to characterize and manipulate various dynamics operating on the network. In the field of complex networks, epidemic spreading is one of the most typical dynamics in networks, while network immunization is one of the effective methods to suppress the epidemics. Accordingly, in this paper, we present a dynamics-preserving graph embedding method (EpiEm) to preserve the property of epidemic dynamics on networks, i.e., the infectiousness and vulnerability of network nodes. Specifically, we first generate a set of propagation sequences through simulating the Susceptible-Infectious process on a network. Then, we learn node embeddings from an influence matrix using a singular value decomposition method. Finally, we show that the node embeddings can be used to solve epidemics-related community mining and network immunization problems. The experimental results in real-world networks show that the proposed embedding method outperforms several benchmark methods with respect to both community mining and network immunization. The proposed method offers new insights into the exploration of other collective dynamics in complex networks using the graph embedding approach, such as opinion formation in social networks.

Reliable Classification of FAQs with Spelling Errors Using an Encoder-Decoder Neural Network in Korean

To resolve lexical disagreement problems between queries and frequently asked questions (FAQs), we propose a reliable sentence classification model based on an encoder-decoder neural network. The proposed model uses three types of word embeddings; fixed word embeddings for representing domain-independent meanings of words, fined-tuned word embeddings for representing domain-specific meanings of words, and character-level word embeddings for bridging lexical gaps caused by spelling errors. It also uses class embeddings to represent domain knowledge associated with each category. In the experiments with an FAQ dataset about online banking, the proposed embedding methods contributed to an improved performance of the sentence classification. In addition, the proposed model showed better performance (with an accuracy of 0.810 in the classification of 411 categories) than that of the comparison model.

ATP: Directed Graph Embedding with Asymmetric Transitivity Preservation

Directed graphs have been widely used in Community Question Answering services (CQAs) to model asymmetric relationships among different types of nodes in CQA graphs, e.g., question, answer, user. Asymmetric transitivity is an essential property of directed graphs, since it can play an important role in downstream graph inference and analysis. Question difficulty and user expertise follow the characteristic of asymmetric transitivity. Maintaining such properties, while reducing the graph to a lower dimensional vector embedding space, has been the focus of much recent research. In this paper, we tackle the challenge of directed graph embedding with asymmetric transitivity preservation and then leverage the proposed embedding method to solve a fundamental task in CQAs: how to appropriately route and assign newly posted questions to users with the suitable expertise and interest in CQAs. The technique incorporates graph hierarchy and reachability information naturally by relying on a nonlinear transformation that operates on the core reachability and implicit hierarchy within such graphs. Subsequently, the methodology levers a factorization-based approach to generate two embedding vectors for each node within the graph, to capture the asymmetric transitivity. Extensive experiments show that our framework consistently and significantly outperforms the state-of-the-art baselines on three diverse realworld tasks: link prediction, and question difficulty estimation and expert finding in online forums like Stack Exchange. Particularly, our framework can support inductive embedding learning for newly posted questions (unseen nodes during training), and therefore can properly route and assign these kinds of questions to experts in CQAs.

Joint graph based embedding and feature weighting for image classification

Recently, several inductive and flexible nonlinear data projection methods for graph-based semi-supervised learning were proposed. These state-of-the art techniques have a good performance. However, they have not taken into account the relevance of the original features in their model estimation. In this paper, we propose a joint graph-based embedding and feature weighting for getting a flexible and inductive nonlinear data representation on manifolds. The proposed criterion explicitly estimates the feature weights together with the projected data and the linear transformation such that data smoothness and large margins are achieved in the projection space. Moreover, the paper introduces a kernel variant of the model in order to get an inductive nonlinear embedding that is close to a real nonlinear subspace for a good approximation of the embedded data. The proposed frameworks can be seamlessly used in semi-supervised and supervised settings. The resulting optimization problems can be solved efficiently. The proposed embedding methods are evaluated on six public scene and face datasets. Experiments on image classification, in a semi-supervised setting, show that our proposed methods can have a performance that is better than that of many state-of-the-art methods including linear and nonlinear methods.

Joint sparse graph and flexible embedding for graph-based semi-supervised learning

This letter introduces a framework for graph-based semi-supervised learning by estimating a flexible non-linear projection and its linear regression model. Unlike existing works, the proposed framework jointly estimates the graph structure, the non-linear projection, and the linear regression model. By adopting this joint estimation an overall optimality can be reached. A series of experiments are conducted on five image datasets in order to compare the proposed method with some state-of-art semi-supervised methods. This evaluation demonstrates the effectiveness of the proposed embedding method. These experiments show the superiority of the proposed framework over the joint estimation of the graph and soft labels.

Learning a discriminant graph-based embedding with feature selection for image categorization

Graph-based embedding methods are very useful for reducing the dimension of high-dimensional data and for extracting their relevant features. In this paper, we introduce a novel nonlinear method called Flexible Discriminant graph-based Embedding with feature selection (FDEFS). The proposed algorithm aims to classify image sample data in supervised learning and semi-supervised learning settings. Specifically, our method incorporates the Manifold Smoothness, Margin Discriminant Embedding and the Sparse Regression for feature selection. The weights add ℓ2,1-norm regularization for local linear approximation. The sparse regression implicitly performs feature selection on the original features of data matrix and of the linear transform. We also provide an effective solution method to optimize the objective function. We apply the algorithm on six public image datasets including scene, face and object datasets. These experiments demonstrate the effectiveness of the proposed embedding method. They also show that proposed the method compares favorably with many competing embedding methods.

Fiber-Wireless Network Virtual Resource Embedding Method Based on Load Balancing and Priority

Network virtualization is becoming one of the most promising ways to solve resource solidification problem of fiber-wireless access networks. To fully utilize substrate resources, a virtual resource embedding method including three sub-mechanisms is presented in this paper. The first one is the priority-based virtual network request (VNR) selecting mechanism, which sorts VNRs and preferentially selects higher priority VNRs for embedding. Then a dynamic embedding mechanism based on load balancing is presented to increase the acceptance rate of VNRs. Second, when embedding is saturated, an embedding update mechanism is proposed to deal with the arrival of new VNRs. It promotes the acceptance rate of high-priority VNRs and the revenue of the infrastructure provider (InP). Finally, a backup resource sharing mechanism is designed to improve reliability, which enables backup resources to be shared and responds to the failures with minimal resources. The simulation results show that the proposed embedding method can balance the resource consumption better than the Greedy Node Embedding and Shortest-path Link Embedding Algorithm, as well as the Greedy Node Embedding and Maximum Residual Bandwidth Link Embedding Algorithm. Besides, by applying our proposed algorithm, higher VNR acceptance rate, InP revenue, and resource utilization are achieved.

Deep Dynamic Network Embedding for Link Prediction

Network embedding task aims at learning low-dimension latent representations of vertices while preserving the structure of a network simultaneously. Most existing network embedding methods mainly focus on static networks, which extract and condense the network information without temporal information. However, in the real world, networks keep evolving, where the linkage states between the same vertex pairs at consequential timestamps have very close correlations. In this paper, we propose to study the network embedding problem and focus on modeling the linkage evolution in the dynamic network setting. To address this problem, we propose a deep dynamic network embedding method. More specifically, the method utilizes the historical information obtained from the network snapshots at past timestamps to learn latent representations of the future network. In the proposed embedding method, the objective function is carefully designed to incorporate both the network internal and network dynamic transition structures. Extensive empirical experiments prove the effectiveness of the proposed model on various categories of real-world networks, including a human contact network, a bibliographic network, and e-mail networks. Furthermore, the experimental results also demonstrate the significant advantages of the method compared with both the state-of-the-art embedding techniques and several existing baseline methods.

An Information Distillation Framework for Extractive Summarization

In the context of natural language processing, representation learning has emerged as a newly active research subject because of its excellent performance in many applications. Learning representations of words is a pioneering study in this school of research. However, paragraph (or sentence and document) embedding learning is more suitable/reasonable for some realistic tasks such as document summarization. Nevertheless, classic paragraph embedding methods infer the representation of a given paragraph by considering all of the words occurring in the paragraph. Consequently, those stop or function words that occur frequently may mislead the embedding learning process to produce a misty paragraph representation. Motivated by these observations, our major contributions in this paper are threefold. First, we propose a novel unsupervised paragraph embedding method, named the essence vector (EV) model, which aims at not only distilling the most representative information from a paragraph but also excluding the general background information to produce a more informative low-dimensional vector representation for the paragraph of interest. Second, in view of the increasing importance of spoken content processing, an extension of the EV model, named the denoising essence vector (D-EV) model, is proposed. The D-EV model not only inherits the advantages of the EV model but also can infer a more robust representation for a given spoken paragraph against imperfect speech recognition. Third, a new summarization framework, which can take both relevance and redundancy information into account simultaneously, is also introduced. We evaluate the proposed embedding methods (i.e., EV and D-EV) and the summarization framework on two benchmark summarization corpora. The experimental results demonstrate the effectiveness and applicability of the proposed framework in relation to several well-practiced and state-of-the-art summarization methods.