Dynamic Embedding Research Articles

Data Protection in Palmprint Recognition via Dynamic Random Invisible Watermark Embedding

Palmprint recognition is one of the most popular biometric technologies. Recent researches mainly focus on the recognition performance, while pay less attention to the data protection issues. In this paper, we propose an active biometric data protection model for securing palmprint images in transmission or storage scenarios, called Dynamic Random Invisible Watermark Embedding (DRIWE) model. The DRIWE model implicitly embeds a watermark in each original palmprint ROI image, and then separates the embedded watermark from the watermarked image before identification. If the separated watermark is consistent with the original watermark, it indicates that the image is trustworthy and can be used in the subsequent recognition process. Otherwise, it proves that the image has been illegally tampered with. Furthermore, a two-dimensional image information entropy loss is proposed to enhance the generalization of the model to different watermarks. It ensures that the model can always assign enough information to the host image (i.e., original palmprint image) when different watermarks are applied. Thus, it enables the separator to extract the complete watermark from the watermarked image. This greatly enhances the dynamics and randomness of the watermark embedding process and further improves the ability to secure the data. Adequate experiments are conducted on two benchmark palmprint databases. The results show that the proposed DRIWE model has satisfactory attack resistance and strong generalization ability: even if only one watermark is used in training stage, it can be generalized to a dozen of other new watermark images in the testing stage. In addition, the optimal accuracy of the watermarked data is only reduced by 0.07% compared with the original data.

ComGCN: Community-Driven Graph Convolutional Network for Link Prediction in Dynamic Networks

Recent advances in deep learning have tremendously leveraged the performance of network representation learning (NRL). Multiple deep learning-based NRL models have been proposed recently to effectively handling primitive tasks of information network analysis and mining (INAM) domain, including link prediction (LP). LP is considered as an important one due to its multiple applications in many disciplines. In the recent few years, LP in dynamic networks has attracted a lot of attention from researchers to propose novel algorithms for better capturing both rich structural and evolutional information of complex information networks (INs). However, recent models are mainly concentrated on preserving the sequential representations of a given network over time. They have largely ignored other important structural features, such as: intracommunity which contributes to the creation of links between network nodes. In this article, we propose a novel community-driven dynamic NRL technique upon the RNN+GCN framework, called: ComGCN. Specifically, the ComGCN model is a combination of microscopic (node embedding-based) and mesoscopic (intracommunity-based) dynamic network embedding approach which enable effectively handling the LP problem in context of dynamism. Extensive experiments on real-world dynamic networks demonstrated the effectiveness of the proposed model compared with recent state-of-the-art baselines.

Context-Aware Sentiment Analysis using Tweet Expansion Method

The large source of information space produced by the plethora of social media platforms in general and microblogging in particular has spawned a slew of new applications and prompted the rise and expansion of sentiment analysis research. We propose a sentiment analysis technique that identifies the main parts to describe tweet intent and also enriches them with relevant words, phrases, or even inferred variables. We followed a state-of-the-art hybrid deep learning model to combine Convolutional Neural Network (CNN) and the Long Short-Term Memory network (LSTM) to classify tweet data based on their polarity. To preserve the latent relationships between tweet terms and their expanded representation, sentence encoding and contextualized word embeddings are utilized. To investigate the performance of tweet embeddings on the sentiment analysis task, we tested several context-free models (Word2Vec, Sentence2Vec, Glove, and FastText), a dynamic embedding model (BERT), deep contextualized word representations (ELMo), and an entity-based model (Wikipedia). The proposed method and results prove that text enrichment improves the accuracy of sentiment polarity classification with a notable percentage.

Uncertainty and Strategic Entrepreneurial Action- Scaling Patterns in Technology-Based New Ventures

By combining strategic and entrepreneurial action theories, we consider scaling as strategic entrepreneurial action and aim to explain how TMT's choice to deal with uncertainty and its two relevant knowledge problems enables them to design scaling patterns. To do so, we adopt an information processing perspective to investigate the uncertainty-relevant knowledge problems posed to TMT of ten technology-based new ventures throughout their successful and unsuccessful scaling efforts in a business setting wherein uncertainty is at its most extreme and imposed on firms by multiple sources. Our findings demonstrate that TMT's approach to dealing with uncertainty and relevant knowledge problems leads the firm to follow two different scaling paths: spiral and dynamic embedding. Spiral refers to a scaling trajectory in which the firm avoids dealing with high environmental uncertainties and chooses to switch between potential markets by adapting to existing environmental structures. In contrast, dynamic embedding refers to a scaling path wherein the firm tends to deal with high environmental uncertainty and chooses to engage in shaping new environmental arrangements to embed its technologies. We also show that spiral scaling is made through endogenously dynamic capabilities, and dynamic embedding scaling is produced through exogenously dynamic capabilities. We contribute to the literature by providing a deeper understanding of when and how strategic actors scale up the firm under uncertainty, as well as the origin of dynamic capabilities required for scaling in technology-based new ventures.

Wireless Virtual Network Embedding Algorithm Based on Deep Reinforcement Learning

Wireless network virtualization is widely used to solve the ossification problem of networks, such as 5G and the Internet of Things. The most crucial method of wireless network virtualization is virtual network embedding, which allows virtual networks to share the substrate network resources. However, in wireless networks, link interference is an inherent problem while mapping virtual networks because of the characteristics of wireless channels. To distribute resources efficiently and address the problem of interference, a dynamic embedding algorithm with deep reinforcement learning is proposed. During the training stage, we take resource use and interference from substrate networks as observations to train the agent, and then the agent generates a resource allocation strategy. Aiming at realizing load balance, we reshape the reward function considering the execution ratio and residual ratio of substrate network resources as well as the cost consumed by current virtual network request. Numerical tests show that our embedding approach increases the acceptance ratio and maintains better robustness. Moreover, the results also illustrate that our algorithm maintains a high acceptance ratio while producing less interference and lower cost.

Deep multi-scale Gaussian residual networks for contextual-aware translation initiation site recognition

The dysregulation of the translation initiation causes some cancers and metabolic disorders. However, the experimental verification of translation initiation sites (TIS) is expensive and small-scale, and the co-occurrence interaction relationship from genomic sequences is essential for knowledge discovery of TIS. In this work, a deep Gaussian residual neural computational model (GNet) is proposed to learn dynamic embeddings for parameter learning of discriminative features via context-aware modeling, and accurately identify TIS via co-occurrence embedding. GNet includes multi-scale Gaussian gated convolutional networks and bidirectional gated recurrent units. Particularly, a Gaussian gated linear unit is devised to extract local co-occurrence embedding vectors of genomic sequences, and the unit can reduce vanishing gradient problems and enable the recognition model to obtain powerful learning capabilities. Moreover, a stochastic linear skip gated connection is designed to boost the information exchange and extract complex contextual features between low and high layers, and vanishing gradients can be largely alleviated during training. Then, the gated recurrent unit is used to extract global long-term dependency features via identity connections. Consequently, to obtain global embedding information of sequences, a concatenation operation is used to fuse local and long discriminative features. Experiments demonstrate that GNet is an efficient and effective TIS recognition model and achieves remarkable results over state-of-the-art methods.

Timing and networks: Embedding patterns of Peruvian migrants in Switzerland

In this article, I analyse the tensions between transnationalism and integration as dynamic embedding practices. To elucidate patterns of embedding, I compare the life stories and networking accounts of an heterogeneous group of Peruvians with different social mobility outcomes. Despite length of stay, embedding practices show scarce and inconstant ties with host-country nationals, whereas Peruvians and other migrants represent persistent and close ties with members of this group. While the quantity and quality of ties with members of the host society have been predominantly analysed to asses degrees of integration, most research has only focused on co-nationals in home and host countries to enquiry about transnationalism. Moreover, the ties with migrants of other nationalities and co-nationals at alternative destinations are subsumed under a broad “transnational” category. Besides the dual compatriot networks, the analysis of Peruvian migrants’ embedding practices reveals multifocal transnationalism based on ties with other Peruvians across several countries that in turn might enhance forms of integration in Switzerland. The result of uppward mobility at destination depending on close and stable ties with geographically scattered co-nationals contributes to the debates about social capital in migration studies.

Dynamic Virtual Network Embedding Algorithm Based on Graph Convolution Neural Network and Reinforcement Learning

Network virtualization (NV) is a technology with broad application prospects. Virtual network embedding (VNE) is the core orientation of VN, which aims to provide more flexible underlying physical resource allocation for user function requests. The classical VNE problem is usually solved by the heuristic method, but this method often limits the flexibility of the algorithm and ignores the time limit. In addition, the partition autonomy of physical domain and the dynamic characteristics of virtual network request (VNR) also increase the difficulty of VNE. This article proposed a new type of VNE algorithm, which applied reinforcement learning (RL) and graph neural network (GNN) theory to the algorithm, especially the combination of graph convolutional neural network (GCNN) and RL algorithm. Based on a self-defined fitness matrix and fitness value, we set up the objective function of the algorithm implementation, realized an efficient dynamic VNE algorithm, and effectively reduced the degree of resource fragmentation. Finally, we used comparison algorithms to evaluate the proposed method. Simulation experiments verified that the dynamic VNE algorithm based on RL and GCNN has good basic VNE characteristics. By changing the resource attributes of physical network and virtual network, it can be proved that the algorithm has good flexibility.

Dynamic embedding behavior of thin-wall expansion tube loaded by explosive shock wave

Expansion tubes have been considered to be promising impact-resistant structures due to their lightweight and high specific energy absorption capacity. The mechanical behaviors of expansion tubes subjected to a quasi-static loading or a dynamic loading have been widely studied; however, no previous study was reported regarding a blast loading. In this paper, the dynamic embedding responses of expansion tubes considering the effects of shock wave properties, structural parameters, and scaled distance were first investigated through orchestrated explosion experiments. The explosion results show that the embedding displacement is affected by the overpressure and impulse simultaneously. A finite element method was then proposed and verified by comparing the numerical solutions with the experimental results, based on which the effects of loading condition, scaled distance, and semi-angle of expansion tube were revealed. Finally, metallic foam impact tests were conducted to reveal the effect of the shell on the embedding displacement. The numerical solutions indicate that the expansion force increases by 22.9% when the quasi-static loading is replaced with a blast loading. The expansion tube can efficiently reduce the maximum shock stress acting on the baseboard and prolong the buffering process in the time domain. Due to the effect of the blast wave with a quasi-triangular stress-time curve, there exists a negative correlation between the effective impulse and the stress threshold of the expansion tube. In addition, the shell shows an insignificant effect on the embedding displacement of expansion tube. The results in this paper provide useful guidance for the application of expansion tubes in practice for improving the blast-resistant of a structure.

Efficiently embedding dynamic knowledge graphs

Knowledge graph (KG) embedding encodes the entities and relations from a KG into low-dimensional vector spaces to support various applications such as KG completion, question answering, and recommender systems. In real world, knowledge graphs (KGs) are dynamic and evolve over time with addition or deletion of triples. However, most existing models focus on embedding static KGs while neglecting dynamics. To adapt to the changes in a KG, these models need to be retrained on the whole KG with a high time cost. In this paper, to tackle the aforementioned problem, we propose a new context-aware Dynamic Knowledge Graph Embedding (DKGE) method which supports the embedding learning in an online fashion. DKGE introduces two different representations (i.e., knowledge embedding and contextual element embedding) for each entity and each relation, in the joint modeling of entities and relations as well as their contexts, by employing two attentive graph convolutional networks, a gate strategy, and translation operations. This effectively helps limit the impacts of a KG update in certain regions, not in the entire graph, so that DKGE can rapidly acquire the updated KG embedding by a proposed online learning algorithm. Furthermore, DKGE can also learn KG embedding from scratch. Experiments on the tasks of link prediction and question answering in a dynamic environment demonstrate the effectiveness and efficiency of DKGE.

LP-ROBIN: Link prediction in dynamic networks exploiting incremental node embedding

In many real-world domains, data can naturally be represented as networks. This is the case of social networks, bibliographic networks, sensor networks and biological networks. Some dynamism often characterizes these networks as their structure (i.e., nodes and edges) continually evolves. Considering this dynamism is essential for analyzing these networks accurately. In this work, we propose LP-ROBIN, a novel method that exploits incremental embedding to capture the dynamism of the network structure and predicts new links, which can be used to suggest friends in social networks, or interactions in biological networks, just to cite some. Differently from the state-of-the-art methods, LP-ROBIN can work with mutable sets of nodes, i.e., new nodes may appear over time without being known in advance. After the arrival of new data, LP-ROBIN does not need to retrain the model from scratch, but learns the embeddings of the new nodes and links, and updates the latent representations of old ones, to reflect changes in the network structure for link prediction purposes. The experimental results show that LP-ROBIN achieves better performances, in terms of AUC and F1-score, and competitive running times with respect to baselines, static node embedding approaches and state-of-the-art methods which use dynamic node embedding.

Long short-term enhanced memory for sequential recommendation

Sequential recommendation is a stream of studies on recommender systems, which focuses on predicting the next item a user interacts with by modeling the dynamic sequence of user-item interactions. Since being born to explore the dynamic tendency of variable-length temporal sequence, Recurrent Neural Networks (RNNs) have been paid much attention in this area. However, the inherent defects caused by the network structure of RNNs have limited their applications in sequential recommendation, which are mainly shown on two factors: RNNs tend to make point-wise predictions and ignore the collective dependencies because the temporal relationships between items change monotonically; RNNs are likely to forget the essential information during processing long sequences. To solve these problems, researchers have done much work to enhance the memory mechanism of RNNs. However, although previous RNN-based methods have achieved promising performance by taking advantage of external knowledge with other advanced techniques, the improvement of the intrinsic property of existing RNNs has not been explored, which is still challenging. Therefore, in this work, we propose a novel architecture based on Long Short-Term Memories (LSTMs), a broadly-used variant of RNNs, specific for sequential recommendation, called Long Short-Term enhanced Memory (LSTeM), which boosts the memory mechanism of original LSTMs in two ways. Firstly, we design a new structure of gates in LSTMs by introducing a “Q-K-V” triplet, a mechanism to accurately and properly model the correlation between the current item and the user’s historical behaviors at each time step. Secondly, we propose a “recover gate” to remedy the inadequacy of memory caused by the forgetting mechanism, which works with a dynamic global memory embedding. Extensive experiments have demonstrated that LSTeM achieves comparable performance to the state-of-the-art methods on the challenging datasets for sequential recommendation.

Dynamic intent-aware iterative denoising network for session-based recommendation

Session-based recommendation aims to predict items that a user will interact with based on historical behaviors in anonymous sessions. It has long faced two challenges: (1) the dynamic change of user intents which makes user preferences towards items change over time; (2) the uncertainty of user behaviors which adds noise to hinder precise preference learning. They jointly preclude recommender system from capturing real intents of users. Existing methods have not properly solved these problems since they either ignore many useful factors like the temporal information when building item embeddings, or do not explicitly filter out noisy clicks in sessions. To tackle above issues, we propose a novel Dynamic Intent-aware Iterative Denoising Network (DIDN) for session-based recommendation. Specifically, to model the dynamic intents of users, we present a dynamic intent-aware module that incorporates item-aware, user-aware and temporal-aware information to learn dynamic item embeddings. A novel iterative denoising module is then devised to explicitly filter out noisy clicks within a session. In addition, we mine collaborative information to further enrich the session semantics. Extensive experimental results on three real-world datasets demonstrate the effectiveness of the proposed DIDN. Specifically, DIDN obtains improvements over the best baselines by 1.66%, 1.75%, and 7.76% in terms of P@20 and 1.70%, 2.20%, and 10.48% in terms of MRR@20 on all datasets.

A Bayesian matrix factorization model for dynamic user embedding in recommender system

A Bayesian matrix factorization model for dynamic user embedding in recommender system

Online author name disambiguation in evolving digital library

Name ambiguity is a prevalent problem in digital library domain where mapping of bibliographic records to authors is a major issue. The unprecedented growth of the bibliographic records and absence of unique identifiers are further exacerbating the problem. Specifically, name ambiguity affects various bibliometric analysis tasks that include record management as well as scientific assessment of the authors thereby necessitating the name disambiguation. The name disambiguation task is to assign the records, possibly with the ambiguous authorship, to corresponding authors. While existing techniques are good at extracting abstract features from set of records with a common author name that can be subsequently used for clustering the records based on unique author identities, however, such techniques usually perform poorly in disambiguating isolated individual record entries that arrive continuously. Disambiguation of only newly arrived records, rather than the whole records of the digital library is challenging, however, computationally rewarding and thus, not only preferable but becoming the necessity due to tremendous growth in the number of bibliographic records with the time, which is likely to continue. In this regard, we propose an online author name disambiguation approach for evolving digital library. Our approach involves representation learning of records in an online manner in evolving (academic networks) digital library using dynamic graph embedding and clustering of latent representation of records. We show the use of our online name disambiguation method in batch setting (for static or initial records of digital library) and incremental setting (for new records of digital library). Significant improvement, over existing state-of-the-art methods in terms of various evaluation metrics, has been observed which indicates the effectiveness of the proposed approach.

TGL

Many real world graphs contain time domain information. Temporal Graph Neural Networks capture temporal information as well as structural and contextual information in the generated dynamic node embeddings. Researchers have shown that these embeddings achieve state-of-the-art performance in many different tasks. In this work, we propose TGL, a unified framework for large-scale offline Temporal Graph Neural Network training where users can compose various Temporal Graph Neural Networks with simple configuration files. TGL comprises five main components, a temporal sampler, a mailbox, a node memory module, a memory updater, and a message passing engine. We design a Temporal-CSR data structure and a parallel sampler to efficiently sample temporal neighbors to form training mini-batches. We propose a novel random chunk scheduling technique that mitigates the problem of obsolete node memory when training with a large batch size. To address the limitations of current TGNNs only being evaluated on small-scale datasets, we introduce two large-scale real-world datasets with 0.2 and 1.3 billion temporal edges. We evaluate the performance of TGL on four small-scale datasets with a single GPU and the two large datasets with multiple GPUs for both link prediction and node classification tasks. We compare TGL with the open-sourced code of five methods and show that TGL achieves similar or better accuracy with an average of 13X speedup. Our temporal parallel sampler achieves an average of 173X speedup on a multi-core CPU compared with the baselines. On a 4-GPU machine, TGL can train one epoch of more than one billion temporal edges within 1-10 hours. To the best of our knowledge, this is the first work that proposes a general framework for large-scale Temporal Graph Neural Networks training on multiple GPUs.

Scalable Representation Learning for Dynamic Heterogeneous Information Networks via Metagraphs

Content representation is a fundamental task in information retrieval. Representation learning is aimed at capturing features of an information object in a low-dimensional space. Most research on representation learning for heterogeneous information networks (HINs) focuses on static HINs. In practice, however, networks are dynamic and subject to constant change. In this article, we propose a novel and scalable representation learning model, M-DHIN , to explore the evolution of a dynamic HIN. We regard a dynamic HIN as a series of snapshots with different time stamps. We first use a static embedding method to learn the initial embeddings of a dynamic HIN at the first time stamp. We describe the features of the initial HIN via metagraphs, which retains more structural and semantic information than traditional path-oriented static models. We also adopt a complex embedding scheme to better distinguish between symmetric and asymmetric metagraphs. Unlike traditional models that process an entire network at each time stamp, we build a so-called change dataset that only includes nodes involved in a triadic closure or opening process, as well as newly added or deleted nodes. Then, we utilize the above metagraph-based mechanism to train on the change dataset. As a result of this setup, M-DHIN is scalable to large dynamic HINs since it only needs to model the entire HIN once while only the changed parts need to be processed over time. Existing dynamic embedding models only express the existing snapshots and cannot predict the future network structure. To equip M-DHIN with this ability, we introduce an LSTM-based deep autoencoder model that processes the evolution of the graph via an LSTM encoder and outputs the predicted graph. Finally, we evaluate the proposed model, M-DHIN , on real-life datasets and demonstrate that it significantly and consistently outperforms state-of-the-art models.

Dynamic Embedding Projection-Gated Convolutional Neural Networks for Text Classification

Text classification is a fundamental and important area of natural language processing for assigning a text into at least one predefined tag or category according to its content. Most of the advanced systems are either too simple to get high accuracy or centered on using complex structures to capture the genuinely required category information, which requires long time to converge during their training stage. In order to address such challenging issues, we propose a dynamic embedding projection-gated convolutional neural network (DEP-CNN) for multi-class and multi-label text classification. Its dynamic embedding projection gate (DEPG) transforms and carries word information by using gating units and shortcut connections to control how much context information is incorporated into each specific position of a word-embedding matrix in a text. To our knowledge, we are the first to apply DEPG over a word-embedding matrix. The experimental results on four known benchmark datasets display that DEP-CNN outperforms its recent peers.

Predicting Dynamic User–Item Interaction with Meta-Path Guided Recursive RNN

Accurately predicting user–item interactions is critically important in many real applications, including recommender systems and user behavior analysis in social networks. One major drawback of existing studies is that they generally directly analyze the sparse user–item interaction data without considering their semantic correlations and the structural information hidden in the data. Another limitation is that existing approaches usually embed the users and items into the different embedding spaces in a static way, but ignore the dynamic characteristics of both users and items. In this paper, we propose to learn the dynamic embedding vector trajectories rather than the static embedding vectors for users and items simultaneously. A Metapath-guided Recursive RNN based Shift embedding method named MRRNN-S is proposed to learn the continuously evolving embeddings of users and items for more accurately predicting their future interactions. The proposed MRRNN-S is extended from our previous model RRNN-S which was proposed in the earlier work. Comparedwith RRNN-S, we add the word2vec module and the skip-gram-based meta-path module to better capture the rich auxiliary information from the user–item interaction data. Specifically, we first regard the interaction data of each user with items as sentence data to model their semantic and sequential information and construct the user–item interaction graph. Then we sample the instances of meta-paths to capture the heterogeneity and structural information from the user–item interaction graph. A recursive RNN is proposed to iteratively and mutually learn the dynamic user and item embeddings in the same latent space based on their historical interactions. Next, a shift embedding module is proposed to predict the future user embeddings. To predict which item a user will interact with, we output the item embedding instead of the pairwise interaction probability between users and items, which is much more efficient. Through extensive experiments on three real-world datasets, we demonstrate that MRRNN-S achieves superior performance by extensive comparison with state-of-the-art baseline models.

Dynamic Heterogeneous User Generated Contents-Driven Relation Assessment via Graph Representation Learning.

Cross-domain decision-making systems are suffering a huge challenge with the rapidly emerging uneven quality of user-generated data, which poses a heavy responsibility to online platforms. Current content analysis methods primarily concentrate on non-textual contents, such as images and videos themselves, while ignoring the interrelationship between each user post’s contents. In this paper, we propose a novel framework named community-aware dynamic heterogeneous graph embedding (CDHNE) for relationship assessment, capable of mining heterogeneous information, latent community structure and dynamic characteristics from user-generated contents (UGC), which aims to solve complex non-euclidean structured problems. Specifically, we introduce the Markov-chain-based metapath to extract heterogeneous contents and semantics in UGC. A edge-centric attention mechanism is elaborated for localized feature aggregation. Thereafter, we obtain the node representations from micro perspective and apply it to the discovery of global structure by a clustering technique. In order to uncover the temporal evolutionary patterns, we devise an encoder–decoder structure, containing multiple recurrent memory units, which helps to capture the dynamics for relation assessment efficiently and effectively. Extensive experiments on four real-world datasets are conducted in this work, which demonstrate that CDHNE outperforms other baselines due to the comprehensive node representation, while also exhibiting the superiority of CDHNE in relation assessment. The proposed model is presented as a method of breaking down the barriers between traditional UGC analysis and their abstract network analysis.