Embedding Approach Research Articles

Automated Analysis of Changes in Privacy Policies: A Structured Self-Attentive Sentence Embedding Approach

The increasing societal concern for consumer information privacy has led to the enforcement of privacy regulations worldwide. In an effort to adhere to privacy regulations such as General Data Protection Regulation (GDPR), many companies’ privacy policies have become increasingly lengthy and complex. In this study, we adopted the computational design science paradigm to design a novel privacy policy evolution analytics framework to help identify how companies change and present their privacy policies based on privacy regulations. The framework includes a Self-Attentive Annotation System (SAAS) that automatically annotates paragraph-length segments in privacy policies to help stakeholders identify data practices of interest for further investigation. We rigorously evaluated SAAS against state-of-the-art Machine Learning (ML) and Deep Learning (DL)-based methods on a well-established privacy policy dataset, OPP-115. SAAS outperformed conventional ML and DL models in terms of F1-score by statistically significant margins. We demonstrate the proposed framework’s practical utility with an in-depth case study of GDPR’s impact on Amazon’s privacy policies. The case study results indicate that Amazon’s post-GDPR privacy policy potentially violates a fundamental principle of GDPR by causing consumers to exert more effort to find information about first-party data collection. Given the increasing importance of consumer information privacy, the proposed framework has important implications for regulators and companies. We discuss several design principles followed by the SAAS that can help guide future design science-based e-commerce, health, and privacy research.

Wasserstein adversarial learning based temporal knowledge graph embedding

Research on knowledge graph embedding (KGE) has emerged as an active field in which most existing KGE approaches mainly focus on static structural data and ignore the influence of temporal variation involved in time-aware triples. In order to deal with this issue, several temporal knowledge graph embedding (TKGE) approaches have been proposed to integrate temporal and structural information. However, these methods only employ a uniformly random sampling to construct negative facts. As a consequence, the corrupted samples are often too simplistic for training an effective model. In this paper, we propose a new temporal knowledge graph embedding framework by introducing adversarial learning to further refine the performance of traditional TKGE models. In our framework, a generator is utilized to construct high-quality plausible quadruples and a discriminator learns to obtain the embeddings of entities and relations based on both positive and negative samples. Meanwhile, we also apply a Gumbel-Softmax relaxation and the Wasserstein distance to prevent vanishing gradient problems on discrete data; an inherent flaw in traditional generative adversarial networks. Through comprehensive experimentation on temporal datasets, the results indicate that our proposed framework can attain significant improvements based on benchmark models and also demonstrate the effectiveness and applicability of our framework.

RoCS: Knowledge Graph Embedding Based on Joint Cosine Similarity

Knowledge graphs usually have many missing links, and predicting the relationships between entities has become a hot research topic in recent years. Knowledge graph embedding research maps entities and relations to a low-dimensional continuous space representation to predict links between entities. The present research shows that the key to the knowledge graph embedding approach is the design of scoring functions. According to the scoring function, knowledge graph embedding methods can be classified into dot product models and distance models. We find that the triple scores obtained using the dot product model or the distance model were unbounded, which leads to large variance. In this paper, we propose RotatE Cosine Similarity (RoCS), a method to compute the joint cosine similarity of complex vectors as a scoring function to make the triple scores bounded. Our approach combines the rotational properties of the complex vector embedding model RotatE to model complex relational patterns. The experimental results demonstrate that the newly introduced RoCS yields substantial enhancements compared to RotatE across various knowledge graph benchmarks, improving up to 4.0% in hits at 1 (Hits@1) on WN18RR and improving up to 3.3% in Hits@1 on FB15K-237. Meanwhile, our method achieves some new state-of-the-art (SOTA), including Hits@3 of 95.6%, Hits@10 of 96.4% on WN18, and mean reciprocal rank (MRR) of 48.9% and Hits@1 of 44.5% on WN18RR.

Performance of point charge embedding schemes for excited states in molecular organic crystals.

Modeling excited state processes in molecular crystals is relevant for several applications. A popular approach for studying excited state molecular crystals is to use cluster models embedded in point charges. In this paper, we compare the performance of several embedding models in predicting excited states and S1-S0 optical gaps for a set of crystals from the X23 molecular crystal database. The performance of atomic charges based on ground or excited states was examined for cluster models, Ewald embedding, and self-consistent approaches. We investigated the impact of various factors, such as the level of theory, basis sets, embedding models, and the level of localization of the excitation. We consider different levels of theory, including time-dependent density functional theory and Tamm-Dancoff approximation (TDA) (DFT functionals: ωB97X-D and PBE0), CC2, complete active space self-consistent field, and CASPT2. We also explore the impact of selection of the QM region, charge leakage, and level of theory for the description of different kinds of excited states. We implemented three schemes based on distance thresholds to overcome overpolarization and charge leakage in molecular crystals. Our findings are compared against experimental data, G0W0-BSE, periodic TDA, and optimally tuned screened range-separated functionals.

Nano-engineered catalysts for high-performance oxygen reduction reaction

The efficient energy conversion of fuel cells is greatly constrained by the slow oxygen reduction reaction (ORR) kinetics, which necessitates the use of highly active metal catalysts such as platinum (Pt). The critical challenge limiting large-scale usage of Pt is the capital cost that can be addressed through a prototypical approach by embedding metal nanoparticles (NPs), e.g., Pt NPs, in the conductive framework. However, previously reported embedding approaches are sophisticated and suffer from limited yields, leading to higher chemical process costs and remaining distant from commercial viability. Here, we report a facile, cost-effective and time-efficient structural tuning approach to synthesizing ultrafine Pt NPs impregnated within a conductive and highly porous carbon framework via a microwave-assisted polyol reduction method. Pt NPs with a uniform size of ∼2.27 nm can be successfully integrated within the pores of the carbon framework, enabling homogeneous dispersion. Benefiting from these highly dispersed and ultrafine Pt NPs, the electrochemical surface area (ECSA) is improved to 142.98 m²/gPt, 2.25 times higher than that of the commercial counterpart (63.52 m²/gPt). Furthermore, our structurally optimized catalyst composite features a remarkably catalytic activity with a high half-wave potential (E1/2) of 0.895 V and an improved mass activity (MA) of 0.2289 A/mgPt, 2.39-fold improvement compared to the commercial counterpart. In addition, orthogonal experiments were designed to identify the key process parameters for fabricating Pt/C catalysts, offering insights for scaled-up and industrial production.

Solving inverse scattering problems via reduced-order model embedding procedures

We present a reduced-order model (ROM) methodology for inverse scattering problems in which the ROMs are data-driven, i.e. they are constructed directly from data gathered by sensors. Moreover, the entries of the ROM contain localised information about the coefficients of the wave equation. We solve the inverse problem by embedding the ROM in physical space. Such an approach is also followed in the theory of ‘optimal grids,’ where the ROMs are interpreted as two-point finite-difference discretisations of an underlying set of equations of a first-order continuous system on this special grid. Here, we extend this line of work to wave equations and introduce a new embedding technique, which we call Krein embedding, since it is inspired by Krein’s seminal work on vibrations of a string. In this embedding approach, an adaptive grid and a set of medium parameters can be directly extracted from a ROM and we show that several limitations of optimal grid embeddings can be avoided. Furthermore, we show how Krein embedding is connected to classical optimal grid embedding and that convergence results for optimal grids can be extended to this novel embedding approach. Finally, we also briefly discuss Krein embedding for open domains, that is, semi-infinite domains that extend to infinity in one direction.

Enhancing knowledge discovery from unstructured data using a deep learning approach to support subsurface modeling predictions.

Subsurface interpretations and models rely on knowledge from subject matter experts who utilize unstructured information from images, maps, cross sections, and other products to provide context to measured data (e. g., cores, well logs, seismic surveys). To enhance such knowledge discovery, we advanced the National Energy Technology Laboratory's (NETL) Subsurface Trend Analysis (STA) workflow with an artificial intelligence (AI) deep learning approach for image embedding. NETL's STA method offers a validated science-based approach of combining geologic systems knowledge, statistical modeling, and datasets to improve predictions of subsurface properties. The STA image embedding tool quickly extracts images from unstructured knowledge products like publications, maps, websites, and presentations; categorically labels the images; and creates a repository for geologic domain postulation. Via a case study on geographic and subsurface literature of the Gulf of Mexico (GOM), results show the STA image embedding tool extracts images and correctly labels them with ~90 to ~95% accuracy.

Multimer Embedding Approach for Molecular Crystals up to Harmonic Vibrational Properties.

Accurate calculations of molecular crystals are crucial for drug design and crystal engineering. However, periodic high-level density functional calculations using hybrid functionals are often prohibitively expensive for the relevant systems. These expensive periodic calculations can be circumvented by the usage of embedding methods in which, for instance, the periodic calculation is only performed at a lower-cost level and then monomer energies and dimer interactions are replaced by those of the higher-level method. Herein, we extend such a multimer embedding approach to enable energy corrections for trimer interactions and the calculation of harmonic vibrational properties up to the dimer level. We evaluate this approach for the X23 benchmark set of molecular crystals by approximating a periodic hybrid density functional (PBE0+MBD) by embedding multimers into less expensive calculations using a generalized-gradient approximation functional (PBE+MBD). We show that trimer interactions are crucial for accurately approximating lattice energies within 1 kJ/mol and might also be needed for further improvement of lattice constants and hence cell volumes. Finally, the vibrational properties are already very well captured at the monomer and dimer level, making it possible to approximate vibrational free energies at room temperature within 1 kJ/mol.

Attention-based multi attribute matrix factorization for enhanced recommendation performance

In E-commerce platforms, auxiliary information containing several attributes (e.g., price, quality, and brand) can improve recommendation performance. However, previous studies used a simple combined embedding approach that did not consider the importance of each attribute embedded in the auxiliary information or only used some attributes of the auxiliary information. However, user purchasing behavior can vary significantly depending on the attributes. Thus, we propose multi attribute-based matrix factorization (MAMF), which considers the importance of each attribute embedded in various auxiliary information. MAMF obtains more representative and specific attention features of the user and item using a self-attention mechanism. By acquiring attentive representation, MAMF learns a high-level interaction precisely between users and items. To evaluate the performance of the proposed MAMF, we conducted extensive experiments using three real-world datasets from amazon.com. The experimental results show that MAMF exhibits excellent recommendation performance compared with various baseline models.

Innovative image interpolation based reversible data hiding for secure communication

In an increasingly digitized world, secure communication plays a pivotal role in safeguarding sensitive information and ensuring the confidentiality of data transmission. Conventional encryption techniques are robust but often result in a loss of data during transmission, making it challenging to achieve both security and image quality simultaneously. To address this issue, an innovative technique for interpolation-based reversible data hiding (RDH) within images is presented in this study. The interpolation scheme considers the Min function of the neighboring pixels and generates the cover image. After that, a data hiding method is applied to the cover image by considering the intensity range of the pixels using the least-significant bit (LSB) substitution method. The suggested embedding approach first encrypts the secret message, and then it encodes the encrypted secret message into interpolated pixels based on pixel intensity range groups. This technique enables the image to be consistently recreated once the data has been extracted. The percentage increment in the embedding capacity and peak signal-to-noise ratio (PSNR) is 40.00 and 8.64% for the proposed method as compared to the existing method. The experimental results indicate that the proposed technique enhanced the embedding capacity for all test pictures and produced higher PSNR values.

Capturing Fine-Grained Regional Differences in Language Use through Voting Precinct Embeddings

Abstract Linguistic variation across a region of interest can be captured by partitioning the region into areas and using social media data to train embeddings that represent language use in those areas. Recent work has focused on larger areas, such as cities or counties, to ensure that enough social media data is available in each area, but larger areas have a limited ability to find fine-grained distinctions, such as intracity differences in language use. We demonstrate that it is possible to embed smaller areas, which can provide higher resolution analyses of language variation. We embed voting precincts, which are tiny, evenly sized political divisions for the administration of elections. The issue with modeling language use in small areas is that the data becomes incredibly sparse, with many areas having scant social media data. We propose a novel embedding approach that alternates training with smoothing, which mitigates these sparsity issues. We focus on linguistic variation across Texas as it is relatively understudied. We develop two novel quantitative evaluations that measure how well the embeddings can be used to capture linguistic variation. The first evaluation measures how well a model can map a dialect given terms specific to that dialect. The second evaluation measures how well a model can map preference of lexical variants. These evaluations show how embedding models could be used directly by sociolinguists and measure how much sociolinguistic information is contained within the embeddings. We complement this second evaluation with a methodology for using embeddings as a kind of genetic code where we identify “genes” that correspond to a sociological variable and connect those “genes” to a linguistic phenomenon thereby connecting sociological phenomena to linguistic ones. Finally, we explore approaches for inferring isoglosses using embeddings.

Detecting Wilson's disease from unstructured connected speech: An embedding-based approach augmented by attention and bi-directional dependency

Wilson's disease (WD) is a neurodegenerative genetic disorder in which dysarthria is the initial neurological symptom. Automated WD diagnosis from speech is thus a promising and clinically valuable approach. The present study investigates the feasibility of WD detection from unstructured connected speech (UCS) using the embedding-based approach augmented by the attention mechanism and bi-directional dependency. The classification experiment was conducted with a sample of 55 WD patients and 55 matched healthy individuals. We compare the proposed embedding approach with two models: the baseline method using the structured task and the model using conventional acoustic features. Results show that the embedding-based model achieves the best accuracy of 90.3 %, which is 4.2 % and 7 % better than the baseline and acoustic approaches, respectively. The bi-directional semantic dependency and attention mechanism can significantly improve detection performance. Moreover, we reveal that the duration of the UCS task affects the model performance, with favorable results achieved using approximately 30 s epochs. Our method provides new insights into the detection of dysarthria-related disorders.

GraphTar: applying word2vec and graph neural networks to miRNA target prediction

BackgroundMicroRNAs (miRNAs) are short, non-coding RNA molecules that regulate gene expression by binding to specific mRNAs, inhibiting their translation. They play a critical role in regulating various biological processes and are implicated in many diseases, including cardiovascular, oncological, gastrointestinal diseases, and viral infections. Computational methods that can identify potential miRNA–mRNA interactions from raw data use one-dimensional miRNA–mRNA duplex representations and simple sequence encoding techniques, which may limit their performance.ResultsWe have developed GraphTar, a new target prediction method that uses a novel graph-based representation to reflect the spatial structure of the miRNA–mRNA duplex. Unlike existing approaches, we use the word2vec method to accurately encode RNA sequence information. In conjunction with the novel encoding method, we use a graph neural network classifier that can accurately predict miRNA–mRNA interactions based on graph representation learning. As part of a comparative study, we evaluate three different node embedding approaches within the GraphTar framework and compare them with other state-of-the-art target prediction methods. The results show that the proposed method achieves similar performance to the best methods in the field and outperforms them on one of the datasets.ConclusionsIn this study, a novel miRNA target prediction approach called GraphTar is introduced. Results show that GraphTar is as effective as existing methods and even outperforms them in some cases, opening new avenues for further research. However, the expansion of available datasets is critical for advancing the field towards real-world applications.

An ensemble approach for imbalanced multiclass malware classification using 1D-CNN

Dependence on the internet and computer programs demonstrates the significance of computer programs in our day-to-day lives. Such demands motivate malware developers to create more malware, both in terms of quantity and variety. Researchers are constantly faced with hurdles while attempting to protect themselves from potential hazards and risks due to malware authors’ usage of code obfuscation techniques. Metamorphic and polymorphic variations are easily able to elude the widely utilized signature-based detection procedures. Researchers are more interested in deep learning approaches than machine learning techniques to analyze the behavior of such a vast number of virus variants. Researchers have been drawn to the categorization of malware within itself in addition to the classification of malware against benign programs to examine the behavioral differences between them. In order to investigate the relationship between the application programming interface (API) calls throughout API sequences and classify them, this work uses the one-dimensional convolutional neural network (1D-CNN) model to solve a multiclass classification problem. On API sequences, feature vectors for distinctive APIs are created using the Word2Vec word embedding approach and the skip-gram model. The one-vs.-rest approach is used to train 1D-CNN models to categorize malware, and all of them are then combined with a suggested ModifiedSoftVoting algorithm to improve classification. On the open benchmark dataset Mal-API-2019, the suggested ensembled 1D-CNN architecture captures improved evaluation scores with an accuracy of 0.90, a weighted average F1-score of 0.90, and an AUC score of more than 0.96 for all classes of malware.

A community detection model using node embedding approach and graph convolutional network with clustering technique

The communities in a network have distinct characteristics and interrelationships. Community discovery methods based on node embedding and deep learning have surpassed spectral clustering and statistical inference as the preferred methods for handling high-dimensional network data. Community detection in graph networks using graph neural networks (GNNs) is a rapidly expanding field of study that has attracted much attention recently. Inspired by the graph convolutional network (GCN) approach, promising results in evaluating graph structure data, a different method for discovering communities using node embedding using Node2Vec via graph convolutional networks (CD2NE-GCN) model has been proposed. It has been evaluated against the most recent standard methods of graph attention network (GAT), GCN-based approach for Unsupervised Community Detection (GUCD), and Graph Adversarial Networks (GAN). In this paper, a technique for community detection that combines the ideas of graph neural network design with node embedding via message forwarding has been presented. The experimentation has made use of five benchmark datasets: Karate, Cora, CiteSeer, Facebook, and DBLP. In the first set of trials, the node2vec uses the skip-gram with negative sampling (SGNS) approach via biased random walks of the second order to produce node embeddings and feed them to the graph convolutional network. The weighted average matrix and scaled normalized matrices of low-dimensional vertices have been calculated. In the next set of experiments, the average weighted matrix was sent to the 3-layer GCN with the activation function for generating feature vectors and applied clustering technique for predicting communities. The CD2NE-GCN approach outperforms previous literature by detecting graph network communities with accuracy scores of 98.45%, 99.04%, 96.39%, 98.79%, and 97.67% for Karate, Cora, CiteSeer, Facebook, and DBLP datasets, respectively. Thus, graph convolutional network models with a node embedding approach enhance the accuracy.

Automatic assignment of microgenres to movies using a word embedding-based approach

Streaming services are increasingly leveraging Artificial Intelligence (AI) technologies for improved content cataloging, user experiences in content discovery, and personalization. A significant challenge in this domain is the automated assignment of microgenres to movies. This study introduces and evaluates approaches based on clustering, topic modeling, and word embedding to address this task. The evaluation employs a preprocessed dataset containing movie-related data—title tags, synopses, genres, and reviews—alongside a predefined microgenre list. Comparisons of three activation functions (binary step, ramp, and sigmoid) gauge their effectiveness in augmenting microgenre tags. Results demonstrate the superiority of the word embedding approach over clustering and topic modeling in terms of mean accuracy. Even more, the word embedding approach stands as the sole fully automated solution. Analysis indicates that incorporating review-based tags introduces noise and undermines accuracy. Besides, the word embedding approach yields optimal outcomes using the sigmoid function, effectively doubling assigned tags while maintaining matching quality. This sheds light on the potential of word embedding methods within the movie domain.

In vitro and in silico prediction of antibacterial interaction between essential oils via graph embedding approach

Essential oils contain a variety of volatile metabolites, and are expected to be utilized in wide fields such as antimicrobials, insect repellents and herbicides. However, it is difficult to foresee the effect of oil combinations because hundreds of compounds can be involved in synergistic and antagonistic interactions. In this research, it was developed and evaluated a machine learning method to classify types of (synergistic/antagonistic/no) antibacterial interaction between essential oils. Graph embedding was employed to capture structural features of the interaction network from literature data, and was found to improve in silico predicting performances to classify synergistic interactions. Furthermore, in vitro antibacterial assay against a standard strain of Staphylococcus aureus revealed that four essential oil pairs (Origanum compactum—Trachyspermum ammi, Cymbopogon citratus—Thujopsis dolabrata, Cinnamomum verum—Cymbopogon citratus and Trachyspermum ammi—Zingiber officinale) exhibited synergistic interaction as predicted. These results indicate that graph embedding approach can efficiently find synergistic interactions between antibacterial essential oils.

Comparison of Pre-trained vs Custom-trained Word Embedding Models for Word Sense Disambiguation

The prime objective of word sense disambiguation (WSD) is to develop such machines that can automatically recognize the actual meaning (sense) of ambiguous words in a sentence. WSD can improve various NLP and HCI challenges. Researchers explored a wide variety of methods to resolve this issue of sense ambiguity. However, majorly, their focus was on English and some other well-reputed languages. Urdu with more than 300 million users and a large amount of electronic text available on the web is still unexplored. In recent years, for a variety of Natural Language Processing tasks, word embedding methods have proven extremely successful. This study evaluates, compares, and applies a variety of word embedding approaches to Urdu Word embedding (both Lexical Sample and All-Words), including pre-trained (Word2Vec, Glove, and FastText) as well as custom-trained (Word2Vec, Glove, and FastText trained on the Ur-Mono corpus). Two benchmark corpora are used for the evaluation in this study: (1) the UAW-WSD-18 corpus and (2) the ULS-WSD-18 corpus. For Urdu All-Words WSD tasks, top results have been achieved (Accuracy=60.07 and F1=0.45) using pre-trained FastText. For the Lexical Sample, WSD has been achieved (Accuracy=70.93 and F1=0.60) using custom-trained GloVe word embedding method.

Anchor Link Prediction for Privacy Leakage via De-Anonymization in Multiple Social Networks

Anchor link prediction exacerbates the risk of privacy leakage via the de-anonymization of social network data. Embedding-based methods for anchor link prediction are limited by the excessive similarity of the associated nodes in a latent feature space and the variation between latent feature spaces caused by the semantics of different networks. In this paper, we propose a novel method which reduces the impact of semantic discrepancies between different networks in the latent feature space. The proposed method consists of two phases. First, graph embedding focuses on the network structural roles of nodes and increases the distinction between the associated nodes in the embedding space. Second, a federated adversarial learning framework which performs graph embedding on each social network and an adversarial learning model on the server according to the observable anchor links is used to associate independent graph embedding approaches on different social networks. The combination of distinction enhancement and the association of graph embedding approaches alleviates variance between the latent feature spaces caused by the semantics of different social networks. Extensive experiments on real social networks demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of both precision and robustness.

Topic Modeling based Text Classification Regarding Islamophobia using Word Embedding and Transformers Techniques

Islamophobia is a rising area of concern in the current era where Muslims face discrimination and receive negative perspectives towards their religion, Islam. Islamophobia is a type of racism that is being practiced by individuals, groups, and organizations worldwide. Moreover, the ease of access to social media platforms and their augmented usage has also contributed to spreading hate speech, false information, and negative opinions about Islam. In this research study, we focused to detect Islamophobic textual content shared on various social media platforms. We explored the state-of-the-art techniques being followed in text data mining and Natural Language Processing (NLP). Topic modelling algorithm Latent Dirichlet Allocation is used to find top topics. Then, word embedding approaches such as Word2Vec and Global Vectors for word representation (GloVe) are used as feature extraction techniques. For text classification, we utilized modern text analysis techniques of transformers-based Deep Learning algorithms named Bidirectional Encoders Representation from Transformers (BERT) and Generative Pre-Trained Transformer (GPT). For results comparison, we conducted an extensive empirical analysis of Machine Learning algorithms and Deep Learning using conventional textual features such as the Term Frequency-Inverse Document Frequency, N-gram, and Bag of words (BoW). The empirical based results evaluated using standard performance evaluation measures show that the proposed approach effectively detects the textual content related to Islamophobia. In the corpus of the study under Machine Learning models Support Vector Machine (SVM) performed best with an F1 score of 91%. The Transformer based core NLP models and the Deep Learning model Convolutional Neural Network (CNN) when combined with GloVe performed best among all the techniques except SVM with BoW. GPT, SVM when combined with BoW and BERT yielded the best F1 score of 92%, 92% and 91.9% respectively, while CNN performed slightly poor with an F1 score of 91%.