Efficient Embedding Research Articles

Spiropyran as Building Block in Peptide Synthesis and Modulation of Photochromic Properties.

Light-controlled triggering of materials requires efficient embedding of molecular photoswitches into larger molecules. We herein present the synthesis of two new building blocks for the synthesis of photoswitchable peptides, embedding spiropyranes as a central unit into peptide-backbones via a novel, yet unreported approach. The synthesis presented here allows us to embed spiropyranes directly into solid-phase peptide synthesis (SPPS), further describing the resulting photophysical properties of the as-prepared photoswitchable peptides.

Efficient Unsupervised Graph Embedding With Attributed Graph Reduction and Dual-Level Loss

Efficient Unsupervised Graph Embedding With Attributed Graph Reduction and Dual-Level Loss

ParaAntiProt provides paratope prediction using antibody and protein language models

Efficiently predicting the paratope holds immense potential for enhancing antibody design, treating cancers and other serious diseases, and advancing personalized medicine. Although traditional methods are highly accurate, they are often time-consuming, labor-intensive, and reliant on 3D structures, restricting their broader use. On the other hand, machine learning-based methods, besides relying on structural data, entail descriptor computation, consideration of diverse physicochemical properties, and feature engineering. Here, we develop a deep learning-assisted prediction method for paratope identification, relying solely on amino acid sequences and being antigen-agnostic. Built on the ProtTrans architecture, and utilizing pre-trained protein and antibody language models, we extract efficient embeddings for predicting paratope. By incorporating positional encoding for Complementarity Determining Regions, our model gains a deeper structural understanding, achieving remarkable performance with a 0.904 ROC AUC, 0.701 F1-score, and 0.585 MCC on benchmark datasets. In addition to yielding accurate antibody paratope predictions, our method exhibits strong performance in predicting nanobody paratope, achieving a ROC AUC of 0.912 and a PR AUC of 0.665 on the nanobody dataset. Notably, our approach outperforms structure-based prediction methods, boasting a PR AUC of 0.731. Various conducted ablation studies, which elaborate on the impact of each part of the model on the prediction task, show that the improvement in prediction performance by applying CDR positional encoding together with CNNs depends on the specific protein and antibody language models used. These results highlight the potential of our method to advance disease understanding and aid in the discovery of new diagnostics and antibody therapies.

Def2Vec: you shall know a word by its definition

AbstractDef2Vec introduces a new perspective on building words embeddings by using dictionary definitions. By leveraging term-document matrices derived from dictionary definitions and employing Latent Semantic Analysis (LSA), our method, Def2Vec, yields embeddings characterized by robust performance and adaptability. Through comprehensive evaluations encompassing token classification, sequence classification and semantic similarity, we show empirically how Def2Vec consistently demonstrates competitiveness with established models like Word2Vec, GloVe, and FastText. Notably, our model’s utilization of all the matrices resulting from LSA factorisation facilitates efficient prediction of embeddings for out-of-vocabulary words, given their definition. By effectively integrating the benefits of dictionary definitions with LSA-based embeddings, Def2Vec builds informative semantic representations, all while minimizing data requirements. In this paper, we run different experiment to assess the quality of our embedding model at word level and at sequence level. Our findings contribute to the ongoing evolution of word embedding methodologies by incorporating structured lexical information and enabling efficient embedding prediction.

Universal Ensemble-Embedding Graph Neural Network for Direct Prediction of Optical Spectra from Crystal Structures.

Optical properties in solids, such as refractive index and absorption, hold vast applications ranging from solar panels to sensors, photodetectors, and transparent displays. However, first-principles computation of optical properties from crystal structures is a complex task due to the high convergence criteria and computational cost. Recent progress in machine learning shows promise in predicting material properties, yet predicting optical properties from crystal structures remains challenging due to the lack of efficient atomic embeddings. Here, Graph Neural Network for Optical spectra prediction (GNNOpt) is introduced, an equivariant graph-neural-network architecture featuring universal embedding with automatic optimization. This enables high-quality optical predictions with a dataset of only 944 materials. GNNOpt predicts all optical properties based on the Kramers-Krönig relations, including absorption coefficient, complex dielectric function, complex refractive index, and reflectance. The trained model is applied to screen photovoltaic materials based on spectroscopic limited maximum efficiency and search for quantum materials based on quantum weight. First-principles calculations validate the efficacy of the GNNOpt model, demonstrating excellent agreement in predicting the optical spectra of unseen materials. The discovery of new quantum materials with high predicted quantum weight, such as SiOs, which host exotic quasiparticles with multifold nontrivial topology, demonstrates the potential of GNNOpt in predicting optical properties across a broad range of materials andapplications.

Accurate and efficient protein embedding using multi-teacher distillation learning.

Protein embedding, which represents proteins as numerical vectors, is a crucial step in various learning-based protein annotation/classification problems, including gene ontology prediction, protein-protein interaction prediction, and protein structure prediction. However, existing protein embedding methods are often computationally expensive due to their large number of parameters, which can reach millions or even billions. The growing availability of large-scale protein datasets and the need for efficient analysis tools have created a pressing demand for efficient protein embedding methods. We propose a novel protein embedding approach based on multi-teacher distillation learning, which leverages the knowledge of multiple pre-trained protein embedding models to learn a compact and informative representation of proteins. Our method achieves comparable performance to state-of-the-art methods while significantly reducing computational costs and resource requirements. Specifically, our approach reduces computational time by ∼70% and maintains ±1.5% accuracy as the original large models. This makes our method well-suited for large-scale protein analysis and enables the bioinformatics community to perform protein embedding tasks more efficiently. The source code of MTDP is available via https://github.com/KennthShang/MTDP.

SpemNet: A Cotton Disease and Pest Identification Method Based on Efficient Multi-Scale Attention and Stacking Patch Embedding.

We propose a cotton pest and disease recognition method, SpemNet, based on efficient multi-scale attention and stacking patch embedding. By introducing the SPE module and the EMA module, we successfully solve the problems of local feature learning difficulty and insufficient multi-scale feature integration in the traditional Vision Transformer model, which significantly improve the performance and efficiency of the model. In our experiments, we comprehensively validate the SpemNet model on the CottonInsect dataset, and the results show that SpemNet performs well in the cotton pest recognition task, with significant effectiveness and superiority. The SpemNet model excels in key metrics such as precision and F1 score, demonstrating significant potential and superiority in the cotton pest and disease recognition task. This study provides an efficient and reliable solution in the field of cotton pest and disease identification, which is of great theoretical and applied significance.

Finding Efficient Graph Embeddings and Processing them by a CNN-based Tool

We introduce new tools to support finding efficient graph embedding techniques for graph databases and to process their outputs using deep learning for classification scenarios. Accordingly, we investigate the possibility of creating an ensemble of different graph embedding methods to raise accuracy and present an interconnected neural network-based ensemble to increase the efficiency of the member classification algorithms. We also introduce a new convolutional neural network-based architecture that can be generally proposed to process vectorized graph data provided by various graph embedding methods and compare it with other architectures in the literature to show the competitiveness of our approach. We also exhibit a statistical-based inhomogeneity level estimation procedure to select the optimal embedding for a given graph database efficiently. The efficiency of our framework is exhaustively tested using several publicly available graph datasets and numerous state-of-the-art graph embedding techniques. Our experimental results for classification tasks have proved the competitiveness of our approach by outperforming the state-of-the-art frameworks.

Intrinsically Stretchable Organic Solar Cells and Sensors Enabled by Extensible Composite Electrodes

AbstractStretchable electrodes are critical to the development of advanced technologies such as human–machine interaction, flexible sensing, and wearable power supply, making them of significant research value. However, the current preparation methods for high‐performance stretchable electrodes are complex and inefficient, posing challenges for their large‐scale application in the realm of flexible wearables. To address this need, a straightforward and efficient embedding strategy is reported for fabricating stretchable silver nanowire/thermoplastic elastomer composite electrodes (referred to as Strem‐AT) utilizing the viscoelasticity and outstanding mechanical properties of polymer elastomers to achieve outstanding extensibility, conductivity, and a smooth surface. These electrodes exhibit excellent tensile behavior, low surface roughness, and stable electrical properties, enabling their successful integration into stretchable sensors and intrinsically stretchable organic photovoltaic cells (IS‐OPV). When applied to human skin joints for motion detection, the sensor demonstrates remarkable stretchability and stable signal output. Importantly, the all‐polymer IS‐OPV exhibits a top‐notch power conversion efficiency (PCE) of >12.5% and a PCE80% strain exceeding 50%. Furthermore, even after subjecting high‐strain stretching at 50% for 1000 cycles, the IS‐OPV can retain 76% of the initial PCE. This study presents a multifunctional stretchable electrode with high repeatability and easy‐to‐scale fabrication in wearable sensors and photovoltaics.

The dipole moment of supercritical water – local vs. mean-field polarisation contributions

Supercritical water has attracted much attention from both fundamental and technological perspectives, based largely on its ability to solvate other molecules. Predicting and controlling this requires a deeper understanding of water's polarisation behaviour. Using the computationally efficient Self-Consistent Electrostatic Embedding method, we were able to calculate the water dipole moment over an unprecedented range of thermodynamic conditions, covering gas, liquid and supercritical states, with large simulation systems and a high-level quantum mechanical method. We find a discontinuous change in the dipole moment along subcritical isotherms, corresponding to the sharp transition between the vapour and liquid states, with the latter exhibiting induced dipole moments between 0.5 and 0.9 D, depending on the temperature. In contrast, the dipole moment changes continuously from gas-like to liquid-like behaviour in the supercritical regime, allowing the degree of polarisation to be controlled through manipulating temperature and pressure. The dipole moment was found to be linearly related to the average number of hydrogen-bonded neighbours of water, emphasising the key role of local interactions to the polarisation process. Mean-field approaches based on a dielectric continuum representation of the solvent are unable to predict this behaviour due to the neglect of local interactions.

A deep position-encoding model for predicting olfactory perception from molecular structures and electrostatics

Predicting olfactory perceptions from odorant molecules is challenging due to the complex and potentially discontinuous nature of the perceptual space for smells. In this study, we introduce a deep learning model, Mol-PECO (Molecular Representation by Positional Encoding of Coulomb Matrix), designed to predict olfactory perceptions based on molecular structures and electrostatics. Mol-PECO learns the efficient embedding of molecules by utilizing the Coulomb matrix, which encodes atomic coordinates and charges, as an alternative of the adjacency matrix and its Laplacian eigenfunctions as positional encoding of atoms. With a comprehensive dataset of odor molecules and descriptors, Mol-PECO outperforms traditional machine learning methods using molecular fingerprints and graph neural networks based on adjacency matrices. The learned embeddings by Mol-PECO effectively capture the odor space, enabling global clustering of descriptors and local retrieval of similar odorants. This work contributes to a deeper understanding of the olfactory sense and its mechanisms.

Development of a Face Recognition System for Authentication in Computer-Based Examinations: A Comparison with Fingerprint Verification

This study aims to overcome the limitations of fingerprint biometric authentication in computer-based examinations by introducing a robust face recognition system. The research targets key challenges such as administrative overhead, prolonged authentication times, and issues with false positives and negatives. By leveraging advanced technologies like Multi-Task Cascaded Convolutional Neural Network (MTCNN) for precise facial detection and FaceNet for efficient facial feature embedding, the developed system achieves substantial improvements across critical metrics. The evaluation metrics consistently report high ratings: accuracy 90%, speed 92%, ease of use 86%, robustness to variations 94%, security 92%, and privacy 90%. Evaluation of the system shows its superiority over traditional fingerprint methods. It demonstrates notably higher accuracy, efficiency, user-friendliness, adaptability to diverse conditions, and enhanced levels of security and privacy. These advancements position the face recognition system as a compelling alternative for enhancing the authentication processes in computer-based examinations. Keywords: FaceNet, Multi-Task Cascaded Convolutional Neural Network, Face Recognition, Security, Face, Fingerprint Detection. Sulaimon O.O., Olabiyisi, S.O. & Ismaila, W.O. (2024): Development of a Face Recognition System for Authentication in Computer-Based Examinations: A Comparison with Fingerprint Verification. Journal of Advances in Mathematical & Computational Science. Vol. 12, No. 2. Pp 117-128. Available online at www.isteams.net/mathematics-computationaljournal. dx.doi.org/10.22624/AIMS/MATHS/V12N2P8

TODEAR: Promoting explainable TKG reasoning through temporal offset enhanced dynamic embedding and adaptive reinforcement learning

Recently, Reinforcement Learning (RL) is utilized in Temporal Knowledge Graph (TKG) reasoning to generate analyzable reasoning paths, which achieves explainable reasoning on queries. The process of generating high-quality reasoning paths is facing two major challenges. The first is to construct an efficient entity embedding model from the complex temporal dependencies among entities. The second is to design a fine-grained reward mechanism for the reasoner based on the deep semantics of exploration paths. Motivated by the two challenges, a TKG reasoning framework based on Temporal Offset Enhanced Dynamic Embedding and Adaptive Reinforcement Learning (TODEAR) is proposed in this paper. Firstly, a temporal offset enhanced dynamic embedding model with a distance scoring function is designed to fully exploit the complex temporal dependencies among entities. In order to capture the evolutionary patterns of historical facts, it encodes both the relational structures of the TKG and the temporal offsets between events and queries. Then, a fine-grained adaptive reward mechanism is designed to optimize the reasoner. It generates real-time rewards by analyzing the logic and semantics of exploration paths to mitigate the adverse effects of sparse rewards. Extensive experiments on four benchmark datasets show that TODEAR significantly outperforms the state-of-the-art methods.

An efficient graph embedding clustering approach for heterogeneous network

An efficient graph embedding clustering approach for heterogeneous network

Scalable and unbiased sequence-informed embedding of single-cell ATAC-seq data with CellSpace

Standard scATAC sequencing (scATAC-seq) analysis pipelines represent cells as sparse numeric vectors relative to an atlas of peaks or genomic tiles and consequently ignore genomic sequence information at accessible loci. Here we present CellSpace, an efficient and scalable sequence-informed embedding algorithm for scATAC-seq that learns a mapping of DNA k-mers and cells to the same space, to address this limitation. We show that CellSpace captures meaningful latent structure in scATAC-seq datasets, including cell subpopulations and developmental hierarchies, and can score transcription factor activities in single cells based on proximity to binding motifs embedded in the same space. Importantly, CellSpace implicitly mitigates batch effects arising from multiple samples, donors or assays, even when individual datasets are processed relative to different peak atlases. Thus, CellSpace provides a powerful tool for integrating and interpreting large-scale scATAC-seq compendia.

Fast and efficient context-aware embedding generation using fuzzy hashing for in-vehicle network intrusion detection

In the rapidly advancing field of automotive cybersecurity, the protection of In-Vehicle Networks (IVNs) against cyber threats is crucial. Current deep learning solutions offer robustness but at the cost of high computational demand and potential privacy breaches due to the extensive IVN data required for model training. Our study proposes a novel intrusion detection system (IDS) specifically designed for IVNs that prioritizes computational efficiency and data privacy. Utilizing fuzzy hashing techniques, we generate context-aware embeddings that effectively preserve the privacy of IVN data. Among the machine learning algorithms evaluated, the Support Vector Machine (SVM) emerged as the most effective, particularly when paired with TLSH hash embeddings. This combination achieved notable detection performance, as substantiated by T-SNE visualizations that demonstrate a distinct segregation of normal and attack traffic within the vector space. To validate the effectiveness and practicality of our proposed IDS, we conducted exhaustive experiments on the well-known car-hacking dataset and the more complex ROAD dataset, which includes diverse and sophisticated attack scenarios. Our findings reveal that the proposed lightweight IDS not only demonstrates high detection accuracy but also maintains this performance within the computational constraints of current IVN systems. The system's capability to operate effectively in real-time environments makes it a viable solution for modern automotive cybersecurity needs.

Attentional decoder networks for chest X-ray image recognition on high-resolution features

Background and objective:This paper introduces an encoder–decoder-based attentional decoder network to recognize small-size lesions in chest X-ray images. In the encoder-only network, small-size lesions disappear during the down-sampling steps or are indistinguishable in the low-resolution feature maps. To address these issues, the proposed network processes images in the encoder–decoder architecture similar to U-Net families and classifies lesions by globally pooling high-resolution feature maps. However, two challenging obstacles prohibit U-Net families from being extended to classification: (1) the up-sampling procedure consumes considerable resources, and (2) there needs to be an effective pooling approach for the high-resolution feature maps. Methods:Therefore, the proposed network employs a lightweight attentional decoder and harmonic magnitude transform. The attentional decoder up-samples the given features with the low-resolution features as the key and value while the high-resolution features as the query. Since multi-scaled features interact, up-sampled features embody global context at a high resolution, maintaining pathological locality. In addition, harmonic magnitude transform is devised for pooling high-resolution feature maps in the frequency domain. We borrow the shift theorem of the Fourier transform to preserve the translation invariant property and further reduce the parameters of the pooling layer by an efficient embedding strategy. Results:The proposed network achieves state-of-the-art classification performance on the three public chest X-ray datasets, such as NIH, CheXpert, and MIMIC-CXR. Conclusions:In conclusion, the proposed efficient encoder–decoder network recognizes small-size lesions well in chest X-ray images by efficiently up-sampling feature maps through an attentional decoder and processing high-resolution feature maps with harmonic magnitude transform. We open-source our implementation at https://github.com/Lab-LVM/ADNet.

On the expressivity of embedding quantum kernels

One of the most natural connections between quantum and classical machine learning has been established in the context of kernel methods. Kernel methods rely on kernels, which are inner products of feature vectors living in large feature spaces. Quantum kernels are typically evaluated by explicitly constructing quantum feature states and then taking their inner product, here called embedding quantum kernels. Since classical kernels are usually evaluated without using the feature vectors explicitly, we wonder how expressive embedding quantum kernels are. In this work, we raise the fundamental question: can all quantum kernels be expressed as the inner product of quantum feature states? Our first result is positive: Invoking computational universality, we find that for any kernel function there always exists a corresponding quantum feature map and an embedding quantum kernel. The more operational reading of the question is concerned with efficient constructions, however. In a second part, we formalize the question of universality of efficient embedding quantum kernels. For shift-invariant kernels, we use the technique of random Fourier features to show that they are universal within the broad class of all kernels which allow a variant of efficient Fourier sampling. We then extend this result to a new class of so-called composition kernels, which we show also contains projected quantum kernels introduced in recent works. After proving the universality of embedding quantum kernels for both shift-invariant and composition kernels, we identify the directions towards new, more exotic, and unexplored quantum kernel families, for which it still remains open whether they correspond to efficient embedding quantum kernels.

Geometry-Based Molecular Generation With Deep Constrained Variational Autoencoder.

Finding target molecules with specific chemical properties plays a decisive role in drug development. We proposed GEOM-CVAE, a constrained variational autoencoder based on geometric representation for molecular generation with specific properties, which is protein-context-dependent. In terms of machine learning, it includes continuous feature embedding encoder and molecular generation decoder. Our key contribution is to propose an efficient geometric embedding method, including the spatial structure representations of drug molecule (converting the 3-D coordinates into image) and the geometric graph representations of protein target (modeling the protein surface as a mesh). The 3-D geometric information is vital to successful molecular generation, which is different from previous molecular generative methods based on 1-D or 2-D. Our model framework generates specific molecules in two phases, by first generating special image with molecular 3-D information to learn latent representations and generating molecules with constrained condition based on geometric graph convolution for specific protein and then inputting the generated structural molecules into a parser network for obtaining Simplified Molecular Input Line Entry System (SMILES) strings. Our model achieves competitive performance that implies its potential effectiveness to enable the exploration of the vast chemical space for drug discovery.

DyGCN: Efficient Dynamic Graph Embedding With Graph Convolutional Network.

Graph embedding, aiming to learn low-dimensional representations (aka. embeddings) of nodes in graphs, has received significant attention. In recent years, there has been a surge of efforts, among which graph convolutional networks (GCNs) have emerged as an effective class of models. However, these methods mainly focus on the static graph embedding. In the present work, an efficient dynamic graph embedding approach is proposed, called dynamic GCN (DyGCN), which is an extension of the GCN-based methods. The embedding propagation scheme of GCN is naturally generalized to a dynamic setting in an efficient manner, which propagates the change in topological structure and neighborhood embeddings along the graph to update the node embeddings. The most affected nodes are updated first, and then their changes are propagated to further nodes, which in turn are updated. Extensive experiments on various dynamic graphs showed that the proposed model can update the node embeddings in a time-saving and performance-preserving way.