Functional Embedding Research Articles

Prediction of Plant Ubiquitylation Proteins and Sites by Fusing Multiple Features

Introduction: Protein ubiquitylation is an important post-translational modification (PTM), which is considered to be one of the most important processes regulating cell function and various diseases. Therefore, accurate prediction of ubiquitylation proteins and their PTM sites is of great significance for the study of basic biological processes and the development of related drugs. Researchers have developed some large-scale computational methods to predict ubiquitylation sites, but there is still much room for improvement. Much of the research related to ubiquitylation is cross-species while the life pattern is diversified, and the prediction method always shows its specificity in practical application. This study just aims at the issue of plants and has constructed computational methods for identifying ubiquitylation protein and ubiquitylation sites. Method: In this work, we constructed two predictive models to identify plant ubiquitylation proteins and sites. First, in the ubiquitylation proteins prediction model, in order to better reflect protein sequence information and obtain better prediction results, the KNN scoring matrix model based on functional domain Gene Ontology (GO) annotation and word embedding model, i.e. Skip-Gram and Continuous Bag of Words (CBOW), are used to extract the features, and the light gradient boosting machine (LGBM) is selected as the ubiquitylation proteins prediction engine. Results: As a result, accuracy (ACC), Precision, recall rate (Recall), F1_score and AUC are respectively 85.12%, 80.96%, 72.80%, 76.37% and 0.9193 in the 10-fold cross-validations on independent dataset. In the ubiquitylation sites prediction model, Skip-Gram, CBOW and enhanced amino acid composition (EAAC) feature extraction codes were used to extract protein sequence fragment features, and the predicted results on training and independent test data have also achieved good performance. Conclusion: In a word, the comparison results demonstrate that our models have a decided advantage in predicting ubiquitylation proteins and sites, and it may provide useful insights for studying the mechanisms and modulation of ubiquitination pathways

Integrating subsystem embedding subalgebras and coupled cluster Green’s function: a theoretical foundation for quantum embedding in excitation manifold

In this study, we introduce a novel approach to coupled-cluster Green’s function (CCGF) embedding by seamlessly integrating conventional CCGF theory with the state-of-the-art sub-system embedding sub-algebras coupled cluster (SES-CC) formalism. This integration focuses primarily on delineating the characteristics of the sub-system and the corresponding segments of the Green’s function, defined explicitly by active orbitals. Crucially, our work involves the adaptation of the SES-CC paradigm, addressing the left eigenvalue problem through a distinct form of Hamiltonian similarity transformation. This advancement not only facilitates a comprehensive representation of the interaction between the embedded sub-system and its surrounding environment but also paves the way for the quantum mechanical description of multiple embedded domains, particularly by employing the emergent quantum flow algorithms. Our theoretical underpinnings further set the stage for a generalization to multiple embedded sub-systems. This expansion holds significant promise for the exploration and application of non-equilibrium quantum systems, enhancing the understanding of system–environment interactions. In doing so, the research underscores the potential of SES-CC embedding within the realm of quantum computations and multi-scale simulations, promising a good balance between accuracy and computational efficiency.

Security-Aware Service Function Chaining and Embedding With Asymmetric Dedicated Protection

Security-Aware Service Function Chaining and Embedding With Asymmetric Dedicated Protection

An object-oriented approach to the analysis of spatial complex data over stream-network domains

We address the problem of spatial prediction for Hilbert data, when their spatial domain of observation is a river network. The reticular nature of the domain requires to use geostatistical methods based on the concept of Stream Distance, which captures the spatial connectivity of the points in the river induced by the network branching. Within the framework of Object Oriented Spatial Statistics (O2S2), where the data are considered as points of an appropriate (functional) embedding space, we develop a class of functional moving average models based on the Stream Distance. Both the geometry of the data and that of the spatial domain are thus taken into account. A consistent definition of covariance structure is developed, and associated estimators are studied. Through the analysis of the summer water temperature profiles in the Middle Fork River (Idaho, USA), our methodology proved to be effective, both in terms of covariance structure characterization and forecasting performance.

BinAIV: Semantic-enhanced vulnerability detection for Linux x86 binaries

Binary code vulnerability detection is an important research direction in the field of network security. The extensive reuse of open-source code has led to the spread of vulnerabilities that originally only affected a small number of targets to other software. Existing vulnerability detection methods are mainly based on binary code similarity analysis, that is, by comparing the similarity of code embedding to detect vulnerabilities. However, existing methods lack semantic understanding of binary code and cannot distinguish between different functions with similar code structures, which reduces the accuracy of vulnerability detection. This paper proposes a binary vulnerability detection method BinAIV based on function semantics. BinAIV is based on a neural network model, which defines and constructs binary function semantics to achieve more accurate similarity analysis. Experimental results show that in terms of binary code similarity analysis performance, BinAIV has a significant improvement compared to traditional methods that only use function embedding. In cross-compiler function search, cross-optimization function search, and cross-obfuscation function search experiments, the average Recall@1 value of BinAIV compared to the best-performing baseline methods increased by 40.1 %, 99.8 %, and 184.0 %. In the real-world vulnerability detection experiment, BinAIV had the highest detection accuracy for all vulnerabilities, with an improvement of 155.1 % and 97.7 % compared to Asm2Vec and SAFE, respectively.

SENSE: An unsupervised semantic learning model for cross-platform vulnerability search

Binary Similarity Analysis (BSA) emerges as a vital approach for identifying homologous vulnerabilities. However, it is constrained by semantic incompleteness, structural differences, and false positives arising from variations in compilation environments. In this paper, we propose a novel Unsupervised Semantic Learning Model named SENSE for cross-platform vulnerability search. The model comprises two main components: semantic learner and graph learner. The semantic learner is pre-trained with a mask language task on a well-normalized binary corpus, enabling it to capture contextual semantic relations and generate block embedding that effectively encode the semantic features. In the graph learner, a gated graph neural network with a self-gating layer is adopted to eliminate redundant features and an adversarial loss is incorporated to enhance the robustness of function embedding across different compiler environments. Finally, SENSE is trained in an unsupervised manner using a batch-wise sampling strategy along with maximum mutual information loss. This encourages semantically similar functions to exhibit tighter embedding representations, thereby reducing false positives and improving search efficiency. Through extensive experiments, we have demonstrated that SENSE outperforms state-of-the-art methods in terms of binary search accuracy. Our results also reveal that SENSE is capable of generating robust function embedding that mitigate the differences arising from diverse architecture and optimization options.

IoTSim: Internet of Things-Oriented Binary Code Similarity Detection with Multiple Block Relations.

Binary code similarity detection (BCSD) plays a crucial role in various computer security applications, including vulnerability detection, malware detection, and software component analysis. With the development of the Internet of Things (IoT), there are many binaries from different instruction architecture sets, which require BCSD approaches robust against different architectures. In this study, we propose a novel IoT-oriented binary code similarity detection approach. Our approach leverages a customized transformer-based language model with disentangled attention to capture relative position information. To mitigate out-of-vocabulary (OOV) challenges in the language model, we introduce a base-token prediction pre-training task aimed at capturing basic semantics for unseen tokens. During function embedding generation, we integrate directed jumps, data dependency, and address adjacency to capture multiple block relations. We then assign different weights to different relations and use multi-layer Graph Convolutional Networks (GCN) to generate function embeddings. We implemented the prototype of IoTSim. Our experimental results show that our proposed block relation matrix improves IoTSim with large margins. With a pool size of 103, IoTSim achieves a recall@1 of 0.903 across architectures, outperforming the state-of-the-art approaches Trex, SAFE, and PalmTree.

Service Function Chaining and Embedding With Heterogeneous Faults Tolerance in Edge Networks

In the 5G-and-beyond era, ultra-reliable low latency communication (URLLC) services are ubiquitous in edge networks. To enhance the performance metrics and the quality of service (QoS), URLLC services are delivered via a sequence of software-based network functions, also known as a service function chain (SFC). Towards reliable SFC delivery, it is imperative to incorporate fault-tolerance during SFC deployments. However, deploying an SFC with fault-tolerance is challenging because the protection mechanism needs to jointly consider multiple concurrent physical/virtual network failures and hardware/software failures. Considering these concurrent heterogeneous failures, this work investigates how to effectively deliver an SFC in edge networks with the objective of minimizing bandwidth resource consumption. First, we introduce the concept of k-heterogeneous-faults-tolerance and propose an augmented protection graph, called k-connected service function slices layered graph (KC-SLG). Based on the KC-SLG, we formulate a novel problem called k-heterogeneous-faults-tolerant SFC embedding and propose an effective algorithm, called fault-tolerant service function graph embedding (FT-SFGE). FT-SFGE employs two proposed techniques: k-connected network slicing (KC-NS) and k-connected function slicing (KC-FS). Via thorough mathematical proofs, we show that KC-NS is 2-approximate. Extensive simulations show that KC-FS has the best average cost-efficiency when k = 2, and FT-SFGE outperforms the schemes directly extended from the state-of-the-art.

A unified density matrix functional construction of quantum baths in density matrix embedding theory beyond the mean-field approximation.

The equivalence in one-electron quantum baths between the practical implementation of density matrix embedding theory (DMET) and the more recent Householder-transformed density matrix functional embedding theory has been shown previously in the standard but special case where the reference full-size (one-electron reduced) density matrix, from which the bath is constructed, is idempotent [S. Yalouz et al., J. Chem. Phys. 157, 214112 (2022)]. We prove mathematically that the equivalence remains valid when the density matrix is not idempotent anymore, thus allowing for the construction of correlated (one-electron) quantum baths. A density-matrix functional exactification of DMET is derived within the present unified quantum embedding formalism. Numerical examples reveal that the embedding cluster can be quite sensitive to the level of density-matrix functional approximation used for computing the reference density matrix.

Solvent Dynamics Are Critical to Understanding Carbon Dioxide Dissolution and Hydration in Water.

Simulations of carbon dioxide (CO2) in water may aid in understanding the impact of its accumulation in aquatic environments and help advance technologies for carbon capture and utilization (via, e.g., mineralization). Quantum mechanical (QM) simulations based on static molecular models with polarizable continuum solvation poorly reproduce the energetics of CO2 hydration to form carbonic acid in water, independent of the level of QM theory employed. Only with density-functional-theory-based molecular dynamics and rare-event sampling, followed by energy corrections based on embedded correlated wavefunction theory (in conjunction with density functional embedding theory), can a close agreement between theory and experiment be achieved. Such multilevel simulations can serve as benchmarks for simpler, less costly models, giving insight into potential errors of the latter. The strong influence of sampling/averaging over dynamical solvent configurations on the energetics stems from the difference in polarity of both the transition state and product (both polar) versus the reactant (nonpolar). When a solute undergoes a change in polarity during reaction, affecting its interaction with the solvent, careful assessment of the energetic contribution of the solvent response to this change is critical. We show that static models (without structural sampling) that incorporate three explicit water molecules can yield far superior results than models with more explicit water molecules because fewer water molecules yield less configurational artifacts. Static models intelligently incorporating both explicit (molecules directly participating in the reaction) and implicit solvation, along with a proper QM theory, e.g., CCSD(T) for closed-shell systems, can close the accuracy gap between static and dynamic models.

Sem2vec : Semantics-aware Assembly Tracelet Embedding

Binary code similarity is the foundation of many security and software engineering applications. Recent works leverage deep neural networks (DNN) to learn a numeric vector representation (namely, embeddings ) of assembly functions, enabling similarity analysis in the numeric space. However, existing DNN-based techniques capture syntactic-, control flow-, or data flow-level information of assembly code, which is too coarse-grained to represent program functionality. These methods can suffer from low robustness to challenging settings such as compiler optimizations and obfuscations. We present sem2vec , a binary code embedding framework that learns from semantics . Given the control-flow graph (CFG), 34 pages. of an assembly function, we divide it into tracelets , denoting continuous and short execution traces that are reachable from the function entry point. We use symbolic execution to extract symbolic constraints and other auxiliary information on each tracelet. We then train masked language models to compute embeddings of symbolic execution outputs. Last, we use graph neural networks, to aggregate tracelet embeddings into the CFG-level embedding for a function. Our evaluation shows that sem2vec extracts high-quality embedding and is robust against different compilers, optimizations, architectures, and popular obfuscation methods including virtualization obfuscation. We further augment a vulnerability search application with embeddings computed by sem2vec and demonstrate a significant improvement in vulnerability search accuracy.

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

Feature embedding learning and feature interaction modeling are two crucial components of deep models for Click-Through Rate (CTR) prediction. Most existing deep CTR models suffer from the following three problems. First, feature interactions are either manually designed or simply enumerated. Second, all the feature interactions are modeled with an identical interaction function. Third, in most existing models, different features share the same embedding size which leads to memory inefficiency. To address these three issues mentioned above, we propose Automatic Interaction Machine (AIM) with three core components, namely, Feature Interaction Search (FIS), Interaction Function Search (IFS) and Embedding Dimension Search (EDS), to select significant feature interactions, appropriate interaction functions and necessary embedding dimensions automatically in a unified framework. Specifically, FIS component automatically identifies different orders of essential feature interactions with useless ones pruned; IFS component selects appropriate interaction functions for each individual feature interaction in a learnable way; EDS component automatically searches proper embedding size for each feature. Offline experiments on three large-scale datasets validate the superior performance of AIM. A three-week online A/B test in the recommendation service of a mainstream app market shows that AIM improves DeepFM model by 4.4% in terms of CTR.

Auto-NFT: Automated Network Function Translator in Virtualized Programmable Data Plane

Programmable data plane (PDP) virtualization is a novel technique that enables multiple instances to be supported on a programmable switch. Conventional hypervisor-based virtualization approaches require the hypervisor installation and manual embedding of network functions (NFs), which increases the complexity of PDP virtualization significantly. To address this problem, we propose an automated NF translator (Auto- NFT) that automatically generates and manages the flow rules for a given NF. In this article, we first present background information about the programmable switch and its virtualization. We then describe the design and provide implementation details of Auto-NFT, which was implemented over a commercial programmable switch. The experimental results demonstrate that Auto-NFT outperforms conventional approaches and shows near-optimal performance in terms of the NF embedding success rate and packet processing latency.

Triple Excitations in Green's Function Coupled Cluster Solver for Studies of Strongly Correlated Systems in the Framework of Self-Energy Embedding Theory.

Embedding theories became important approaches used for accurate calculations of both molecules and solids. In these theories, a small chosen subset of orbitals is treated with an accurate method, called an impurity solver, capable of describing higher correlation effects. Ideally, such a chosen fragment should contain multiple orbitals responsible for the chemical and physical behavior of the compound. Handling a large number of chosen orbitals presents a very significant challenge for the current generation of solvers used in the physics and chemistry community. Here, we develop a Green's function coupled cluster singles doubles and triples (GFCCSDT) solver that can be used for a quantitative description in both molecules and solids. This solver allows us to treat orbital spaces that are inaccessible to other accurate solvers. At the same time, GFCCSDT maintains high accuracy of the resulting self-energy. Moreover, in conjunction with the GFCCSD solver, it allows us to test the systematic convergence of computational studies. Developing the CC family of solvers paves the road to fully systematic Green's function embedding calculations in solids. In this paper, we focus on the investigation of GFCCSDT self-energies for a strongly correlated problem of SrMnO3 solid. Subsequently, we apply this solver to solid MnO showing that an approximate variant of GFCCSDT is capable of yielding a high accuracy orbital resolved spectral function.

Functional re-organization of hippocampal-cortical gradients during naturalistic memory processes

The functional organization of the hippocampus mirrors that of the cortex, changing smoothly along connectivity gradients and abruptly at inter-areal boundaries. Hippocampal-dependent cognitive processes require flexible integration of these hippocampal gradients into functionally related cortical networks. To understand the cognitive relevance of this functional embedding, we acquired fMRI data while participants viewed brief news clips, either containing or lacking recently familiarized cues. Participants were 188 healthy mid-life adults and 31 adults with mild cognitive impairment (MCI) or Alzheimer's disease (AD). We employed a recently developed technique - connectivity gradientography - to study gradually changing patterns of voxel to whole brain functional connectivity and their sudden transitions. We observed that functional connectivity gradients of the anterior hippocampus map onto connectivity gradients across the default mode network during these naturalistic stimuli. The presence of familiar cues in the news clips accentuates a stepwise transition across the boundary from the anterior to the posterior hippocampus. This functional transition is shifted in the posterior direction in the left hippocampus of individuals with MCI or AD. These findings shed new light on the functional integration of hippocampal connectivity gradients into large-scale cortical networks, how these adapt with memory context and how these change in the presence of neurodegenerative disease.

On the influence of over-parameterization in manifold based surrogates and deep neural operators

Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question is what type of surrogate models we should construct and should these models be under-parameterized or over-parameterized. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. While m-PCE enables the construction of a mapping by first identifying low-dimensional embeddings of the input functions, parameters, and quantities of interest (QoIs), a neural operator learns the nonlinear mapping via the use of deep neural networks. We demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a mapping between latent spaces formed by two separate embeddings of the input functions and the output QoIs. To further enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output mappings. However, when highly non-smooth dynamics is considered, DeepONet shows higher approximation accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another recently proposed operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Taken together, our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.

Projection-Based Density Matrix Renormalization Group in Density Functional Theory Embedding.

The density matrix renormalization group (DMRG) method has already proved itself as a very efficient and accurate computational method, which can treat large active spaces and capture the major part of strong correlation. Its application on larger molecules is, however, limited by its own computational scaling as well as demands of methods for treatment of the missing dynamical electron correlation. In this work, we present the first step in the direction of combining DMRG with density functional theory (DFT), one of the most employed quantum chemical methods with favorable scaling, by means of the projection-based wave function (WF)-in-DFT embedding. On two proof-of-concept but important molecular examples, we demonstrate that the developed DMRG-in-DFT approach provides a very accurate description of molecules with a strongly correlated fragment.

Quantum embedding of multi-orbital fragments using the block-Householder transformation.

Recently, some of the authors introduced the use of the Householder transformation as a simple and intuitive method for embedding local molecular fragments [see Sekaran et al., Phys. Rev. B 104, 035121 (2021) and Sekaran et al., Computation 10, 45 (2022)]. In this work, we present an extension of this approach to the more general case of multi-orbital fragments using the block version of the Householder transformation applied to the one-body reduced density matrix, unlocking the applicability to general quantum chemistry/condensed matter physics Hamiltonians. A step-by-step construction of the block Householder transformation is presented. Both physical and numerical areas of interest of the approach are highlighted. The specific mean-field (noninteracting) case is thoroughly detailed as it is shown that the embedding of a given N spin-orbital fragment leads to the generation of two separated sub-systems: (1) a 2N spin-orbitals "fragment+bath" cluster that exactly contains N electrons and (2) a remaining cluster's "environment" described by so-called core electrons. We illustrate the use of this transformation in different cases of embedding scheme for practical applications. We particularly focus on the extension of the previously introduced Local Potential Functional Embedding Theory and Householder-transformed Density Matrix Functional Embedding Theory to the case of multi-orbital fragments. These calculations are realized on different types of systems, such as model Hamiltonians (Hubbard rings) and ab initio molecular systems (hydrogen rings).

Efficient Implementation of Density Functional Theory Based Embedding for Molecular and Periodic Systems Using Gaussian Basis Functions.

A practical and effective implementation of density functional theory based embedding is reported, which allows us to treat both periodic and aperiodic systems on an equal footing. Its essence is the expansion of orbitals and electron density of the periodic system using Gaussian basis functions, rather than plane-waves, which provides a unique all-electron direct-space representation, thus avoiding the need for pseudopotentials. This makes the construction of embedding potential for a molecular active subsystem due to a periodic environment quite convenient, as transformation between representations is far from trivial. The three flavors of embedding, molecule-in-molecule, molecule-in-periodic, and periodic-in-periodic embedding, are implemented using embedding potentials based on non-additive kinetic energy density functionals (approximate) and level-shift projection operator (exact). The embedding scheme is coupled with a variety of correlated wave function theory (WFT) methods, thereby providing an efficient way to study the ground and excited state properties of low-dimensional systems using high-level methods for the region of interest. Finally, an implementation of real time-time-dependent density functional embedding theory (RT-TDDFET) is presented that uses a projection operator-based embedding potential and provides accurate results compared to full RT-TDDFT for systems with uncoupled excitations. The embedding potential is calculated efficiently using a combination of density fitting and continuous fast multipole method for the Coulomb term. The applicability of (i) WFT-in-DFT embedding, in predicting the adsorption and excitation energies, and (ii) RT-TDDFET, in predicting the absorption spectra, is explored for various test systems.

Reactivity of VC Electrolyte at Li-Metal Electrode: New Insights on SEI Initial Formation from Density Functional Embedding Theory

The metallic lithium (Li) represents the most promising anode material among the next generation of solid-state lithium batteries [1]. An efficient strategy to achieve durable and effective Li-anode batteries is by engineering the solid-electrolyte interphase (SEI) with purposely designed molecules.To this aim, the vinylene carbonate (VC) is one of the most used additives in conventional electrolytes. Some recent experiments proved that the VC promotes the formation of a stable and protective SEI layer between Li metal and electrolyte [2, 3]. Unless the well-known SEI composition, it is difficult to control the VC reactivity, that involves dissociation and polymerization at the electrode surface. Therefore, to dissect these tangled processes, here we present new atomistic insights on VC-Lithium SEI formation via first-principles calculations by Density Functional Embedding Theory (DFET) [4, 5]. Such approach has potentialities for modeling complex reactions at hybrid interfaces in electrocatalysis: it is well suited to combine the best feasible approaches for molecular species (in this case, hybrid HF-DFT for VC molecules and derivatives) and for Li metal electrode (semi-local GGA density functional).Our results highlight different VC dissociation pathways, with formation of reactive radical species and localized cluster of Li2O and Li2CO3. The use of hybrid-DFT-in-DFT embedding is crucial for obtaning energy barriers and qualtitative results in agreement with experiments [3]. Overall, the energetics and structural features of these intermediates improve the current understanding of SEI formation process and can be exploited to drive the reactions toward the desired interfacial properties.[1] J. Janek, W. G. Zeier, Nat Energy 1 (2016) 16141.[2] A. L. Michan et al. Chem. Mater. 28 (2016) 8149.[3] Y. Kamikawa, K. Amezawa, K. Terada, J. Phys. Chem. C 124 (2020) 19937[4] C. Huang, M. Pavone, E. A. Carter, J. Chem. Phys. 134 (2011) 154110[5] C. Huang, A.B. Muñoz-García, M. Pavone, J. Chem. Phys. 145 (2016) 244103