Generalized Ensemble Methods Research Articles

Replica Exchange of Expanded Ensembles: A Generalized Ensemble Approach with Enhanced Flexibility and Parallelizability.

Generalized ensemble methods such as Hamiltonian replica exchange (HREX) and expanded ensemble (EE) have been shown effective in free energy calculations for various contexts, given their ability to circumvent free energy barriers via nonphysical pathways defined by states with different modified Hamiltonians. However, both HREX and EE methods come with drawbacks, such as limited flexibility in parameter specification or the lack of parallelizability for more complicated applications. To address this challenge, we present the method of replica exchange of expanded ensembles (REXEE), which integrates the principles of HREX and EE methods by periodically exchanging coordinates of EE replicas sampling different yet overlapping sets of alchemical states. With the solvation free energy calculation of anthracene and binding free energy calculation of the CB7-10 binding complex, we show that the REXEE method achieves the same level of accuracy in free energy calculations as the HREX and EE methods, while offering enhanced flexibility and parallelizability. Additionally, we examined REXEE simulations with various setups to understand how different exchange frequencies and replica configurations influence the sampling efficiency in the fixed-weight phase and the weight convergence in the weight-updating phase. The REXEE approach can be further extended to support asynchronous parallelization schemes, allowing looser communications between larger numbers of loosely coupled processors such as cloud computing and therefore promising much more scalable and adaptive executions of alchemical free energy calculations. All algorithms for the REXEE method are available in the Python package ensemble_md, which offers an interface for REXEE simulation management without modifying the source code in GROMACS.

Theoretical Analysis of the Aggregation-Inhibition Effect of Arginine on Polyglutamine Protein by the Generalized-Ensemble Method

Theoretical Analysis of the Aggregation-Inhibition Effect of Arginine on Polyglutamine Protein by the Generalized-Ensemble Method

Fly casting with ligand sliding and orientational selection supporting complex formation of a GPCR and a middle sized flexible molecule

A GA-guided multidimensional virtual-system coupled molecular dynamics (GA-mD-VcMD) simulation was conducted to elucidate binding mechanisms of a middle-sized flexible molecule, bosentan, to a GPCR protein, human endothelin receptor type B (hETB). GA-mD-VcMD is a generalized ensemble method that produces a free-energy landscape of the ligand-receptor binding by searching large-scale motions accompanied with stable maintenance of the fragile cell-membrane structure. All molecular components (bosentan, hETB, membrane, and solvent) were represented with an all-atom model. Then sampling was conducted from conformations where bosentan was distant from the binding site in the hETB binding pocket. The deepest basin in the resultant free-energy landscape was assigned to native-like complex conformation. The following binding mechanism was inferred. First, bosentan fluctuating randomly in solution is captured using a tip region of the flexible N-terminal tail of hETB via nonspecific attractive interactions (fly casting). Bosentan then slides occasionally from the tip to the root of the N-terminal tail (ligand–sliding). During this sliding, bosentan passes the gate of the binding pocket from outside to inside of the pocket with an accompanying rapid reduction of the molecular orientational variety of bosentan (orientational selection). Last, in the pocket, ligand–receptor attractive native contacts are formed. Eventually, the native-like complex is completed. The bosentan-captured conformations by the tip-region and root-region of the N-terminal tail correspond to two basins in the free-energy landscape. The ligand-sliding corresponds to overcoming of a free-energy barrier between the basins.

Corn Yield Prediction With Ensemble CNN-DNN.

We investigate the predictive performance of two novel CNN-DNN machine learning ensemble models in predicting county-level corn yields across the US Corn Belt (12 states). The developed data set is a combination of management, environment, and historical corn yields from 1980 to 2019. Two scenarios for ensemble creation are considered: homogenous and heterogenous ensembles. In homogenous ensembles, the base CNN-DNN models are all the same, but they are generated with a bagging procedure to ensure they exhibit a certain level of diversity. Heterogenous ensembles are created from different base CNN-DNN models which share the same architecture but have different hyperparameters. Three types of ensemble creation methods were used to create several ensembles for either of the scenarios: Basic Ensemble Method (BEM), Generalized Ensemble Method (GEM), and stacked generalized ensembles. Results indicated that both designed ensemble types (heterogenous and homogenous) outperform the ensembles created from five individual ML models (linear regression, LASSO, random forest, XGBoost, and LightGBM). Furthermore, by introducing improvements over the heterogenous ensembles, the homogenous ensembles provide the most accurate yield predictions across US Corn Belt states. This model could make 2019 yield predictions with a root mean square error of 866 kg/ha, equivalent to 8.5% relative root mean square and could successfully explain about 77% of the spatio-temporal variation in the corn grain yields. The significant predictive power of this model can be leveraged for designing a reliable tool for corn yield prediction which will in turn assist agronomic decision makers.

Generalized-ensemble method study: A helix-mimetic compound inhibits protein-protein interaction by long-range and short-range intermolecular interactions.

A heterocyclic compound mS-11 is a helix-mimetic designed to inhibit binding of an intrinsic disordered protein neural restrictive silence factor/repressor element 1 silencing factor (NRSF/REST) to a receptor protein mSin3B. We apply a generalized ensemble method, multi-dimensional virtual-system coupled molecular dynamics developed by ourselves recently, to a system consisting of mS-11 and mSin3B, and obtain a thermally equilibrated distribution, which is comprised of the bound and unbound states extensively. The lowest free-energy position of mS-11 coincides with the NRSF/REST position in the experimentally-determined NRSF/REST-mSin3B complex. Importantly, the molecular orientation of mS-11 is ordering in a wide region around mSin3B. The resultant binding scenario is: When mS-11 is distant from the binding site of mSin3B, mS-11 descends the free-energy slope toward the binding site maintaining the molecular orientation to be advantageous for binding. Then, finally a long and flexible hydrophobic sidechain of mS-11 fits into the binding site, which is the lowest-free-energy complex structure inhibiting NRSF/REST binding to mSin3B.

MyPresto/omegagene 2020: a molecular dynamics simulation engine for virtual-system coupled sampling.

The molecular dynamics (MD) method is a promising approach for investigating the molecular mechanisms of microscopic phenomena. In particular, generalized ensemble MD methods can efficiently explore the conformational space with a rugged free-energy surface. However, the implementation and acquisition of technical knowledge for each generalized ensemble MD method are not straightforward for end-users. Here, we present a new version of the myPresto/omegagene software, which is an MD simulation engine tailored for a series of generalized ensemble methods, which are virtual-system coupled multicanonical MD (V-McMD), virtual-system coupled adaptive umbrella sampling (V-AUS), and virtual-system coupled canonical MD (VcMD). This program has been applied in several studies analyzing free-energy landscapes of a variety of molecular systems with all-atom simulations. The updated version provides new functionality for coarse-grained simulations powered by the hydrophobicity scale method. The software package includes a step-by-step tutorial document for enhanced conformational sampling of the poly-glutamine (poly-Q) oligomer expressed as a one-bead per residue model. The myPresto/omegagene software is freely available at the following URL: https://github.com/kotakasahara/omegagene under the Apache2 license.

Multidimensional virtual-system coupled canonical molecular dynamics to compute free-energy landscapes of peptide multimer assembly.

An enhanced-sampling method termed multidimensional virtual-system coupled canonical molecular dynamics (mD-VcMD) method is developed. In many cases, generalized-ensemble methods realizing enhanced sampling, for example, adaptive umbrella sampling, apply an effective potential, which is derived from temporarily assumed canonical distribution as a function of one or more arbitrarily defined reaction coordinates. However, it is not straightforward to estimate the appropriate canonical distribution, especially for cases applying multiple reaction coordinates. The current method, mD-VcMD, does not rely on the form of the canonical distribution. Therefore, it is practically useful to explore a high-dimensional reaction-coordinate space. In this article, formulation of mD-VcMD and its evaluation with the simple molecular models consisting of three or four alanine peptides are presented. We confirmed that mD-VcMD efficiently searched 2D and 3D reaction-coordinate spaces defined as interpeptide distances. Direct comparisons with results of long-term canonical MD simulations revealed that mD-VcMD produces correct canonical ensembles. © 2019 Wiley Periodicals, Inc.

Multimodal Structural Distribution of the p53 C-Terminal Domain upon Binding to S100B via a Generalized Ensemble Method: From Disorder to Extradisorder.

Intrinsically disordered regions (IDRs) of a protein employ a flexible binding manner when recognizing a partner molecule. Moreover, it is recognized that binding of IDRs to a partner molecule is accompanied by folding, with a variety of bound conformations often being allowed in formation of the complex. In this study, we investigated a fragment of the disordered p53 C-terminal domain (CTDf) that interacts with one of its partner molecules, S100B, as a representative IDR. Although the 3D structure of CTDf in complex with S100B has been previously reported, the specific interactions remained controversial. To clarify these interactions, we performed generalized ensemble molecular dynamics (MD) simulations (virtual-system coupled multicanonical MD, termed V-McMD), which enable effective conformational sampling beyond that provided by conventional MD. These simulations generated a multimodal structural distribution for our system including CTDf and S100B, indicating that CTDf forms a variety of complex structures upon binding to S100B. We confirmed that our results are consistent with chemical shift perturbations and nuclear Overhauser effects that were observed in previous studies. Furthermore, we calculated the conformational entropy of CTDf in bound and isolated (free) states. Comparison of these CTDf entropies indicated that the disordered CTDf shows further increase in conformational diversity upon binding to S100B. Such entropy gain by binding may comprise an important feature of complex formation for IDRs.

Gibbs Sampler-Based λ-Dynamics and Rao–Blackwell Estimator for Alchemical Free Energy Calculation

λ-dynamics is a generalized ensemble method for alchemical free energy calculations. In traditional λ-dynamics, the alchemical switch variable λ is treated as a continuous variable ranging from 0 to 1 and an empirical estimator is utilized to approximate the free energy. In the present article, we describe an alternative formulation of λ-dynamics that utilizes the Gibbs sampler framework, which we call Gibbs sampler-based λ-dynamics (GSLD). GSLD, like traditional λ-dynamics, can be readily extended to calculate free energy differences between multiple ligands in one simulation. We also introduce a new free energy estimator, the Rao-Blackwell estimator (RBE), for use in conjunction with GSLD. Compared with the current empirical estimator, the advantage of RBE is that RBE is an unbiased estimator and its variance is usually smaller than the current empirical estimator. We also show that the multistate Bennett acceptance ratio equation or the unbinned weighted histogram analysis method equation can be derived using the RBE. We illustrate the use and performance of this new free energy computational framework by application to a simple harmonic system as well as relevant calculations of small molecule relative free energies of solvation and binding to a protein receptor. Our findings demonstrate consistent and improved performance compared with conventional alchemical free energy methods.

Comparing generalized ensemble methods for sampling of systems with many degrees of freedom.

We compare two standard replica exchange methods using temperature and dielectric constant as the scaling variables for independent replicas against two new corresponding enhanced sampling methods based on non-equilibrium statistical cooling (temperature) or descreening (dielectric). We test the four methods on a rough 1D potential as well as for alanine dipeptide in water, for which their relatively small phase space allows for the ability to define quantitative convergence metrics. We show that both dielectric methods are inferior to the temperature enhanced sampling methods, and in turn show that temperature cool walking (TCW) systematically outperforms the standard temperature replica exchange (TREx) method. We extend our comparisons of the TCW and TREx methods to the 5 residue met-enkephalin peptide, in which we evaluate the Kullback-Leibler divergence metric to show that the rate of convergence between two independent trajectories is faster for TCW compared to TREx. Finally we apply the temperature methods to the 42 residue amyloid-β peptide in which we find non-negligible differences in the disordered ensemble using TCW compared to the standard TREx. All four methods have been made available as software through the OpenMM Omnia software consortium (http://www.omnia.md/).

Risk upper bounds for general ensemble methods with an application to multiclass classification

This paper generalizes a pivotal result from the PAC-Bayesian literature —the C-bound— primarily designed for binary classification to the general case of ensemble methods of voters with arbitrary outputs. We provide a generic version of the C-bound, an upper bound over the risk of models expressed as a weighted majority vote that is based on the first and second statistical moments of the vote's margin. On the one hand, this bound may advantageously be applied on more complex outputs than mere binary outputs, such as multiclass labels and multilabel, and on the other hand, it allows us to consider margin relaxations. We provide a specialization of the bound to multiclass classification together with empirical evidence that the presented theoretical result is tightly bound to the risk of the majority vote classifier. We also give insights as to how the proposed bound may be of use to characterize the risk of multilabel predictors.

Performance of Various Machine Learning Classifiers on Small Datasets with Varying Dimensionalities: A Study

Classification is an important supervised learning technique that is used by many applications. An important factor on which the performance of a classifier depends is the size of the dataset using which the classifier is going to be trained. In this manuscript the authors analyzed five different classification techniques (namely decision trees, KNN, SVM, linear discriminant and Ensemble method) in terms of AUC and predictive accuracy when trained using small datasets with different dimensionalities. The study was done using a dataset with 24 features and 400 instances (samples). The results showed that in general ensemble method (using boosted trees) performed better than others but its performance degraded a bit with reduced dimensionality.

Enhanced conformational sampling to visualize a free-energy landscape of protein complex formation

We introduce various, recently developed, generalized ensemble methods, which are useful to sample various molecular configurations emerging in the process of protein–protein or protein–ligand binding. The methods introduced here are those that have been or will be applied to biomolecular binding, where the biomolecules are treated as flexible molecules expressed by an all-atom model in an explicit solvent. Sampling produces an ensemble of conformations (snapshots) that are thermodynamically probable at room temperature. Then, projection of those conformations to an abstract low-dimensional space generates a free-energy landscape. As an example, we show a landscape of homo-dimer formation of an endothelin-1-like molecule computed using a generalized ensemble method. The lowest free-energy cluster at room temperature coincided precisely with the experimentally determined complex structure. Two minor clusters were also found in the landscape, which were largely different from the native complex form. Although those clusters were isolated at room temperature, with rising temperature a pathway emerged linking the lowest and second-lowest free-energy clusters, and a further temperature increment connected all the clusters. This exemplifies that the generalized ensemble method is a powerful tool for computing the free-energy landscape, by which one can discuss the thermodynamic stability of clusters and the temperature dependence of the cluster networks.

Simulations in generalized ensembles through noninstantaneous switches.

Generalized-ensemble simulations, such as replica exchange and serial generalized-ensemble methods, are powerful simulation tools to enhance sampling of free energy landscapes in systems with high energy barriers. In these methods, sampling is enhanced through instantaneous transitions of replicas, i.e., copies of the system, between different ensembles characterized by some control parameter associated with thermodynamical variables (e.g., temperature or pressure) or collective mechanical variables (e.g., interatomic distances or torsional angles). An interesting evolution of these methodologies has been proposed by replacing the conventional instantaneous (trial) switches of replicas with noninstantaneous switches, realized by varying the control parameter in a finite time and accepting the final replica configuration with a Metropolis-like criterion based on the Crooks nonequilibrium work (CNW) theorem. Here we revise these techniques focusing on their correlation with the CNW theorem in the framework of Markovian processes. An outcome of this report is the derivation of the acceptance probability for noninstantaneous switches in serial generalized-ensemble simulations, where we show that explicit knowledge of the time dependence of the weight factors entering such simulations is not necessary. A generalized relationship of the CNW theorem is also provided in terms of the underlying equilibrium probability distribution at a fixed control parameter. Illustrative calculations on a toy model are performed with serial generalized-ensemble simulations, especially focusing on the different behavior of instantaneous and noninstantaneous replica transition schemes.

Conformational Sampling of Unmodified and Acetylated H3 Histone Tails on a Nucleosome by All-Atom Model Molecular Dynamics Simulations

A nucleosome, 146 or 147 base-pair DNA wrapped around a histone octamer composed of two copies of each H3, H4, H2A, and H2B histone proteins, is a compact unit structure to store eukaryotic DNA into the cell nucleus. Although the X-ray conformations of the core region have been already determined, the conformations of the disordered histone terminal regions (histone tails) remain poorly understood. Recent experimental evidences suggest that chemical modifications on the histone tails regulate DNA functions, such as transcription, duplication, and splicing. To understand the regulation mechanism, it is necessary to elucidate difference between conformational states of unmodified and modified histone tails. Molecular dynamics (MD) simulations by generalized ensemble methods such as replica exchange MD and multicanonical MD are effective means to investigate the conformational ensemble of biological systems and have been applied for protein systems with small charge polarization. However, these means are not suitable to obtain the ensemble of histone tails in a form of a nucleosome, because strong electrostatic interactions between positively charged histone tails and negatively charged DNA compared with those in regular protein systems makes difficult to realize a sufficient sampling. Thus, we applied adaptive lambda square dynamics (ALSD) simulation we developed recently to investigate the conformations of H3 histone tails. ALSD dynamically scales the simulation parameters (charge, van der Walls and torsion energies) only for the histone tails during the simulations. This successfully sampled various histone tail conformations. In this poster, we introduce the ALSD simulation results and the differences in conformational ensembles between unmodified and acetylated H3 histone tails.

Particle Swarm Optimization Based Selective Ensemble of Online Sequential Extreme Learning Machine

A novel particle swarm optimization based selective ensemble (PSOSEN) of online sequential extreme learning machine (OS-ELM) is proposed. It is based on the original OS-ELM with an adaptive selective ensemble framework. Two novel insights are proposed in this paper. First, a novel selective ensemble algorithm referred to as particle swarm optimization selective ensemble is proposed, noting that PSOSEN is a general selective ensemble method which is applicable to any learning algorithms, including batch learning and online learning. Second, an adaptive selective ensemble framework for online learning is designed to balance the accuracy and speed of the algorithm. Experiments for both regression and classification problems with UCI data sets are carried out. Comparisons between OS-ELM, simple ensemble OS-ELM (EOS-ELM), genetic algorithm based selective ensemble (GASEN) of OS-ELM, and the proposed particle swarm optimization based selective ensemble of OS-ELM empirically show that the proposed algorithm achieves good generalization performance and fast learning speed.

In silico concurrent multisite pH titration in proteins.

The concurrent proton binding at multiple sites in macromolecules such as proteins and nucleic acids is an important yet challenging problem in biochemistry. We develop an efficient generalized Hamiltonian approach to attack this issue. Based on the previously developed generalized-ensemble methods, an effective potential energy is constructed which combines the contributions of all (relevant) protonation states of the molecule. The effective potential preserves important phase regions of all states and, thus, allows efficient sampling of these regions in one simulation. The need for intermediate states in alchemical free energy simulations is greatly reduced. Free energy differences between different protonation states can be determined accurately and enable one to construct the grand canonical partition function. Therefore, the complicated concurrent multisite proton titration process of protein molecules can be satisfactorily simulated. Application of this method to the simulation of the pKa of Glu49, Asp50, and C-terminus of bovine pancreatic trypsin inhibitor shows reasonably good agreement with published experimental work. This method provides an unprecedented vivid picture of how different protonation states change their relative population upon pH titration. We believe that the method will be very useful in deciphering the molecular mechanism of pH-dependent biomolecular processes in terms of a detailed atomistic description.

Adaptive generalized ensemble construction with feature selection and its application in recommendation

AbstractThis paper presents an adaptive generalized ensemble method with refined feature selection strategy and self-adjusted mechanism for ensemble size. The coevolutionary algorithm is introduced to optimize the ensemble and the feature weighting. There are two stages in the proposed method. In the coevolutionary stage, a component network corresponds to a subpopulation and the feature set is designed in another subpopulation. All subpopulations are coevolved simultaneously. Moreover, the study on the ensemble size is conducted in the structure refining stage. Finally, we apply the proposed approach to a recommendation task. Experimental results indicate that the proposed algorithm can achieve good classification performance, small feature subsets and compact ensemble structure.

A virtual-system coupled multicanonical molecular dynamics simulation: Principles and applications to free-energy landscape of protein–protein interaction with an all-atom model in explicit solvent

We propose a novel generalized ensemble method, a virtual-system coupled multicanonical molecular dynamics (V-McMD), to enhance conformational sampling of biomolecules expressed by an all-atom model in an explicit solvent. In this method, a virtual system, of which physical quantities can be set arbitrarily, is coupled with the biomolecular system, which is the target to be studied. This method was applied to a system of an Endothelin-1 derivative, KR-CSH-ET1, known to form an antisymmetric homodimer at room temperature. V-McMD was performed starting from a configuration in which two KR-CSH-ET1 molecules were mutually distant in an explicit solvent. The lowest free-energy state (the most thermally stable state) at room temperature coincides with the experimentally determined native complex structure. This state was separated to other non-native minor clusters by a free-energy barrier, although the barrier disappeared with elevated temperature. V-McMD produced a canonical ensemble faster than a conventional McMD method.

Efficiency of Replica Exchange Sampling in Protein Folding

Replica exchange molecular dynamics (REX-MD) is a generalized ensemble method, which periodically exchanges replicas between neighbor temperature windows to help cross energy barriers in energy space, therefore enhancing the sampling efficiency. It has been shown to be very effective on simple two-state model systems. However, for more complicated processes such as protein folding, REMD has not been adequately tested. In particular, in simulations of proteins in the current physics-based force fields, different replicas often end up trapped in segregated regions of the phase space, which significantly reduces the sampling efficiency. Here we systematically investigate the efficiency of REX sampling using simplified, yet realistic, corse-grained protein models with various degree of frustration in the protein energy landscape. We also investigate the efficacy of using several previously proposed optimal setups, such as the highest temperature, in REXMD. At the end, we also propose a simple strategy to circumvent the phase space trapping by periodically forcing replicas to visit different temperature ranges.