Boltzmann Probability Research Articles

Solving Boltzmann optimization problems with deep learning

Decades of exponential scaling in high-performance computing (HPC) efficiency is coming to an end. Transistor-based logic in complementary metal-oxide semiconductor (CMOS) technology is approaching physical limits beyond which further miniaturization will be impossible. Future HPC efficiency gains will necessarily rely on new technologies and paradigms of computing. The Ising model shows particular promise as a future framework for highly energy-efficient computation. Ising systems are able to operate at energies approaching thermodynamic limits for energy consumption of computation. Ising systems can function as both logic and memory. Thus, they have the potential to significantly reduce energy costs inherent to CMOS computing by eliminating costly data movement. The challenge in creating Ising-based hardware is in optimizing useful circuits that produce correct results on fundamentally nondeterministic hardware. The contribution of this paper is a novel machine learning approach, a combination of deep neural networks and random forests, for efficiently solving optimization problems that minimize sources of error in the Ising model. In addition, we provide a process to express a Boltzmann probability optimization problem as a supervised machine learning problem.

On the relative abundances of Cavansite and Pentagonite

Cavansite is a visually stunning blue vanadosilicate mineral with limited occurrences worldwide, whereas Pentagonite is a closely related dimorph with similar physical and chemical properties, yet is extremely rare compared to Cavansite. The reasons behind Pentagonite’s exceptional rarity remain largely unknown. In this study, (a) density functional theory (DFT) is utilized to investigate the electronic structures of Cavansite and Pentagonite at ground state and finite pressures; (b) a two-state Boltzmann probability model is then employed to construct a comprehensive phase diagram that reveals the abundance of each species across a wide range of pressure and temperature conditions; and (c) dehydration characteristics of these two minerals are explored. The present analysis reveals the key factors that contribute to the relative scarcity of Pentagonite, including differences in structural arrangement and electronic configurations between the two minerals. Specifically, it shows that (a) because of the peculiar arrangements of SiO4 polyhedra, Cavansite forms a compact structure (about 2.7% less in volume) resulting in lower energy; (b) at a temperature of about 650K only about 1% Pentagonite can form; (c) vanadium induces a highly localized state in both of these otherwise large-band-gap insulators resulting in an extremely weak magnetic phase that is unlikely to be observed at any reasonable finite temperature; and (d) water molecules are loosely bound inside the microporous crystals of Cavansite and Pentagonite, suggesting potential applications of these minerals in various technological fields.

A probabilistic view of protein stability, conformational specificity, and design

Various approaches have used neural networks as probabilistic models for the design of protein sequences. These "inverse folding" models employ different objective functions, which come with trade-offs that have not been assessed in detail before. This study introduces probabilistic definitions of protein stability and conformational specificity and demonstrates the relationship between these chemical properties and the p(text {structure}|text {seq}) Boltzmann probability objective. This links the Boltzmann probability objective function to experimentally verifiable outcomes. We propose a novel sequence decoding algorithm, referred to as “BayesDesign”, that leverages Bayes’ Rule to maximize the p(text {structure}|text {seq}) objective instead of the p(text {seq}|text {structure}) objective common in inverse folding models. The efficacy of BayesDesign is evaluated in the context of two protein model systems, the NanoLuc enzyme and the WW structural motif. Both BayesDesign and the baseline ProteinMPNN algorithm increase the thermostability of NanoLuc and increase the conformational specificity of WW. The possible sources of error in the model are analyzed.

The non-deterministic genotype-phenotype map of RNA secondary structure.

Selection and variation are both key aspects in the evolutionary process. Previous research on the mapping between molecular sequence (genotype) and molecular fold (phenotype) has shown the presence of several structural properties in different biological contexts, implying that these might be universal in evolutionary spaces. The deterministic genotype-phenotype (GP) map that links short RNA sequences to minimum free energy secondary structures has been studied extensively because of its computational tractability and biologically realistic nature. However, this mapping ignores the phenotypic plasticity of RNA. We define a GP map that incorporates non-deterministic (ND) phenotypes, and take RNA as a case study; we use the Boltzmann probability distribution of folded structures and examine the structural properties of ND GP maps for RNA sequences of length 12 and coarse-grained RNA structures of length 30 (RNAshapes30). A framework is presented to study robustness, evolvability and neutral spaces in the ND map. This framework is validated by demonstrating close correspondence between the ND quantities and sample averages of their deterministic counterparts. When using the ND framework we observe the same structural properties as in the deterministic GP map, such as bias, negative correlation between genotypic robustness and evolvability, and positive correlation between phenotypic robustness and evolvability.

Modeling of micellar‐catalyzed bimolecular ionic reactions by a nonlinear Poisson−Boltzmann equation and an electrostatic free energy model

AbstractThe hydrolysis reactions of crystal violet (CV) and p‐nitrophenyl acetate (PNPA) by sodium hydroxide in cetyltrimethylammonium bromide (CTAB) micellar solution were studied in this work. Combined with the Boltzmann probability of ionic species in an electrostatic potential field, a model with a numerical scheme was developed based on the nonlinear Poisson−Boltzmann equation in the micelle‐cell consisting of the reactants and the surfactant. The numerical solution gives the ionic species distribution in the micelle phase and the radius of the micelle‐phase. The dielectric constant of the micelle‐phase varies in the range of 39.8−54.4. The second‐order reaction rate constants in the micelle‐phase were determined using the electrostatic and dipolar free energy models. For the basic hydrolysis of CV and PNPA, the overall second‐order rate constants are three to five times and 1.4 times greater than the corresponding values in water, respectively. The rate enhancement occurs due to the distribution of the reactive ionic species in the cell region and the enhancement of the Coulombic and dipolar interactions among the ionic and polar reactants.

Replica symmetry breaking for Ulam's problem

We study increasing subsequences (IS) for an ensemble of sequences given by permutation of numbers {1,2,...,n}. We consider a Boltzmann ensemble at temperature T. Thus each IS appears with the corresponding Boltzmann probability where the energy is the negative length -l of the IS. For T -> 0, only ground states, i.e. longest IS (LIS) contribute, also called Ulam's problem. We introduce an algorithm which allows us to directly sample IS in perfect equilibrium in polynomial time, for any given sequence and any temperature. Thus, we can study very large sizes. We obtain averages for the first and second moments of number of IS as function of $n$ and confirm analytical predictions. Furthermore, we analyze for low temperature $T$ the sampled ISs by computing the distribution of overlaps and performing hierarchical cluster analyses. In the thermodynamic limit the distribution of overlaps stays broad and the configuration landscape remains complex. Thus, Ulam's problem exhibits replica symmetry breaking. This means it constitutes a model with complex behavior which can be studied numerically exactly in a highly efficient way, in contrast to other RSB-showing models, like spin glasses or NP-hard optimization problems, where no fast exact algorithms are known.

Hawking radiation as quantum mechanical reflection

In this article, we explore an alternative derivation of Hawking radiation. Instead of the field-theoretic derivation, we have suggested a simpler calculation based on quantum mechanical reflection from a one-dimensional potential. The reflection coefficient shows an exponential fall in energy which, in comparison with the Boltzmann probability distribution, yields a temperature. The temperature is the same as Hawking temperature for spherically symmetric black holes. The derivation gives an exact local calculation of Hawking temperature that involves a region lying entirely outside the horizon. This is a crucial difference from the tunneling calculation, where it is necessary to involve a region inside the horizon.

Statistical Mechanics of Political Polarization.

Rapidly increasing political polarization threatens democracies around the world. Scholars from several disciplines are assessing and modeling polarization antecedents, processes, and consequences. Social systems are complex and networked. Their constant shifting hinders attempts to trace causes of observed trends, predict their consequences, or mitigate them. We propose an equivalent-neighbor model of polarization dynamics. Using statistical physics techniques, we generate anticipatory scenarios and examine whether leadership and/or external events alleviate or exacerbate polarization. We consider three highly polarized USA groups: Democrats, Republicans, and Independents. We assume that in each group, each individual has a political stance s ranging between left and right. We quantify the noise in this system as a “social temperature” T. Using energy E, we describe individuals’ interactions in time within their own group and with individuals of the other groups. It depends on the stance s as well as on three intra-group and six inter-group coupling parameters. We compute the probability distributions of stances at any time using the Boltzmann probability weight exp(−E/T). We generate average group-stance scenarios in time and explore whether concerted interventions or unexpected shocks can alter them. The results inform on the perils of continuing the current polarization trends, as well as on possibilities of changing course.

Induced circular dichroism as a tool to monitor the displacement of ligands between albumins

The induction of chirality in a ligand can be a powerful analytical tool for studying protein–ligand interactions. Here, we advanced by applying the technique to monitor the inversion of the induced circular dichroism (ICD) spectrum when ligands move between human and bovine serum albumin proteins (HSA and BSA). ICD experiments were performed using dimers of methyl vanillate (DVT) and vanillin (DVN). The sign and spectra shape were dependent on the albumin type. DVN presented a positive maximum in 312 nm when complexed with HSA and a negative one in BSA. It was possible to induce and follow the time-dependent displacement of the ligand from BSA (2.2 × 106 M−1) to HSA (6.6 × 105 M−1) via ICD inversion. The Molecular Mechanics Generalized Born Surface Area approach was used to calculate the binding free energy of the conformers, and a dissociation pathway for each system was proposed using Umbrella Sampling calculations. Four energy minima dihedral angle conformers were identified, and the corresponding CD spectra were calculated using the quantum chemistry approach. Then, weighted spectra for the conformationally accessible conformers were obtained based on each conformer's Boltzmann probability distribution. In conclusion, the methodology described in the manuscript might be helpful in monitoring the movement of ligands between proteins that they bind.

Formulation of temperature dependent effective Hartree potential incorporating quadratic over linear molecular DOFs-surface modes couplings and its effect on quantum dynamics of D2 (v = 0, j = 0)/D2 (v = 0, j = 2) on Cu(111) metal surface

The effect of surface temperature on the scattering and dissociation processes of D2 (v = 0, j = 0)/D2 (v = 0, j = 2) on Cu(111) metal plane is investigated by formulating a time and temperature dependent effective Hamiltonian within mean-field approach, where the molecular degrees of freedom and surface mode vibrations are assumed to be weakly coupled with each other. While constructing such effective Hamiltonian with a Hartree product type wavefunction, we incorporate: (i) an operator algebra method to circumscribe higher order-quadratic (anharmonic) molecular DOFs-surface modes couplings over the linear coupling case implemented previously [Adhikari and coworkers, J. Chem. Phys. 154, 104103 (2021)]; (ii) surface temperature by considering Bose–Einstein or Maxwell–Boltzmann probability factors for the initial state distribution of the surface vibrational modes. Finally, a six-dimensional (6D) quantum dynamical (QD) calculation is performed with such newly formulated effective Hartree potential and calculated transition/dissociation probabilities at various surface temperature situations. The quadratic (anharmonic) molecular DOFs-surface modes couplings modify the reaction and transition probabilities substantially with change of surface temperature, which indicates the crucial role of surface mode vibrations on the molecule-surface scattering phenomena.

Scaling and renormalization in the modern theory of polarization: Application to disordered systems

We develop a scaling theory and a renormalization technique in the context of the modern theory of polarization. The central idea is to use the characteristic function (also known as the polarization amplitude) in place of the free energy in the scaling theory and in place of the Boltzmann probability in a position-space renormalization scheme. We derive a scaling relation between critical exponents which we test in a variety of models in one and two dimensions. We then apply the renormalization to disordered systems. In one dimension the renormalized disorder strength tends to infinity indicating the entire absence of extended states. Zero(infinite) disorder is a repulsive(attractive) fixed point. In two and three dimensions, at small system sizes, two additional fixed points appear, both at finite disorder, $W_a$($W_r$) is attractive(repulsive) such that $W_a<W_r$. In three dimensions $W_a$ tends to zero, $W_r$ remains finite, indicating metal-insulator transition at finite disorder. In two dimensions we are limited by system size, but we find that both $W_a$ and $W_r$ decrease significantly as system size is increased.

Investigating sizing induced surface alterations in crystalline powders using surface energy heterogeneity determination

Particle sizing is the most commonly employed and critical unit operation across powder processing industries. In this work, we show the surface energy changes prompted by the sizing operations like milling and sieving in α-lactose monohydrate powders using Finite Dilution Inverse gas chromatography (FD-IGC) analysis. Three separate sieved fractions of α-lactose monohydrate powder were divided into a top, middle and bottom fraction from the same starting material. Similarly, a custom grade α-lactose monohydrate was milled for a different duration to produce two milled samples with different median particle sizes. Sieved sample results showed that the bottom fraction exhibited higher heterogeneity with higher dispersive (γd) surface energy values ranging from 42.5 mJ/m2 to 45.9 mJ/m2 compared to the top and middle fractions. The finest fraction contains more cleaved surfaces that are exposed during the preparation process, i.e. milling, of the material resulting in different surface properties. Furthermore, the surface energy analysis of the milled samples revealed slight but vital differences in the γd heterogeneity profiles. The site-specific distribution of energies was obtained using the Boltzmann probability distribution model and revealed two distinct regions for crystalline α-lactose monohydrate. Thus, our work confirmed that sizing operations like milling and sieving affect the surface energy of the particulate solids due to the changes in properties like size, shape, exposure of internal cleavage planes, etc. and that the surface energy heterogeneity determination using FD-IGC helped in characterising these changes.

Knowledge-Based Unfolded State Model for Protein Design.

The design of proteins and miniproteins is an important challenge. Designed variants should be stable, meaning the folded/unfolded free energy difference should be large enough. Thus, the unfolded state plays a central role. An extended peptide model is often used, where side chains interact with solvent and nearby backbone, but not each other. The unfolded energy is then a function of sequence composition only and can be empirically parametrized. If the space of sequences is explored with a Monte Carlo procedure, protein variants will be sampled according to a well-defined Boltzmann probability distribution. We can then choose unfolded model parameters to maximize the probability of sampling native-like sequences. This leads to a well-defined maximum likelihood framework. We present an iterative algorithm that follows the likelihood gradient. The method is presented in the context of our Proteus software, as a detailed downloadable tutorial. The unfolded model is combined with a folded model that uses molecular mechanics and a Generalized Born solvent. It was optimized for three PDZ domains and then used to redesign them. The sequences sampled are native-like and similar to a recent PDZ design study that was experimentally validated.

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (beta ) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

Kinetic investigation of the reaction of ethylperoxy radicals with ethanol

AbstractThe thermodynamic and kinetic investigation for the reaction of ethylperoxy radical (CH3CH2OO•) with ethanol (CH3CH2OH) was studied computationally with variational transition state theory. The geometry optimization calculations showed that both the reactants have two conformers. The energetics and thermodynamic properties were calculated using the G4 composite method. The results showed that the abstraction of H atom from the ─CH2─ site is most feasible, when compared to the other pathways, since it has the lowest energy barrier. The rate coefficient calculations for the title reaction were carried out using the multistructural canonical variational transition state theory with small curvature tunneling corrections in the temperature range of 400‐1500 K. The results showed a similar trend to that of the energetics in which the abstraction of the H atom from the ─CH2─ site has a larger rate coefficient, when compared to other reaction pathways. The reactivity trend towards the H atom abstraction by ethylperoxy radicals varies as ─CH2 &gt; ─CH3 &gt; ─OH. The contributions from the ground‐state conformers of both the reactants were included into the total kinetics using the Boltzmann probability distribution. The obtained temperature‐dependent expression for the studied reaction is ktotal = 1.51 × 10−32 T6.3 exp(−3691/T) cm3 molecule−1 s−1.

Conformational Entropy from Mobile Bond Vectors in Proteins: A Viewpoint that Unifies NMR Relaxation Theory and Molecular Dynamics Simulation Approaches.

A new method for determining conformational entropy in proteins is reported. Proteins prevail as conformational ensembles, p ∝ exp(-u). By selecting a bond vector (e.g., N-H) as a conformation representative, molecular dynamics simulations can provide (relative to a reference structure) p as approximate Boltzmann probability density and u as N-H potential of mean force (POMF). The latter is as accurate as implied by the force field but statistical in character; this limits the insights it can provide and its utilization. Conformational entropy is given exclusively by u. Deriving it from POMFs renders it accurate but statistical in character. Previously, we devised explicit (i.e., analytical but not exact) potentials made of Wigner functions, D0KL, with L ≤ 4, which closely resemble the corresponding POMFs in form; hence, they also approach the latter in accuracy. Such potentials can be beneficially characterized/compared in terms of composition, symmetry, and associated order parameters. In this study, we develop a method for deriving conformational entropy from them, which also features the benefits specified above. The method developed is applied to the dimerization of the Rho GTPase-binding domain of plexin-B1. Insights into local ordering, entropy compensation, and features of allostery are gained. In previous work, we developed the slowly relaxing local structure (SRLS) approach for the analysis of NMR relaxation from restricted bond vector motion in proteins. SRLS comprises explicit (restricting) potentials of the kind developed here. It also comprises diffusion tensors describing the local motion and related features of local geometry. The complete model fits experimental data. In future work, the explicit potentials developed here will be inserted unchanged in SRLS-based data fitting, thereby improving the picture of structural dynamics. Given that SRLS is unique in featuring potentials that can closely approach the corresponding POMFs in accuracy, the present study is an important step toward generally improving protein dynamics by NMR relaxation.

Data-Driven Fuzzy Modeling Using Restricted Boltzmann Machines and Probability Theory

Fuzzy modeling has many advantages over the non-fuzzy methods, such as robustness against uncertainties and less sensitivity to the varying dynamics of nonlinear systems. Data-driven fuzzy modeling needs to extract fuzzy rules from the input/output data, and train the fuzzy parameters. This paper takes advantages from deep learning, probability theory, fuzzy modeling, and extreme learning machines. We use the restricted Boltzmann machine (RBM) and probability theory to overcome some common problems in data based modeling methods. The RBM is modified such that it can be trained with continuous values. A probability based clustering method is proposed to partition the hidden features from the RBM, and extract fuzzy rules with probability measurement. An extreme learning machine and an optimization method are applied to train the consequent part of the fuzzy rules and the probability parameters. The proposed method is validated with two benchmark problems.

A novel memetic genetic algorithm for solving traveling salesman problem based on multi-parent crossover technique

In the present study, a Novel Memetic Genetic Algorithm (NMGA) is developed to solve the Traveling Salesman Problem (TSP). The proposed NMGA is the combination of Boltzmann probability selection and a multi-parent crossover technique with known random mutation. In the proposed multi-parent crossover parents and common crossing point are selected randomly. After comparing the cost/distance with the adjacent nodes (genes) of participated parents, two offspring’s are produced. To establish the efficiency of the developed algorithm standard benchmarks are solved from TSPLIB against classical genetic algorithm (GA) and the fruitfulness of the proposed algorithm is recognized. Some statistical test has been done and the parameters are studied.

Freezing phase transition in a fractal potential

In this work we propose a simple example of a one-dimensional thermodynamic system where non-interacting particles are allowed to move over the interval, which are influenced by a potential with a fractal structure. We prove that the system exhibits a phase transition at a finite temperature, which is characterized by the fact that the Gibbs–Boltzmann probability measure passes from being absolutely continuous with respect to Lebesgue (at high temperatures) to being singular continuous (at low temperatures). We prove that below the critical temperature (when the Gibbs–Boltzmann probability measure is singular continuous) the probability measure is supported on the middle-third Cantor set and that further lowering the temperature, the probability measure does not change anymore. This means that, in some sense, the system reaches the ground-state before the zero temperature, indicating that the system ‘freezes’ at a positive temperature.

1-Hexanol conformers in a nitrogen matrix: FTIR study and high-level ab initio calculations

FTIR spectra of 1-hexanol trapped in a nitrogen matrix were registered in the temperature range from 10 to 40 K. The experiment has shown that only non‑hydrogen-bonded alcohol molecules were present in the sample at 10 K. Conformational analysis using high-level ROCBS-QB3 ab-initio calculations has shown that only 111 of 122 1-hexanol conformers can exist. The most stable conformer was determined. Anharmonic quantum-chemistry calculations and further special scheme of principal component analysis resulted in at least 12 unneglectable conformers in the sample, 6 of which form about 85% of FTIR spectra in the region of the stretching OH vibrations. Sample heating leads to a significant redistribution between conformers. Relative weights of different conformers at all investigated temperatures did not match Boltzmann probability distribution. Moreover, 1-hexanol structure was formed not by all the most stable structures, but only by some of them.