Log-likelihood Gradient Research Articles

Many-core algorithms for high-dimensional gradients on phylogenetic trees.

Advancements in high-throughput genomic sequencing are delivering genomic pathogen data at an unprecedented rate, positioning statistical phylogenetics as a critical tool to monitor infectious diseases globally. This rapid growth spurs the need for efficient inference techniques, such as Hamiltonian Monte Carlo (HMC) in a Bayesian framework, to estimate parameters of these phylogenetic models where the dimensions of the parameters increase with the number of sequences N. HMC requires repeated calculation of the gradient of the data log-likelihood with respect to (wrt) all branch-length-specific (BLS) parameters that traditionally takes O(N2) operations using the standard pruning algorithm. A recent study proposes an approach to calculate this gradient in O(N), enabling researchers to take advantage of gradient-based samplers such as HMC. The CPU implementation of this approach makes the calculation of the gradient computationally tractable for nucleotide-based models but falls short in performance for larger state-space size models, such as Markov-modulated and codon models. Here, we describe novel massively parallel algorithms to calculate the gradient of the log-likelihood wrt all BLS parameters that take advantage of graphics processing units (GPUs) and result in many fold higher speedups over previous CPU implementations. We benchmark these GPU algorithms on three computing systems using three evolutionary inference examples exploring complete genomes from 997 dengue viruses, 62 carnivore mitochondria and 49 yeasts, and observe a >128-fold speedup over the CPU implementation for codon-based models and >8-fold speedup for nucleotide-based models. As a practical demonstration, we also estimate the timing of the first introduction of West Nile virus into the continental Unites States under a codon model with a relaxed molecular clock from 104 full viral genomes, an inference task previously intractable. We provide an implementation of our GPU algorithms in BEAGLE v4.0.0 (https://github.com/beagle-dev/beagle-lib), an open-source library for statistical phylogenetics that enables parallel calculations on multi-core CPUs and GPUs. We employ a BEAGLE-implementation using the Bayesian phylogenetics framework BEAST (https://github.com/beast-dev/beast-mcmc).

Accelerated Iterated Filtering

Simulation-based inferences have attracted much attention in recent years, as the direct computation of the likelihood function in many real-world problems is difficult or even impossible. Iterated filtering (Ionides, Bretó, and King 2006; Ionides, Bhadra, Atchadé,and King 2011) enables maximization of likelihood function via model perturbations and approximation of the gradient of loglikelihood through sequential Monte Carlo filtering. By an application of Stein’s identity, Doucet, Jacob, and Rubenthaler (2013) developed asecond-order approximation of the gradient of log-likelihood using sequential Monte Carlo smoothing. Based on these gradient approximations, we develop a new algorithm for maximizing the likelihood using the Nesterov accelerated gradient. We adopt the accelerated inexact gradient algorithm (Ghadimi and Lan 2016) to iterated filtering framework, relaxing the unbiased gradient approximation condition. We devise a perturbation policy for iterated filtering, allowing the new algorithm to converge at an optimal rate for both concave and non-concave log-likelihood functions. It is comparable to the recently developed Bayes map iterated filtering approach and outperforms the original iterated filtering approach.

Equilibrium and non-equilibrium regimes in the learning of restricted Boltzmann machines* *This article is an updated version of: Decelle A, Furtlehner C and Seoane B 2021 Equilibrium and non-equilibrium regimes in the learning of restricted Boltzmann machines Advances in Neural Information Processing Systems vol 34 ed M Ranzato, A Beygelzimer, Y Dauphin, P S Liang and J Wortman Vaughan (New

Training restricted Boltzmann machines (RBMs) have been challenging for a long time due to the difficulty of precisely computing the log-likelihood gradient. Over the past few decades, many works have proposed more or less successful training recipes but without studying the crucial quantity of the problem: the mixing time, i.e. the number of Monte Carlo iterations needed to sample new configurations from a model. In this work, we show that this mixing time plays a crucial role in the dynamics and stability of the trained model, and that RBMs operate in two well-defined regimes, namely equilibrium and out-of-equilibrium, depending on the interplay between this mixing time of the model and the number of steps, k, used to approximate the gradient. We further show empirically that this mixing time increases with the learning, which often implies a transition from one regime to another as soon as k becomes smaller than this time. In particular, we show that using the popular k (persistent) contrastive divergence approaches, with k small, the dynamics of the learned model are extremely slow and often dominated by strong out-of-equilibrium effects. On the contrary, RBMs trained in equilibrium display faster dynamics, and a smooth convergence to dataset-like configurations during the sampling. Finally, we discuss how to exploit in practice both regimes depending on the task one aims to fulfill: (i) short k can be used to generate convincing samples in short learning times, (ii) large k (or increasingly large) is needed to learn the correct equilibrium distribution of the RBM. Finally, the existence of these two operational regimes seems to be a general property of energy based models trained via likelihood maximization.

Post-hoc regularisation of unfolded cross-section measurements

Neutrino cross-section measurements are often presented as unfolded binned distributions in “true” variables. The ill-posedness of the unfolding problem can lead to results with strong anti-correlations and fluctuations between bins, which make comparisons to theoretical models in plots difficult. To alleviate this problem, one can introduce regularisation terms in the unfolding procedure. These suppress the anti-correlations in the result, at the cost of introducing some bias towards the expected shape of the data.This paper discusses a method using simple linear algebra, which makes it is possible to regularise any result that is presented as a central value and a covariance matrix. This “post-hoc” regularisation is generally much faster than repeating the unfolding method with different regularisation terms. The method also yields a regularisation matrix which connects the regularised to the unregularised result, and can be used to retain the full statistical power of the unregularised result when publishing a nicer looking regularised result.In addition to the regularisation method, this paper also presents some thoughts on the presentation of correlated data in general. When using the proposed method, the bias of the regularisation can be understood as a data visualisation problem rather than a statistical one. The strength of the regularisation can be chosen by minimising the difference between the implicitly uncorrelated distribution shown in the plots and the actual distribution described by the unregularised central value and covariance.Aside from minimising the difference between the shown and the actual result, additional information can be provided by showing the local log-likelihood gradient of the models shown in the plots. This adds more information about where the model is “pulled” by the data than just comparing the bin values to the data's central values.

Comparing the Effects of Boltzmann Machines as Associative Memory in Generative Adversarial Networks between Classical and Quantum Samplings

We investigate the quantum effect on machine learning (ML) models exemplified by the Generative Adversarial Network (GAN), which is a promising deep learning framework. In the general GAN framework the generator maps uniform noise to a fake image. In this study, we utilize the Associative Adversarial Network (AAN), which consists of a standard GAN and an associative memory. Further, we set a Boltzmann Machine (BM), which is an undirected graphical model that learns low-dimensional features extracted from a discriminator, as the memory. Owing to difficulty calculating the BM's log-likelihood gradient, it is necessary to approximate it by using the sample mean obtained from the BM, which has tentative parameters. To calculate the sample mean, a Markov Chain Monte Carlo (MCMC) is often used. In a previous study, this was performed using a quantum annealer device, and the performance of the "Quantum" AAN was compared to that of the standard GAN. However, its better performance than the standard GAN is not well understood. In this study, we introduce two methods to draw samples: classical sampling via MCMC and quantum sampling via quantum Monte Carlo (QMC) simulation, which is quantum simulation on the classical computer. Then, we compare these methods to investigate whether quantum sampling is advantageous. Specifically, calculating the discriminator loss, the generator loss, inception score and Fr\'echet inception distance, we discuss the possibility of AAN. We show that the AANs trained by both MCMC and QMC are more stable during training and produce more varied images than the standard GANs. However, the results indicate no difference in sampling by QMC simulation compared to that by MCMC.

BAYESIAN MITIGATION OF SPATIAL COARSENING FOR A HAWKES MODEL APPLIED TO GUNFIRE, WILDFIRE AND VIRAL CONTAGION.

Self-exciting spatiotemporal Hawkes processes have found increasing use in the study of large-scale public health threats, ranging from gun violence and earthquakes to wildfires and viral contagion. Whereas many such applications feature locational uncertainty, that is, the exact spatial positions of individual events are unknown, most Hawkes model analyses to date have ignored spatial coarsening present in the data. Three particular 21st century public health crises-urban gun violence, rural wildfires and global viral spread-present qualitatively and quantitatively varying uncertainty regimes that exhibit: (a) different collective magnitudes of spatial coarsening, (b) uniform and mixed magnitude coarsening, (c) differently shaped uncertainty regions and-less orthodox-(d) locational data distributed within the "wrong" effective space. We explicitly model such uncertainties in a Bayesian manner and jointly infer unknown locations together with all parameters of a reasonably flexible Hawkes model, obtaining results that are practically and statistically distinct from those obtained while ignoring spatial coarsening. This work also features two different secondary contributions: first, to facilitate Bayesian inference of locations and background rate parameters, we make a subtle yet crucial change to an established kernel-based rate model, and second, to facilitate the same Bayesian inference at scale, we develop a massively parallel implementation of the model's log-likelihood gradient with respect to locations and thus avoid its quadratic computational cost in the context of Hamiltonian Monte Carlo. Our examples involve thousands of observations and allow us to demonstrate practicality at moderate scales.

Restricted Boltzmann machine based on a Fermi sea

In recent years, there has been an intensive research on how to exploit the quantum laws of nature in the machine learning. Models have been put forward which employ spins, photons, and cold atoms. In this work we study the possibility of using the lattice fermions to learn the classical data. We propose an alternative to the quantum Boltzmann machine, the so-called spin-fermion machine (SFM), in which the spins represent the degrees of freedom of the observable data (to be learned), and the fermions represent the correlations between the data. The coupling is linear in spins and quadratic in fermions. The fermions are allowed to tunnel between the lattice sites. The training of SFM can be efficiently implemented since there are closed expressions for the log-likelihood gradient. We find that SFM is more powerful than the classical restricted Boltzmann machine with the same number of physical degrees of freedom. The reason is that SFM has additional freedom due to the rotation of the Fermi sea. We show examples for several data sets.

Efficient training of energy-based models via spin-glass control

We introduce a new family of energy-based probabilistic graphical models for efficient unsupervised learning. Its definition is motivated by the control of the spin-glass properties of the Ising model described by the weights of Boltzmann machines. We use it to learn the Bars and Stripes dataset of various sizes and the MNIST dataset, and show how they quickly achieve the performance offered by standard methods for unsupervised learning. Our results indicate that the standard initialization of Boltzmann machines with random weights equivalent to spin-glass models is an unnecessary bottleneck in the process of training. Furthermore, this new family allows for very easy access to low-energy configurations, which points to new, efficient training algorithms. The simplest variant of such algorithms approximates the negative phase of the log-likelihood gradient with no Markov chain Monte Carlo sampling costs at all, and with an accuracy sufficient to achieve good learning and generalization.

Sensitivity estimation for calculated phase equilibria

Abstract

A gradient approximation algorithm based weight momentum for restricted Boltzmann machine

Restricted Boltzmann Machine (RBM) is a powerful generative model in deep learning that can automatically conduct learning of data probability distributions without supervision. Deep architectures can effectively enhance the capability of image feature expression in image recognition. However, their learning time is long or they present poor performance given the same running time in a deep model. To address these problems, we propose a new gradient approximation algorithm called average contrastive divergence (ACD) with weight-decay momentum for training the RBM. It is an improved contrastive divergence (CD) algorithm combined with weight-decay momentum. Different combinations of the weight momentum term are added in the pre-training and fine-tuning phases of the RBM to accelerate the network convergence and improve the classification effect. Finally, the proposed algorithm is evaluated on the MNIST database, Extended Yale B and CMU-PIE face databases. The experimental results show that the proposed learning algorithm is a better approximation of the log-likelihood gradient method and outperforms the other algorithms.

Population-Contrastive-Divergence: Does consistency help with RBM training?

Estimating the log-likelihood gradient with respect to the parameters of a Restricted Boltzmann Machine (RBM) typically requires sampling using Markov Chain Monte Carlo (MCMC) techniques. To save computation time, the Markov chains are only run for a small number of steps, which leads to a biased estimate. This bias can cause RBM training algorithms such as Contrastive Divergence (CD) learning to deteriorate. We adopt the idea behind Population Monte Carlo (PMC) methods to devise a new RBM training algorithm termed Population-Contrastive-Divergence (pop-CD). Compared to CD, it leads to a consistent estimate and may have a significantly lower bias. Its computational overhead is negligible compared to CD, but the variance of the gradient estimate increases. We experimentally show that pop-CD can significantly outperform CD. In many cases, we observed a smaller bias and achieved higher log-likelihood values. However, when the RBM distribution has many hidden neurons, the consistent estimate of pop-CD may still have a considerable bias and the variance of the gradient estimate requires a smaller learning rate. Thus, despite its superior theoretical properties, it is not advisable to use pop-CD in its current form on large problems.

The role of different sampling methods in improving biological activity prediction using deep belief network.

Thousands of molecules and descriptors are available for a medicinal chemist thanks to the technological advancements in different branches of chemistry. This fact as well as the correlation between them has raised new problems in quantitative structure activity relationship studies. Proper parameter initialization in statistical modeling has merged as another challenge in recent years. Random selection of parameters leads to poor performance of deep neural network (DNN). In this research, deep belief network (DBN) was applied to initialize DNNs. DBN is composed of some stacks of restricted Boltzmann machine, an energy-based method that requires computing log likelihood gradient for all samples. Three different sampling approaches were suggested to solve this gradient. In this respect, the impact of DBN was applied based on the different sampling approaches mentioned above to initialize the DNN architecture in predicting biological activity of all fifteen Kaggle targets that contain more than 70k molecules. The same as other fields of processing research, the outputs of these models demonstrated significant superiority to that of DNN with random parameters. © 2016 Wiley Periodicals, Inc.

Average Contrastive Divergence for Training Restricted Boltzmann Machines

This paper studies contrastive divergence (CD) learning algorithm and proposes a new algorithm for training restricted Boltzmann machines (RBMs). We derive that CD is a biased estimator of the log-likelihood gradient method and make an analysis of the bias. Meanwhile, we propose a new learning algorithm called average contrastive divergence (ACD) for training RBMs. It is an improved CD algorithm, and it is different from the traditional CD algorithm. Finally, we obtain some experimental results. The results show that the new algorithm is a better approximation of the log-likelihood gradient method and outperforms the traditional CD algorithm.

Using SAD data inPhaser

Phaser is a program that implements likelihood-based methods to solve macromolecular crystal structures, currently by molecular replacement or single-wavelength anomalous diffraction (SAD). SAD phasing is based on a likelihood target derived from the joint probability distribution of observed and calculated pairs of Friedel-related structure factors. This target combines information from the total structure factor (primarily non-anomalous scattering) and the difference between the Friedel mates (anomalous scattering). Phasing starts from a substructure, which is usually but not necessarily a set of anomalous scatterers. The substructure can also be a protein model, such as one obtained by molecular replacement. Additional atoms are found using a log-likelihood gradient map, which shows the sites where the addition of scattering from a particular atom type would improve the likelihood score. An automated completion algorithm adds new sites, choosing optionally among different atom types, adds anisotropic B-factor parameters if appropriate and deletes atoms that refine to low occupancy. Log-likelihood gradient maps can also identify which atoms in a refined protein structure are anomalous scatterers, such as metal or halide ions. These maps are more sensitive than conventional model-phased anomalous difference Fouriers and the iterative completion algorithm is able to find a significantly larger number of convincing sites.

Bounding the Bias of Contrastive Divergence Learning

Optimization based on k-step contrastive divergence (CD) has become a common way to train restricted Boltzmann machines (RBMs). The k-step CD is a biased estimator of the log-likelihood gradient relying on Gibbs sampling. We derive a new upper bound for this bias. Its magnitude depends on k, the number of variables in the RBM, and the maximum change in energy that can be produced by changing a single variable. The last reflects the dependence on the absolute values of the RBM parameters. The magnitude of the bias is also affected by the distance in variation between the modeled distribution and the starting distribution of the Gibbs chain.

Experimental phasing: best practice and pitfalls

Developments in protein crystal structure determination by experimental phasing are reviewed, emphasizing the theoretical continuum between experimental phasing, density modification, model building and refinement. Traditional notions of the composition of the substructure and the best coefficients for map generation are discussed. Pitfalls such as determining the enantiomorph, identifying centrosymmetry (or pseudo-symmetry) in the substructure and crystal twinning are discussed in detail. An appendix introduces combined real-imaginary log-likelihood gradient map coefficients for SAD phasing and their use for substructure completion as implemented in the software Phaser. Supplementary material includes animated probabilistic Harker diagrams showing how maximum-likelihood-based phasing methods can be used to refine parameters in the case of SIR and MIR; it is hoped that these will be useful for those teaching best practice in experimental phasing methods.

Justifying and Generalizing Contrastive Divergence

We study an expansion of the log likelihood in undirected graphical models such as the restricted Boltzmann machine (RBM), where each term in the expansion is associated with a sample in a Gibbs chain alternating between two random variables (the visible vector and the hidden vector in RBMs). We are particularly interested in estimators of the gradient of the log likelihood obtained through this expansion. We show that its residual term converges to zero, justifying the use of a truncation--running only a short Gibbs chain, which is the main idea behind the contrastive divergence (CD) estimator of the log-likelihood gradient. By truncating even more, we obtain a stochastic reconstruction error, related through a mean-field approximation to the reconstruction error often used to train autoassociators and stacked autoassociators. The derivation is not specific to the particular parametric forms used in RBMs and requires only convergence of the Gibbs chain. We present theoretical and empirical evidence linking the number of Gibbs steps k and the magnitude of the RBM parameters to the bias in the CD estimator. These experiments also suggest that the sign of the CD estimator is correct most of the time, even when the bias is large, so that CD-k is a good descent direction even for small k.

Likelihood Gradient Evaluation Using Square-Root Covariance Filters

Using the array form of numerically stable square-root implementation methods for Kalman filtering formulas, we construct a new square-root algorithm for the log-likelihood gradient (score) evaluation. This avoids the use of the conventional Kalman filter with its inherent numerical instabilities and improves the robustness of computations against roundoff errors. The new algorithm is developed in terms of covariance quantities and based on the ldquocondensed formrdquo of the array square-root filter.

Contrastive Divergence Learning May Diverge When Training Restricted Boltzmann Machines

Event Abstract Back to Event Contrastive Divergence Learning May Diverge When Training Restricted Boltzmann Machines Asja Fischer1, 2 and Christian Igel1, 2* 1 Ruhr-Universität Bochum, Bernstein Center for Computational Neuroscience, Germany 2 Ruhr-Universität Bochum, Institut für Neuroinformatik, Germany Understanding and modeling how brains learn higher-level representations from sensory input is one of the key challenges in computational neuroscience and machine learning. Layered generative models such as deep belief networks (DBNs) are promising for unsupervised learning such representations, and new algorithms that operate in a layer-wise fashion make learning these models computationally tractable [1-5]. Restricted Boltzmann Machines (RBMs) are the typical building blocks for DBN layers. They are undirected graphical models, and their structure is a bipartite graph connecting input (visible) and hidden neurons. Training large undirected graphical models by likelihood maximization in general involves averages over an exponential number of terms, and obtaining unbiased estimates of these averages by Markov chain Monte Carlo methods typically requires many sampling steps. However, recently it was shown that estimates obtained after running the chain for just a few steps can be sufficient for model training [3]. In particular, gradient-ascent on the k-step Contrastive Divergence (CD-k), which is a biased estimator of the log-likelihood gradient based on k steps of Gibbs sampling, has become the most common way to train RBMs [1-5]. Contrastive Divergence learning does not necessarily reach the maximum likelihood estimate of the parameters (e.g., because of the bias). However, we show that the situation is much worse. We demonstrate empirically that for some benchmark problems taken from the literature [6], CD learning systematically leads to a steady decrease of the log-likelihood after an initial increase (see supplementary Figure 1). This seems to happen especially when trying to learn more complex distributions, which are the targets if RBMs are used within DBNs. The reason for the decreasing log-likelihood is an increase of the model parameter magnitudes. The estimation bias depends on the mixing rate of the Markov chain, and it is well-known that mixing slows down with growing magnitude of model parameters [1,3]. Weight-decay can therefore solve the problem if the strength of the regularization term is adjusted correctly. If chosen too large, learning is not accurate enough. If chosen too small, learning still divergences. For large k, the effect is less pronounced. Increasing k, as suggested in [1] for finding parameters with higher likelihood, may therefore prevent divergence. However, divergence occurs even for values of k too large to be computationally tractable for large models. Thus, a dynamic schedule to control k is needed.

Exact filters for Newton–Raphson parameter estimation algorithms for continuous-time partially observed stochastic systems

This paper presents explicit finite-dimensional filters for implementing Newton–Raphson (NR) parameter estimation algorithms. The models which exhibit nonlinear parameter dependence are stochastic, continuous-time and partially observed. The implementation of the NR algorithm requires evaluation of the log-likelihood gradient and the Fisher information matrix. Fisher information matrices are important in bounding the estimation error from below, via the Cramer–Rao bound. The derivations are based on relations between incomplete and complete data, likelihood, gradient and Hessian likelihood functions, which are derived using Girsanov's measure transformations.