Inverse Ising Problem Research Articles

Data quality for the inverse lsing problem

There are many methods proposed for inferring parameters of the Ising model from given data, that is a set of configurations generated according to the model itself. However little attention has been paid until now to the data, e.g. how the data is generated, whether the inference error using one set of data could be smaller than using another set of data, etc. In this paper we discuss the data quality problem in the inverse Ising problem, using as a benchmark the kinetic Ising model. We quantify the quality of data using effective rank of the correlation matrix, and show that data gathered in a out-of-equilibrium regime has a better quality than data gathered in equilibrium for coupling reconstruction. We also propose a matrix-perturbation based method for tuning the quality of given data and for removing bad-quality (i.e. redundant) configurations from data.

Solving the inverse Ising problem by mean-field methods in a clustered phase space with many states.

In this work we explain how to properly use mean-field methods to solve the inverse Ising problem when the phase space is clustered, that is, many states are present. The clustering of the phase space can occur for many reasons, e.g., when a system undergoes a phase transition, but also when data are collected in different regimes (e.g., quiescent and spiking regimes in neural networks). Mean-field methods for the inverse Ising problem are typically used without taking into account the eventual clustered structure of the input configurations and may lead to very poor inference (e.g., in the low-temperature phase of the Curie-Weiss model). In this work we explain how to modify mean-field approaches when the phase space is clustered and we illustrate the effectiveness of our method on different clustered structures (low-temperature phases of Curie-Weiss and Hopfield models).

Inference of the sparse kinetic Ising model using the decimation method.

In this paper we study the inference of the kinetic Ising model on sparse graphs by the decimation method. The decimation method, which was first proposed in Decelle and Ricci-Tersenghi [Phys. Rev. Lett. 112, 070603 (2014)] for the static inverse Ising problem, tries to recover the topology of the inferred system by setting the weakest couplings to zero iteratively. During the decimation process the likelihood function is maximized over the remaining couplings. Unlike the ℓ(1)-optimization-based methods, the decimation method does not use the Laplace distribution as a heuristic choice of prior to select a sparse solution. In our case, the whole process can be done auto-matically without fixing any parameters by hand. We show that in the dynamical inference problem, where the task is to reconstruct the couplings of an Ising model given the data, the decimation process can be applied naturally into a maximum-likelihood optimization algorithm, as opposed to the static case where pseudolikelihood method needs to be adopted. We also use extensive numerical studies to validate the accuracy of our methods in dynamical inference problems. Our results illustrate that, on various topologies and with different distribution of couplings, the decimation method outperforms the widely used ℓ(1)-optimization-based methods.

Simple method for inference in inverse Ising problem using full data

We consider inference in the inverse Ising problem using full data, which means incorporating sets of spin configurations. We approximate the Boltzmann distribution of the system to generate a frequency distribution derived from the given data. Then, the ratio between two Boltzmann distributions with different spin configurations eliminates the partition function and we obtain linear equations which can be solved to yield statistical parameters. Our method is applicable to cases where the absolute values of the coupling parameters and external fields are large. Compared to pseudolikelihood maximization, the accuracy of the inference obtained from our method is similar, although our approach is less labor intensive.

Monte Carlo Integration Using Spatial Structure of Markov Random Field

Monte Carlo integration (MCI) techniques are important in various fields. In this study, a new MCI technique for Markov random fields (MRFs) is proposed. MCI consists of two successive parts: the first involves sampling using a technique such as the Markov chain Monte Carlo method, and the second involves an averaging operation using the obtained sample points. In the averaging operation, a simple sample averaging technique is often employed. The method proposed in this paper improves the averaging operation by addressing the spatial structure of the MRF and is mathematically guaranteed to statistically outperform standard MCI using the simple sample averaging operation. Moreover, the proposed method can be improved in a systematic manner and is numerically verified by numerical simulations using planar Ising models. In the latter part of this paper, the proposed method is applied to the inverse Ising problem and we observe that it outperforms the maximum pseudo-likelihood estimation.

Inverse Ising inference with correlated samples

Correlations between two variables of a high-dimensional system can be indicative of an underlying interaction, but can also result from indirect effects. Inverse Ising inference is a method to distinguish one from the other. Essentially, the parameters of the least constrained statistical model are learned from the observed correlations such that direct interactions can be separated from indirect correlations. Among many other applications, this approach has been helpful for protein structure prediction, because residues which interact in the 3D structure often show correlated substitutions in a multiple sequence alignment. In this context, samples used for inference are not independent but share an evolutionary history on a phylogenetic tree. Here, we discuss the effects of correlations between samples on global inference. Such correlations could arise due to phylogeny but also via other slow dynamical processes. We present a simple analytical model to address the resulting inference biases, and develop an exact method accounting for background correlations in alignment data by combining phylogenetic modeling with an adaptive cluster expansion algorithm. We find that popular reweighting schemes are only marginally effective at removing phylogenetic bias, suggest a rescaling strategy that yields better results, and provide evidence that our conclusions carry over to the frequently used mean-field approach to the inverse Ising problem.

Estimation of quenched random fields in the inverse Ising problem using a diagonal matching method.

We consider a method for the accurate estimation of quenched random fields in the inverse Ising problem. Approximations such as the mean-field or Bethe methods are applied to estimate quenched random coupling parameters and external fields. A diagonal matching method is introduced to ensure consistency of the diagonal part of the susceptibility, and the method yields an accurate estimation of the external fields. We introduce the diagonal matching method into the mean-field, Thouless-Anderson-Palmer, and Bethe approximations, and we investigate the effect of the diagonal matching method on the accuracy of estimation of the external fields.

Bayesian approach to inverse statistical mechanics.

Inverse statistical mechanics aims to determine particle interactions from ensemble properties. This article looks at this inverse problem from a Bayesian perspective and discusses several statistical estimators to solve it. In addition, a sequential Monte Carlo algorithm is proposed that draws the interaction parameters from their posterior probability distribution. The posterior probability involves an intractable partition function that is estimated along with the interactions. The method is illustrated for inverse problems of varying complexity, including the estimation of a temperature, the inverse Ising problem, maximum entropy fitting, and the reconstruction of molecular interaction potentials.

The Inverse Ising Problem

Abstract The inverse Ising problem consists of taking a set of Ising configurations generated with unknown interaction parameters, and deter- mining reliable estimates for the values of those interaction parameters. The problem first arose in connection with the Monte Carlo renormalization group, and was solved thirty years ago. Recently, there has been renewed interest in the inverse Ising problem due to biological applications. The original solution seems to have been forgotten, as it was rediscovered in a different representation by Aurell and Ekeberg in 2012. In this paper we modify the earlier equations to solve problems that are not translationally invariant.

Approximate inverse Ising models close to a Bethe reference point

We investigate different ways of generating approximate solutions to the inverse Ising problem (IIP). Our approach consists in taking as a starting point for further perturbation procedures, a Bethe mean-field solution obtained with a maximum spanning tree (MST) of pairwise mutual information, which we refer to as the Bethe reference point. We consider three different ways of following this idea: in the first one, we discuss a greedy procedure by which optimal links to be added starting from the Bethe reference point are selected and calibrated iteratively; the second one is based on the observation that the natural gradient can be computed analytically at the Bethe point; the last one deals with loop corrections to the Bethe point. Assuming no external field and using a dual transform we develop a dual loop joint model based on a well-chosen cycle basis. This leads us to identify a subclass of planar models, which we refer to as dual-loop-free models, having possibly many loops, but characterized by a singly connected dual factor graph, for which the partition function and the linear response can be computed exactly in respectively O(N) and O(N2) operations, thanks to a dual weight propagation (DWP) message passing procedure that we set up. When restricted to this subclass of models, the inverse Ising problem being convex, becomes tractable at any temperature. Numerical experiments show that this can serve to some extent as a good approximation for models with dual loops.

Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields

The adaptive Thouless-Anderson-Palmer equation is derived for inverse Ising problems in the presence of quenched random fields. We test the proposed scheme on Sherrington-Kirkpatrick, Hopfield, and random orthogonal models and find that the adaptive Thouless-Anderson-Palmer approach allows accurate inference of quenched random fields whose distribution can be either Gaussian or bimodal. In particular, another competitive method for inferring external fields, namely, the naive mean field method with diagonal weights, is compared and discussed.

Mean-field method with correlations determined by linear response

We introduce a mean-field approximation based on the reconciliation of maximum entropy and linear response for correlations in the cluster variation method. Within a general formalism that includes previous mean-field methods, we derive formulas improving on, e.g., the Bethe approximation and the Sessak-Monasson result at high temperature. Applying the method to direct and inverse Ising problems, we find improvements over standard implementations.

Beyond Inverse Ising Model: Structure of the Analytical Solution

I consider the problem of deriving couplings of a statistical model from measured correlations, a task which generalizes the well-known inverse Ising problem. After reminding that such problem can be mapped on the one of expressing the entropy of a system as a function of its corresponding observables, I show the conditions under which this can be done without resorting to iterative algorithms. I find that inverse problems are local (the inverse Fisher information is sparse) whenever the corresponding models have a factorized form, and the entropy can be split in a sum of small cluster contributions. I illustrate these ideas through two examples (the Ising model on a tree and the one-dimensional periodic chain with arbitrary order interaction) and support the results with numerical simulations. The extension of these methods to more general scenarios is finally discussed.

Susceptibility propagation by using diagonal consistency

A susceptibility propagation that is constructed by combining a belief propagation and a linear response method is used for approximate computation for Markov random fields. Herein, we formulate an improved susceptibility propagation by using the concept of a diagonal matching method that is based on mean-field approaches to inverse Ising problems. The proposed susceptibility propagation is robust for various network structures, and it is reduced to the ordinary susceptibility propagation and to the adaptive Thouless-Anderson-Palmer equation in special cases.

Mean-Field Theory for the Inverse Ising Problem at Low Temperatures

The large amounts of data from molecular biology and neuroscience have lead to a renewed interest in the inverse Ising problem: how to reconstruct parameters of the Ising model (couplings between spins and external fields) from a number of spin configurations sampled from the Boltzmann measure. To invert the relationship between model parameters and observables (magnetizations and correlations), mean-field approximations are often used, allowing the determination of model parameters from data. However, all known mean-field methods fail at low temperatures with the emergence of multiple thermodynamic states. Here, we show how clustering spin configurations can approximate these thermodynamic states and how mean-field methods applied to thermodynamic states allow an efficient reconstruction of Ising models also at low temperatures.

The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods

The inverse Ising problem consists in inferring the coupling constants of an Ising model given the correlation matrix. The fastest methods for solving this problem are based on mean-field approximations, but which one performs better in the general case is still not completely clear. In the first part of this work, I summarize the formulas for several mean-field approximations and I derive new analytical expressions for the Bethe approximation, which allow one to solve the inverse Ising problem without running the susceptibility propagation algorithm (thus avoiding the lack of convergence). In the second part, I compare the accuracy of different mean-field approximations on several models (diluted ferromagnets and spin glasses) defined on random graphs and regular lattices, showing which one is in general more effective. A simple improvement over these approximations is proposed. Also a fundamental limitation is found in using methods based on TAP and Bethe approximations in the presence of an external field.

Adaptive Cluster Expansion for the Inverse Ising Problem: Convergence, Algorithm and Tests

We present a procedure to solve the inverse Ising problem, that is, to find the interactions between a set of binary variables from the measure of their equilibrium correlations. The method consists in constructing and selecting specific clusters of spins, based on their contributions to the cross-entropy of the Ising model. Small contributions are discarded to avoid overfitting and to make the computation tractable. The properties of the cluster expansion and its performances on synthetic data are studied. To make the implementation easier we give the pseudo-code of the algorithm.

Bethe–Peierls approximation and the inverse Ising problem

We apply the Bethe–Peierls approximation to the inverse Ising problem and show how thelinear response relation leads to a simple method for reconstructing couplings and fields ofthe Ising model. This reconstruction is exact on tree graphs, yet its computational expenseis comparable to those of other mean-field methods. We compare the performance of thismethod to the independent-pair, naive mean-field, and Thouless–Anderson–Palmerapproximations, the Sessak–Monasson expansion, and susceptibility propagation on theCayley tree, SK model and random graph with fixed connectivity. At low temperatures,Bethe reconstruction outperforms all of these methods, while at high temperatures it iscomparable to the best method available so far (the Sessak–Monasson method).The relationship between Bethe reconstruction and other mean-field methods isdiscussed.

Inverse Ising Inference Using All the Data

We show that a method based on logistic regression, using all the data, solves the inverse Ising problem far better than mean-field calculations relying only on sample pairwise correlation functions, while still computationally feasible for hundreds of nodes. The largest improvement in reconstruction occurs for strong interactions. Using two examples, a diluted Sherrington-Kirkpatrick model and a two-dimensional lattice, we also show that interaction topologies can be recovered from few samples with good accuracy and that the use of l(1) regularization is beneficial in this process, pushing inference abilities further into low-temperature regimes.

Counting solutions from finite samplings

We formulate the solution counting problem within the framework of the inverse Ising problem and use fast belief propagation equations to estimate the entropy whose value provides an estimate of the true one. We test this idea on both diluted models [random 2-SAT (2-satisfiability) and 3-SAT problems] and a fully connected model (binary perceptron), and show that when the constraint density is small, this estimate can be very close to the true value. The information stored by the salamander retina under the natural movie stimuli can also be estimated, and our result is consistent with that obtained by the Monte Carlo method. Of particular significance is that the sizes of other metastable states for this real neuronal network are predicted.