Mean-field Spin Glass Research Articles

Unlearnable Games and “Satisficing” Decisions: A Simple Model for a Complex World

As a schematic model of the complexity economic agents are confronted with, we introduce the “Sherrington-Kirkpatrick game,” a discrete time binary choice model inspired from mean-field spin glasses. We show that, even in a completely static environment, agents are unable to learn collectively optimal strategies. This is either because the learning process gets trapped in a suboptimal fixed point or because learning never converges and leads to a never-ending evolution of agent intentions. Contrarily to the hope that learning might save the standard “rational expectation” framework in economics, we argue that complex situations are generically and agents must do with solutions, as argued long ago by Simon []. Only a centralized, omniscient agent endowed with enormous computing power could qualify to determine the optimal strategy of all agents. Using a mix of analytical arguments and numerical simulations, we find that (i) long memory of past rewards is beneficial to learning, whereas overreaction to recent past is detrimental and leads to cycles or chaos; (ii) increased competition (nonreciprocity) destabilizes fixed points and leads first to chaos and, in the high competition limit, to quasicycles; (iii) some amount of randomness in the learning process, perhaps paradoxically, allows the system to reach better collective decisions; (iv) nonstationary, “aging” behavior spontaneously emerges in a large swath of parameter space of our complex but static world. On the positive side, we find that the learning process allows cooperative systems to coordinate around satisficing solutions with rather high (but markedly suboptimal) average reward. However, hypersensitivity to the game parameters makes it impossible to predict who will be better or worse off in our stylized economy. The statistical description of the space of satisficing solutions is an open problem. Published by the American Physical Society 2024

NoRA: A Tensor Network Ansatz for Volume-Law Entangled Equilibrium States of Highly Connected Hamiltonians

Motivated by the ground state structure of quantum models with all-to-all interactions such as mean-field quantum spin glass models and the Sachdev-Ye-Kitaev (SYK) model, we propose a tensor network architecture which can accomodate volume law entanglement and a large ground state degeneracy. We call this architecture the non-local renormalization ansatz (NoRA) because it can be viewed as a generalization of MERA, DMERA, and branching MERA networks with the constraints of spatial locality removed. We argue that the architecture is potentially expressive enough to capture the entanglement and complexity of the ground space of the SYK model, thus making it a suitable variational ansatz, but we leave a detailed study of SYK to future work. We further explore the architecture in the special case in which the tensors are random Clifford gates. Here the architecture can be viewed as the encoding map of a random stabilizer code. We introduce a family of codes inspired by the SYK model which can be chosen to have constant rate and linear distance at the cost of some high weight stabilizers. We also comment on potential similarities between this code family and the approximate code formed from the SYK ground space.

Entropy and dimension spectrum of the mean-field spin glass model

A self-contained new constructive solution of the mean-field Sherrington-Kirkpatric spin-glass model is obtained from the study of the behaviour of the entropy of the Gibbs measure at low temperatures.

Optimizing mean field spin glasses with external field

Optimizing mean field spin glasses with external field

On weak ergodicity breaking in mean-field spin glasses

The weak ergodicity breaking hypothesis postulates that out-of-equilibrium glassy systems lose memory of their initial state despite being unable to reach an equilibrium stationary state. It is a milestone of glass physics, and has provided a lot of insight on the physical properties of glass aging. Despite its undoubted usefulness as a guiding principle, its general validity remains a subject of debate. Here, we present evidence that this hypothesis does not hold for a class of mean-field spin glass models. While most of the qualitative physical picture of aging remains unaffected, our results suggest that some important technical aspects should be revisited.

Ultrametric identities in glassy models of natural evolution

Spin-glasses constitute a well-grounded framework for evolutionary models. Of particular interest for (some of) these models is the lack of self-averaging of their order parameters (e.g. the Hamming distance between the genomes of two individuals), even in asymptotic limits, much as like what happens to the overlap between the configurations of two replica in mean-field spin-glasses. In the latter, this lack of self-averaging is related to a peculiar behavior of the overlap fluctuations, as described by the Ghirlanda–Guerra identities and by the Aizenman–Contucci polynomials, that cover a pivotal role in describing the ultrametric structure of the spin-glass landscape. As for evolutionary models, such identities may therefore be related to a taxonomic classification of individuals, yet a full investigation on their validity is missing. In this paper, we study ultrametric identities in simple cases where solely random mutations take place, while selective pressure is absent, namely in flat landscape models. In particular, we study three paradigmatic models in this setting: the one parent model (which, by construction, is ultrametric at the level of single individuals), the homogeneous population model (which is replica symmetric), and the species formation model (where a broken-replica scenario emerges at the level of species). We find analytical and numerical evidence that in the first and in the third model nor the Ghirlanda–Guerra neither the Aizenman–Contucci constraints hold, rather a new class of ultrametric identities is satisfied; in the second model all these constraints hold trivially. Very preliminary results on a real biological human genome derived by The 1000 Genome Project Consortium and on two artificial human genomes (generated by two different types neural networks) seem in better agreement with these new identities rather than the classic ones.

Free energy upper bound for mean-field vector spin glasses

Free energy upper bound for mean-field vector spin glasses

On the REM approximation of TAP free energies

The free energy of TAP-solutions for the SK-model of mean field spin glasses can be expressed as a nonlinear functional of local terms: we exploit this feature in order to contrive abstract REM-like models which we then solve by a classical large deviations treatment. This allows to identify the origin of the physically unsettling quadratic (in the inverse of temperature) correction to the Parisi free energy for the SK-model, and formalizes the true cavity dynamics which acts on TAP-space, i.e. on the space of TAP-solutions. From a non-spin glass point of view, this work is the first in a series of refinements which addresses the stability of hierarchical structures in models of evolving populations.

Mean Field Spin Glass Models Under Weak External Field

We study the fluctuation and limiting distribution of free energy in mean-field spin glass models with Ising spins under weak external fields. We prove that at high temperature, there are three sub-regimes concerning the strength of external field $h \approx \rho N^{-\alpha}$ with $\rho,\alpha\in (0,\infty)$. In the super-critical regime $\alpha < 1/4$, the variance of the log-partition function is $\approx N^{1-4\alpha}$. In the critical regime $\alpha = 1/4$, the fluctuation is of constant order but depends on $\rho$. Whereas, in the sub-critical regime $\alpha>1/4$, the variance is $\Theta(1)$ and does not depend on $\rho$. We explicitly express the asymptotic mean and variance in all three regimes and prove Gaussian central limit theorems. Our proofs mainly follow two approaches. One utilizes quadratic coupling and Guerra's interpolation scheme for Gaussian disorder, extending to many other spin glass models. However, this approach can prove the CLT only at very high temperatures. The other one is a cluster-based approach for general symmetric disorders, first used in the seminal work of Aizenman, Lebowitz, and Ruelle~(Comm.~Math.~Phys.~112 (1987), no.~1, 3--20) for the zero external field case. It was believed that this approach does not work if the external field is present. We show that if the external field is present but not too strong, it still works with a new cluster structure. In particular, we prove the CLT up to the critical temperature in the Sherrington-Kirkpatrick (SK) model when $\alpha \ge 1/4$. We further address the generality of this cluster-based approach. Specifically, we give similar results for the multi-species SK model and diluted SK model.

Free energy of a diluted spin glass model with quadratic Hamiltonian

We study a diluted mean-field spin glass model with a quadratic Hamiltonian. Our main result establishes the limiting free energy in terms of an integral of a family of random variables that are the weak limits of the quenched variances of the spins in the system with varying edge connectivity. The key ingredient in our argument is played by the identification of these random variables as the unique solution to a recursive distributional equation. Our results in particular provide the first example of the diluted Shcherbina–Tirozzi model, whose limiting free energy can be derived at any inverse temperature and external field.

Generalized random energy models in a transversal magnetic field: Free energy and phase diagrams

We determine explicit variational expressions for the free energy of mean-field spin glasses in a transversal magnetic field, whose glass interaction is given by a hierarchical Gaussian potential as in Derrida's Generalized Random Energy Model (GREM), its continuous version (CREM) or the non-hierarchical GREM. The corresponding phase diagrams, which generally include glass transitions as well as magnetic transitions, are discussed. In the glass phase, the free energy is generally determined by both the parameters of the classical model and the transversal field.

Crisanti–Sommers Formula and Simultaneous Symmetry Breaking in Multi-species Spherical Spin Glasses

There is a rich history of expressing the limiting free energy of mean-field spin glasses as a variational formula over probability measures on $[0,1]$, where the measure represents the similarity (or "overlap") of two independently sampled spin configurations. At high temperatures, the formula's minimum is achieved at a measure which is a point mass, meaning sample configurations are asymptotically orthogonal up to a magnetic field correction. At low temperatures, though, a very different behavior emerges known as replica symmetry breaking (RSB). The deep wells in the energy landscape create more rigid structure, and the optimal overlap measure is no longer a point mass. The exact size of its support remains in many cases an open problem. Here we consider these themes for multi-species spherical spin glasses. Following a companion work in which we establish the Parisi variational formula, here we present this formula's Crisanti-Sommers representation. In the process, we gain new access to a problem unique to the multi-species setting. Namely, if RSB occurs for one species, does it necessarily occur for other species as well? We provide sufficient conditions for the answer to be yes. For instance, we show that if two species share any quadratic interaction, then RSB for one implies RSB for the other. Moreover, the level of symmetry breaking must be identical, even in cases of full RSB. In the presence of an external field, any type of interaction suffices.

Marginal stability of soft anharmonic mean field spin glasses

We investigate the properties of the glass phase of a recently introduced spin glass model of soft spins subjected to an anharmonic quartic local potential, which serves as a model of low temperature molecular or soft glasses. We solve the model using mean field theory and show that, at low temperatures, it is described by full replica symmetry breaking. As a consequence, at zero temperature the glass phase is marginally stable. We show that in this case, marginal stability comes from a combination of both soft linear excitations—appearing in a gapless spectrum of the Hessian of linear excitations—and pseudogapped non-linear excitations—corresponding to nearly degenerate two level systems. Therefore, this model is a natural candidate to describe what happens in soft glasses, where quasi localized soft modes in the density of states appear together with non-linear modes triggering avalanches and conjectured to be essential to describe the universal low temperature anomalies of glasses.

On ℓp-Gaussian–Grothendieck Problem

Abstract For $p\geq 1$ and $(g_{ij})_{1\leq i,j\leq n}$ being a matrix of i.i.d. standard Gaussian entries, we study the $n$-limit of the $\ell _p$-Gaussian–Grothendieck problem defined as $$\begin{align*} &amp; \max\Bigl\{\sum_{i,j=1}^n g_{ij}x_ix_j: x\in \mathbb{R}^n,\sum_{i=1}^n |x_i|^p=1\Bigr\}. \end{align*}$$The case $p=2$ corresponds to the top eigenvalue of the Gaussian orthogonal ensemble; when $p=\infty $, the maximum value is essentially the ground state energy of the Sherrington–Kirkpatrick mean-field spin glass model and its limit can be expressed by the famous Parisi formula. In the present work, we focus on the cases $1\leq p&amp;lt;2$ and $2&amp;lt;p&amp;lt;\infty .$ For the former, we compute the limit of the $\ell _p$-Gaussian–Grothendieck problem and investigate the structure of the set of all near optimizers along with stability estimates. In the latter case, we show that this problem admits a Parisi-type variational representation and the corresponding optimizer is weakly delocalized in the sense that its entries vanish uniformly in a polynomial order of $n^{-1}$.

Optimization of mean-field spin glasses

Mean-field spin glasses are families of random energy functions (Hamiltonians) on high-dimensional product spaces. In this paper, we consider the case of Ising mixed p-spin models,; namely, Hamiltonians HN:ΣN→R on the Hamming hypercube ΣN={±1}N, which are defined by the property that {HN(σ)}σ∈ΣN is a centered Gaussian process with covariance E{HN(σ1)HN(σ2)} depending only on the scalar product ⟨σ1,σ2⟩. The asymptotic value of the optimum maxσ∈ΣNHN(σ) was characterized in terms of a variational principle known as the Parisi formula, first proved by Talagrand and, in a more general setting, by Panchenko. The structure of superlevel sets is extremely rich and has been studied by a number of authors. Here, we ask whether a near optimal configuration σ can be computed in polynomial time. We develop a message passing algorithm whose complexity per-iteration is of the same order as the complexity of evaluating the gradient of HN, and characterize the typical energy value it achieves. When the p-spin model HN satisfies a certain no-overlap gap assumption, for any ε>0, the algorithm outputs σ∈ΣN such that HN(σ)≥(1−ε)maxσ′HN(σ′), with high probability. The number of iterations is bounded in N and depends uniquely on ε. More generally, regardless of whether the no-overlap gap assumption holds, the energy achieved is given by an extended variational principle, which generalizes the Parisi formula.

A Statistical Physics approach to a multi-channel Wigner spiked model

In this letter we present a finite temperature approach to a high-dimensional inference problem, the Wigner spiked model, with group-dependent signal-to-noise ratios. For two classes of convex and non-convex network architectures the error in the reconstruction is described in terms of the solution of a mean-field spin-glass on the Nishimori line. In the cases studied the order parameters do not fluctuate and are the solution of finite dimensional variational problems. The deep architecture is optimized in order to confine the high temperature phase where reconstruction fails.

Thouless–Anderson–Palmer Equations for the Ghatak–Sherrington Mean Field Spin Glass Model

We derive the Thouless-Anderson-Palmer (TAP) equations for the Ghatak and Sherrington model. Our derivation, based on the cavity method, holds at high temperature and at all values of the crystal field. It confirms the prediction of Yokota.

Phase transition in random tensors with multiple independent spikes

Consider a spiked random tensor obtained as a mixture of two components: noise in the form of a symmetric Gaussian p-tensor for p≥3 and signal in the form of a symmetric low-rank random tensor. The latter is defined as a linear combination of k independent symmetric rank-one random tensors, referred to as spikes, with weights referred to as signal-to-noise ratios (SNRs). The entries of the vectors that determine the spikes are i.i.d. sampled from general probability distributions supported on bounded subsets of R. This work focuses on the problem of detecting the presence of these spikes, and establishes the phase transition of this detection problem for any fixed k≥1. In particular, it shows that for a set of relatively low SNRs it is impossible to distinguish between the spiked and nonspiked Gaussian tensors. Furthermore, in the interior of the complement of this set, where at least one of the k SNRs is relatively high, these two tensors are distinguishable by the likelihood ratio test. In addition, when the total number of low-rank components, k, of the p-tensor of size N grows in the order o(N(p−2)/4) as N tends to infinity, the problem exhibits an analogous phase transition. This theory for spike detection is also shown to imply that recovery of the spikes by the minimum mean square error exhibits the same phase transition. The main methods used in this work arise from the study of mean field spin glass models, where the phase transition thresholds are identified as the critical inverse temperatures distinguishing the high and low-temperature regimes of the free energies. In particular, our result formulates the first full characterization of the high temperature regime for vector-valued spin glass models with independent coordinates.

Nonconvex interactions in mean-field spin glasses

We propose a conjecture for the limit of mean-field spin glasses with a bipartite structure, and show that the conjectured limit is an upper bound. The conjectured limit is described in terms of the solution of an infinite-dimensional Hamilton-Jacobi equation. A fundamental difficulty of the problem is that the nonlinearity in this equation is not convex. We also question the possibility to characterize this conjectured limit in terms of a saddle-point problem.

Nonconvex interactions in mean-field spin glasses

Nonconvex interactions in mean-field spin glasses