One-step Replica Symmetry Breaking Research Articles

One-Step Replica Symmetry Breaking of Random Regular NAE-SAT II

Continuing our earlier work in Nam et al. (One-step replica symmetry breaking of random regular NAE-SAT I, arXiv:2011.14270, 2020), we study the random regular k-nae-sat model in the condensation regime. In Nam et al. (2020), the (1rsb) properties of the model were established with positive probability. In this paper, we improve the result to probability arbitrarily close to one. To do so, we introduce a new framework which is the synthesis of two approaches: the small subgraph conditioning and a variance decomposition technique using Doob martingales and discrete Fourier analysis. The main challenge is a delicate integration of the two methods to overcome the difficulty arising from applying the moment method to an unbounded state space.

Statistical mechanics of the maximum-average submatrix problem

We study the maximum-average submatrix problem, wherein given an N × N matrix J, one needs to find the k × k submatrix with the largest average number of entries. We investigate the problem for random matrices J, whose entries are i.i.d. random variables, by mapping it to a variant of the Sherrington–Kirkpatrick spin-glass model at fixed magnetisation. We analytically characterise the phase diagram of the model as a function of the submatrix average and the size of the submatrix k in the limit N→∞ . We consider submatrices of size k=mN with 0<m<1 . We find a rich phase diagram, including dynamical, static one-step replica symmetry breaking (1-RSB) and full-step RSB. In the limit of m → 0, we find a simpler phase diagram featuring a frozen 1-RSB phase, where the Gibbs measure comprises exponentially many pure states each with zero entropy. We discover an interesting phenomenon, reminiscent of the phenomenology of the binary perceptron: there are efficient algorithms that provably work in the frozen 1-RSB phase.

Replica symmetry broken states of some glass models.

We have studied in detail the M-p balanced spin-glass model, especially the case p=4. These types of model have relevance to structural glasses. The models possess two kinds of broken replica states; those with one-step replica symmetry breaking (1RSB) and those with full replica symmetry breaking (FRSB). To determine which arises requires studying the Landau expansion to quintic order. There are nine quintic-order coefficients, and five quartic-order coefficients, whose values we determine for this model. We show that it is only for 2≤M<2.4714⋯ that the transition at mean-field level is to a state with FRSB, while for larger M values there is either a continuous transition to a state with 1RSB (when M≤3) or a discontinuous transition for M>3. The Gardner transition from a 1RSB state at low temperatures to a state with FRSB also requires the Landau expansion to be taken to quintic order. Our result for the form of FRSB in the Gardner phase is similar to that found when 2≤M<2.4714⋯, but differs from that given in the early paper of Gross etal. [Phys. Rev. Lett. 55, 304 (1985)0031-900710.1103/PhysRevLett.55.304]. Finally we discuss the effects of fluctuations on our mean-field solutions using the scheme of Höller and Read [Phys. Rev. E 101, 042114 (2020)2470-004510.1103/PhysRevE.101.042114] and argue that such fluctuations will remove both the continuous 1RSB transition and discontinuous 1RSB transitions when 8>d≥6 leaving just the FRSB continuous transition. We suggest values for M and p which might be used in simulations to confirm whether fluctuation corrections do indeed remove the 1RSB transitions.

A tale of two saddles

We find a new on-shell replica wormhole in a computation of the generating functional of JT gravity coupled to matter. We show that this saddle has lower action than the disconnected one, and that it is stable under restriction to real Lorentzian sections, but can be unstable otherwise. The behavior of the classical generating functional thus may be strongly dependent on the signature of allowed perturbations. As part of our analysis, we give an LM-style construction for computing the on-shell action of replicated manifolds even as the number of boundaries approaches zero, including a type of one-step replica symmetry breaking that is necessary to capture the contribution of the new saddle. Our results are robust against quantum corrections; in fact, we find evidence that such corrections may sometimes stabilize this new saddle.

Planted XY model: Thermodynamics and inference.

In this paper we study a fully connected planted spin glass named the planted XY model. Motivation for studying this system comes both from the spin glass field and the one of statistical inference where it models the angular synchronization problem. We derive the replica symmetric (RS) phase diagram in the temperature, ferromagnetic bias plane using the approximate message passing (AMP) algorithm and its state evolution (SE). While the RS predictions are exact on the Nishimori line (i.e., when the temperature is matched to the ferromagnetic bias), they become inaccurate when the parameters are mismatched, giving rise to a spin glass phase where AMP is not able to converge. To overcome the defects of the RS approximation we carry out a one-step replica symmetry-breaking (1RSB) analysis based on the approximate survey propagation (ASP) algorithm. Exploiting the state evolution of ASP, we count the number of metastable states in the measure, derive the 1RSB free entropy and find the behavior of the Parisi parameter throughout the spin glass phase.

Replica Symmetry Breaking in Dense Hebbian Neural Networks

Understanding the glassy nature of neural networks is pivotal both for theoretical and computational advances in Machine Learning and Theoretical Artificial Intelligence. Keeping the focus on dense associative Hebbian neural networks, the purpose of this paper is two-fold: at first we develop rigorous mathematical approaches to address properly a statistical mechanical picture of the phenomenon of {\em replica symmetry breaking} (RSB) in these networks, then -- deepening results stemmed via these routes -- we aim to inspect the {\em glassiness} that they hide. In particular, regarding the methodology, we provide two techniques: the former is an adaptation of the transport PDE to the case, while the latter is an extension of Guerra's interpolation breakthrough. Beyond coherence among the results, either in replica symmetric and in the one-step replica symmetry breaking level of description, we prove the Gardner's picture and we identify the maximal storage capacity by a ground-state analysis in the Baldi-Venkatesh high-storage regime. In the second part of the paper we investigate the glassy structure of these networks: in contrast with the replica symmetric scenario (RS), RSB actually stabilizes the spin-glass phase. We report huge differences w.r.t. the standard pairwise Hopfield limit: in particular, it is known that it is possible to express the free energy of the Hopfield neural network as a linear combination of the free energies of an hard spin glass (i.e. the Sherrington-Kirkpatrick model) and a soft spin glass (the Gaussian or "spherical" model). This is no longer true when interactions are more than pairwise (whatever the level of description, RS or RSB): for dense networks solely the free energy of the hard spin glass survives, proving a huge diversity in the underlying glassiness of associative neural networks.

Proof of the satisfiability conjecture for large $k$

We establish the satisfiability threshold for random $k$-SAT for all $k\ge k_0$, with $k_0$ an absolute constant. That is, there exists a limiting density $\alpha_{\rm sat}(k)$ such that a random $k$-SAT formula of clause density $\alpha$ is with high probability satisfiable for $\alpha\lt \alpha_{\rm sat}$, and unsatisfiable for $\alpha>\alpha_{\rm sat}$. We show that the threshold $\alpha_{\rm sat}(k)$ is given explicitly by the one-step replica symmetry breaking prediction from statistical physics. The proof develops a new analytic method for moment calculations on random graphs, mapping a high-dimensional optimization problem to a more tractable problem of analyzing tree recursions. We believe that our method may apply to a range of random CSPs in the $1$-RSB universality class.

Role of frustration in a weakly disordered checkerboard lattice

Quenched disorder effects on frustrated systems are explored by considering random fluctuations on the antiferromagnetic (AF) interactions between spins on the checkerboard lattice. The replica framework is adopted within a cluster mean-field approach, resulting in an effective single-cluster model. This effective model is treated within a one-step replica symmetry breaking (RSB) approach with exact evaluations for all intracluster interactions. Competing interactions are introduced by tuning the ratio J2/J1 (where J1 and J2 are first-neighbor and second-neighbor interactions, respectively), which can lead to a highly frustrated scenario when J2/J1→1, where a phase transition between AF orders takes place in the absence of disorder. In particular, the AF order appears at lower values of J2/J1, with the Néel temperature decreasing as the frustration increases. However, quenched disorder changes this description, introducing a RSB spin-glass phase for strong enough disorder intensity J. In fact, for low levels of disorder, a RSB solution with staggered magnetization (mixed phase) emerges from the maximum frustration region. It suggests that, in the presence of weak quenched disorder, systems with competing interactions are prone to present a glassy behavior instead of conventional orders.

The number of solutions for random regular NAE-SAT

Recent work has made substantial progress in understanding the transitions of random constraint satisfaction problems. In particular, for several of these models, the exact satisfiability threshold has been rigorously determined, confirming predictions of statistical physics. Here we revisit one of these models, random regular k-nae-sat: knowing the satisfiability threshold, it is natural to study, in the satisfiable regime, the number of solutions in a typical instance. We prove here that these solutions have a well-defined free energy (limiting exponential growth rate), with explicit value matching the one-step replica symmetry breaking prediction. The proof develops new techniques for analyzing a certain “survey propagation model” associated to this problem. We believe that these methods may be applicable in a wide class of related problems.

Approximating the Cumulant Generating Function of Triangles in the Erdös–Rényi Random Graph

We study the pressure of the “edge-triangle model”, which is equivalent to the cumulant generating function of triangles in the Erdös–Rényi random graph. The investigation involves a population dynamics method on finite graphs of increasing volume, as well as a discretization of the graphon variational problem arising in the infinite volume limit. As a result, we locate a curve in the parameter space where a one-step replica symmetry breaking transition occurs. Sampling a large graph in the broken symmetry phase is well described by a graphon with a structure very close to the one of an equi-bipartite graph.

Dynamical mean-field theory and aging dynamics

Dynamical mean-field theory (DMFT) replaces the many-body dynamical problem with one for a single degree of freedom in a thermal bath whose features are determined self-consistently. By focusing on models with soft disordered p-spin interactions, we show how to incorporate the mean-field theory of aging within DMFT. We study cases with only one slow time-scale, corresponding statically to the one-step replica symmetry breaking phase, and cases with an infinite number of slow time-scales, corresponding statically to the full replica symmetry breaking (FRSB) phase. For the former, we show that the effective temperature of the slow degrees of freedom is fixed by requiring critical dynamical behavior on short time-scales, i.e. marginality. For the latter, we find that aging on an infinite number of slow time-scales is governed by a stochastic equation where the clock for dynamical evolution is fixed by the change of the effective temperature, hence obtaining a dynamical derivation of the stochastic equation at the basis of the FRSB phase. Our results extend the realm of the mean-field theory of aging to all situations where DMFT holds.

Compressed Sensing Performance Analysis Via Replica Method Using Bayesian Framework

Compressive sensing (CS) is a new methodology to capture signals at lower rate than the Nyquist sampling rate when the signals are sparse or sparse in some domain. The performance of CS estimators is analyzed in this paper using tools from statistical mechanics, especially called replica method. This method has been used to analyze communication systems like Code Division Multiple Access (CDMA) and multiple input multi- ple output (MIMO) systems with large size. Replica analysis, now days rigorously proved, is an efficient tool to analyze large systems in general. Specifically, we analyze the performance of some of the estimators used in CS like LASSO (the Least Absolute Shrinkage and Selection Operator) estimator and Zero-Norm regularizing estimator as a special case of maximum a posteriori (MAP) estimator by using Bayesian framework to connect the CS estimators and replica method. We use both replica symmetric (RS) ansatz and one-step replica symmetry breaking (1RSB) ansatz, clamming the latter is efficient when the problem is not convex. This work is more analytical in its form. It is deferred for next step to focus on the numerical results.

One-step replica-symmetry-breaking phase below the de Almeida-Thouless line in low-dimensional spin glasses.

The de Almeida-Thouless (AT) line is the phase boundary in the temperature-magnetic field plane of an Ising spin glass at which a continuous (i.e., second-order) transition from a paramagnet to a replica-symmetry-breaking (RSB) phase occurs, according to mean-field theory. Here, using field-theoretic perturbative renormalization group methods on the Bray-Roberts reduced Landau-Ginzburg-type theory for a short-range Ising spin glass in space of dimension d, we show that at nonzero magnetic field the nature of the corresponding transition is modified as follows: (a) For d-6 small and positive, with increasing field on the AT line, first, the ordered phase just below the transition becomes the so-called one-step RSB, instead of the full RSB that occurs in mean-field theory; the transition on the AT line remains continuous with a diverging correlation length. Then at a higher field, a tricritical point separates the latter transition from a quasi-first-order one, that is one at which the correlation length does not diverge, and there is a jump in part of the order parameter, but no latent heat. The location of the tricritical point tends to zero as d→6^{+}. (b) For d≤6, we argue that the quasi-first-order transition could persist down to arbitrarily small nonzero fields, with a transition to full RSB still expected at lower temperature. Whenever the quasi-first-order transition occurs, it is at a higher temperature than the AT transition would be for the same field, preempting it as the temperature is lowered. These results may explain the reported absence of a diverging correlation length in the presence of a magnetic field in low-dimensional spin glasses in some simulations and in high-temperature series expansions. We also draw attention to the similarity of the "dynamically frozen" state, which occurs at temperatures just above the quasi-first-order transition, and the "metastate-average state" of the one-step RSB phase, and discuss the issue of the number of pure states in either.

Possible instability of one-step replica symmetry breaking in p-spin Ising models outside mean-field theory.

The fully connected Ising p-spin model has for p>2 a discontinuous phase transition from the paramagnetic phase to a stable state with one-step replica symmetry breaking (1RSB). However, simulations in three dimension do not look like these mean-field results and have features more like those which would arise with full replica symmetry breaking (FRSB). To help understand how this might come about we have studied in the fully connected p-spin model the state of two-step replica symmetry breaking (2RSB). It has a free energy degenerate with that of 1RSB, but the weight of the additional peak in P(q) vanishes. We expect that the state with full replica symmetry breaking (FRSB) is also degenerate with that of 1RSB. We suggest that finite-size effects will give a nonvanishing weight to the FRSB features, as also will fluctuations about the mean-field solution. Our conclusion is that outside the fully connected model in the thermodynamic limit, FRSB is to be expected rather than 1RSB.

Large deviations of the free energy in the p -spin glass spherical model

We investigate the behavior of the rare fluctuations of the free energy in the p-spin spherical model, evaluating the corresponding rate function via the G\"artner-Ellis theorem. This approach requires the knowledge of the analytic continuation of the disorder-averaged replicated partition function to arbitrary real number of replicas. In zero external magnetic field, we show via a one-step replica symmetry breaking (1RSB) calculation that the rate function is infinite for fluctuations of the free energy above its typical value, corresponding to an anomalous, super-extensive suppression of rare fluctuations. We extend this calculation to non-zero magnetic field, showing that in this case this very large deviation disappears and we try to motivate this finding in light of a geometrical interpretation of the scaled cumulant generating function.

Replica-nondiagonal solutions in the SYK model

We study the SYK model in the large N limit beyond the replica-diagonal approximation. First we show that there are exact replica-nondiagonal solutions of the saddle point equations for q = 2 for any finite replica number M.In the interacting q = 4 case we are able to construct the numerical solutions, which are in one-to-one correspondence to the analytic solutions of the quadratic model. These solutions are singular in the M → 0 limit in both quadratic and quartic interaction cases. The calculations of the on-shell action at finite integer M show that the nondiagonal replica-symmetric saddles are subleading in both quadratic and quartic cases.We also study replica-nondiagonal solutions of the SYK in the strong coupling limit. For arbitrary q we show that besides the usual solutions of the replica-diagonal saddle point equations in the conformal limit, there are also replica-nondiagonal solutions for any value of M (including zero). The specific configurations that we study, have factorized time and replica dependencies. The corresponding saddle point equations are separable at strong coupling, and can be solved using the Parisi ansatz from spin glass theory. We construct the solutions which correspond to the replica-symmetric case and to one-step replica symmetry breaking. We compute the regularizized free energy on these solutions in the limit of zero replicas. It is observed that there are nondiagonal solutions with the regularized free energy lower than that of the standard diagonal conformal solution.

Approximate survey propagation for statistical inference

Approximate message passing algorithm enjoyed considerable attention in the last decade. In this paper we introduce a variant of the AMP algorithm that takes into account glassy nature of the system under consideration. We coin this algorithm as the approximate survey propagation (ASP) and derive it for a class of low-rank matrix estimation problems. We derive the state evolution for the ASP algorithm and prove that it reproduces the one-step replica symmetry breaking (1RSB) fixed-point equations, well-known in physics of disordered systems. Our derivation thus gives a concrete algorithmic meaning to the 1RSB equations that is of independent interest. We characterize the performance of ASP in terms of convergence and mean-squared error as a function of the free Parisi parameter s. We conclude that when there is a model mismatch between the true generative model and the inference model, the performance of AMP rapidly degrades both in terms of MSE and of convergence, while for well-chosen values of the Parisi parameter s ASP converges in a larger regime and can reach lower errors. Among other results, our analysis leads us to a striking hypothesis that whenever s (or other parameters) can be set in such a way that the Nishimori condition M = Q > 0 is restored, then the corresponding algorithm is able to reach mean-squared error as low as the Bayes-optimal error obtained when the model and its parameters are known and exactly matched in the inference procedure. The remaining drawback is that we have not found a procedure that would systematically find a value of s leading to such low errors, this is a challenging problem let for future work.

Specific heat and nonlinear susceptibility in spin glasses with random fields

We study magnetic properties of spin glass (SG) systems under a random field (RF), based on the suggestion that RFs can be induced by a weak transverse field in the compound ${\mathrm{LiHo}}_{x}{\mathrm{Y}}_{1\ensuremath{-}x}{\mathrm{F}}_{4}$. We consider a cluster spin model that allows long-range disordered interactions among clusters and short-range interactions inside the clusters, besides a local RF for each spin following a Gaussian distribution with standard deviation $\mathrm{\ensuremath{\Delta}}$. We adopt the one-step replica symmetry breaking approach to get an exactly solvable single-cluster problem. We discuss the behavior of order parameters, specific heat ${C}_{m}$, nonlinear susceptibility ${\ensuremath{\chi}}_{3}$, and phase diagrams for different disorder configurations. In the absence of RF, the ${\ensuremath{\chi}}_{3}$ exhibits a divergence at ${T}_{f}$, while the ${C}_{m}$ shows a broad maximum at a temperature ${T}^{**}$ around $30%$ above ${T}_{f}$, as expected for conventional SG systems. The presence of RF changes this scenario. The ${C}_{m}$ still shows the maximum at ${T}^{**}$ that is weakly dependent on $\mathrm{\ensuremath{\Delta}}$. However, the ${T}_{f}$ is displaced to lower temperatures, enhancing considerably the ratio ${T}^{**}/{T}_{f}$. Furthermore, the divergence in ${\ensuremath{\chi}}_{3}$ is replaced by a rounded maximum at a temperature ${T}^{*}$, which becomes increasingly higher than ${T}_{f}$ as $\mathrm{\ensuremath{\Delta}}$ is enhanced. As a consequence, the paramagnetic phase is unfolded in three regions: (i) a conventional paramagnetism ($T&gt;{T}^{**}$); (ii) a region with formation of short-range order with frozen spins (${T}^{*}&lt;T&lt;{T}^{**}$); (iii) a region with slow growth of free-energy barriers slowing down the spin dynamics before the SG transition (${T}_{f}&lt;T&lt;{T}^{*}$) suggesting an intermediate Griffiths phase before the SG state. Our results reproduce qualitatively some findings of ${\mathrm{LiHo}}_{x}{\mathrm{Y}}_{1\ensuremath{-}x}{\mathrm{F}}_{4}$ as the rounded maximum of ${\ensuremath{\chi}}_{3}$ behavior triggered by RF and the deviation of the conventional relationship between the ${T}_{f}$ and ${T}^{**}$.

Hessian spectrum at the global minimum of high-dimensional random landscapes

Using the replica method we calculate the mean spectral density of the Hessian matrix at the global minimum of a random dimensional isotropic, translationally invariant Gaussian random landscape confined by a parabolic potential with fixed curvature . Simple landscapes with generically a single minimum are typical for , and we show that the Hessian at the global minimum is always gapped, with the low spectral edge being strictly positive. When approaching from above the transitional point separating simple landscapes from ‘glassy’ ones, with exponentially abundant minima, the spectral gap vanishes as . For the Hessian spectrum is qualitatively different for ‘moderately complex’ and ‘genuinely complex’ landscapes. The former are typical for short-range correlated random potentials and correspond to one-step replica-symmetry breaking mechanism. Their Hessian spectra turn out to be again gapped, with the gap vanishing on approaching from below with a larger critical exponent, as . At the same time in the ‘most complex’ landscapes with long-ranged power-law correlations the replica symmetry is completely broken. We show that in that case the Hessian remains gapless for all values of , indicating the presence of ‘marginally stable’ spatial directions. Finally, the potentials with logarithmic correlations share both 1RSB nature and gapless spectrum. The spectral density of the Hessian always takes the semi-circular form, up to a shift and an amplitude that we explicitly calculate.

The effects of the one-step replica symmetry breaking on the Sherrington–Kirkpatrick spin glass model in the presence of random field with a joint Gaussian probability density function for the exchange interactions and random fields

The Sherrington–Kirkpatrick Ising spin glass model, in the presence of a random magnetic field, is investigated within the framework of the one-step replica symmetry breaking. The two random variables (exchange integral interaction Jij and random magnetic field hi) are drawn from a joint Gaussian probability density function characterized by a correlation coefficient ρ, assuming positive and negative values. The thermodynamic properties, the three different phase diagrams and system’s parameters are computed with respect to the natural parameters of the joint Gaussian probability density function at non-zero and zero temperatures. The low temperature negative entropy controversy, a result of the replica symmetry approach, has been partly remedied in the current study, leading to a less negative result. In addition, the present system possesses two successive spin glass phase transitions with characteristic temperatures.