Replica Symmetric Saddle-point Research Articles

Equivalence between belief propagation instability and transition to replica symmetry breaking in perceptron learning systems

The binary perceptron is a fundamental model of supervised learning for nonconvex optimization, which is a root of the popular deep learning. The binary perceptron is able to achieve a classification of random high-dimensional data based on the marginal probabilities of binary synapses. The relationship between the belief propagation instability and the equilibrium analysis of the model remains elusive. Here, we establish the relationship by showing that the instability condition around the belief propagation fixed point is identical to the instability for breaking the replica symmetric saddle-point solution of the free-energy function. Therefore our analysis will hopefully provide insight towards other learning systems in bridging the gap between nonconvex learning dynamics and statistical mechanics properties of more complex neural networks.

Statistical mechanics of the spherical hierarchical model with random fields

We study analytically the equilibrium properties of the spherical hierarchical model in the presence of random fields. The expression for the critical line separating a paramagnetic from a ferromagnetic phase is derived. The critical exponents characterising this phase transition are computed analytically and compared with those of the corresponding D-dimensional short-range model, leading to conclude that the usual mapping between one dimensional long-range models and D-dimensional short-range models holds exactly for this system, in contrast to models with Ising spins. Moreover, the critical exponents of the pure model and those of the random field model satisfy a relationship that mimics the dimensional reduction rule. The absence of a spin-glass phase is strongly supported by the local stability analysis of the replica symmetric saddle-point as well as by an independent computation of the free-energy using a renormalization-like approach. This latter result enlarges the class of random field models for which the spin-glass phase has been recently ruled out.

Stability of the replica-symmetric saddle point in general mean-field spin-glassmodels

Within the replica approach to mean-field spin glasses the transition from ergodichigh-temperature behaviour to the glassy low-temperature phase is marked bythe instability of the replica-symmetric saddle point. For general spin-glassmodels with non-Gaussian field distributions the corresponding Hessian is a2n × 2n matrix withthe number n of replicas tending to zero eventually. We block-diagonalize this Hessian matrix usingrepresentation theory of the permutation group and identify the blocks related to thespin-glass susceptibility. Taking the limit within these blocks we derive expressions for the de Almeida–Thouless line ofgeneral spin-glass models. Making these expressions specific to the cases of theSherrington–Kirkpatrick, Viana–Bray, and Lévy spin glasses, we obtain results in agreementwith previous findings using the cavity approach.

Effect of second-rank random anisotropy on critical phenomena of a random-fieldO(N)spin model in the largeNlimit

We study the critical behavior of a random field O($N$) spin model with a second-rank random anisotropy term in spatial dimensions $4<d<6$, by means of the replica method and the 1/N expansion. We obtain a replica-symmetric solution of the saddle-point equation, and we find the phase transition obeying dimensional reduction. We study the stability of the replica-symmetric saddle point against the fluctuation induced by the second-rank random anisotropy. We show that the eigenvalue of the Hessian at the replica-symmetric saddle point is strictly positive. Therefore, this saddle point is stable and the dimensional reduction holds in the 1/N expansion. To check the consistency with the functional renormalization group method, we obtain all fixed points of the renormalization group in the large $N$ limit and discuss their stability. We find that the analytic fixed point yielding the dimensional reduction is practically singly unstable in a coupling constant space of the given model with large $N$. Thus, we conclude that the dimensional reduction holds for sufficiently large $N$.

Effect of Second-Rank Random Anisotropy on Critical Phenomena of Random Field O(N) Spin Model

We study the critical behavior of a random field $\mathrm{O}(N)$ spin model with a second-rank random anisotropy term in spatial dimensions $4ldl6$, by means of the replica method and the $1∕N$ expansion. We obtain a replica-symmetric solution of the saddle-point equation, and we find the phase transition obeying dimensional reduction. We study the stability of the replica-symmetric saddle point against the fluctuation induced by the second-rank random anisotropy. We show that the eigenvalue of the Hessian at the replica-symmetric saddle point is strictly positive. Therefore, this saddle point is stable and the dimensional reduction holds in the $1∕N$ expansion. To check the consistency with the functional renormalization group method, we obtain all fixed points of the renormalization group in the large $N$ limit and discuss their stability. We find that the analytic fixed point yielding the dimensional reduction is practically singly unstable in a coupling constant space of the given model with large $N$. Thus, we conclude that the dimensional reduction holds for sufficiently large $N$.

The replica limit of unitary matrix integrals

Abstract We investigate the replica trick for the microscopic spectral density, ρ s (x) , of the Euclidean QCD Dirac operator. Our starting point is the low-energy limit of the QCD partition function for n fermionic flavors (or replicas) in the sector of topological charge ν . In the domain of the smallest eigenvalues, this partition function is simply given by a U(n) unitary matrix integral. We show that the asymptotic expansion of ρ s (x) for x→∞ is obtained from the n→0 limit of this integral. The smooth contributions to this series are obtained from an expansion about the replica symmetric saddle-point, whereas the oscillatory terms follow from an expansion about a saddle-point that breaks the replica symmetry. For ν=0 we recover the small- x logarithmic singularity of the resolvent by means of the replica trick. For half integer ν , when the saddle point expansion of the U(n) integral terminates, the replica trick reproduces the exact analytical result. In all other cases only an asymptotic series that does not uniquely determine the microscopic spectral density is obtained. We argue that bosonic replicas fail to reproduce the microscopic spectral density. In all cases, the exact answer is obtained naturally by means of the supersymmetric method.

Replica symmetry instability in perceptron models

It is demonstrated that the replica symmetric saddle point is unstable when the distribution of aligned fields displays a gap.