Theory Of Spin Glasses Research Articles

Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration

We investigate the signal reconstruction performance of sparse linear regression in the presence of noise when piecewise continuous nonconvex penalties are used. Among such penalties, we focus on the smoothly clipped absolute deviation (SCAD) penalty. The contributions of this study are three-fold: we first present a theoretical analysis of a typical reconstruction performance, using the replica method, under the assumption that each component of the design matrix is given as an independent and identically distributed (i.i.d.) Gaussian variable. This clarifies the superiority of the SCAD estimator compared with in a wide parameter range, although the nonconvex nature of the penalty tends to lead to solution multiplicity in certain regions. This multiplicity is shown to be connected to replica symmetry breaking in the spin-glass theory, and associated phase diagrams are given. We also show that the global minimum of the mean square error between the estimator and the true signal is located in the replica symmetric phase. Second, we develop an approximate formula efficiently computing the cross-validation error without actually conducting the cross-validation, which is also applicable to the non-i.i.d. design matrices. It is shown that this formula is only applicable to the unique solution region and tends to be unstable in the multiple solution region. We implement instability detection procedures, which allows the approximate formula to stand alone and resultantly enables us to draw phase diagrams for any specific dataset. Third, we propose an annealing procedure, called nonconvexity annealing, to obtain the solution path efficiently. Numerical simulations are conducted on simulated datasets to examine these results to verify the consistency of the theoretical results and the efficiency of the approximate formula and nonconvexity annealing. The characteristic behaviour of the annealed solution in the multiple solution region is addressed. Another numerical experiment on a real-world dataset of Type Ia supernovae is conducted; its results are consistent with those of earlier studies using the formulation. A MATLAB package of numerical codes implementing the estimation of the solution path using the annealing with respect to in conjunction with the approximate CV formula and the instability detection routine is distributed in Obuchi (2019 https://github.com/T-Obuchi/SLRpackage_AcceleratedCV_matlab).

Replica-nondiagonal solutions in the SYK model

We study the SYK model in the large N limit beyond the replica-diagonal approximation. First we show that there are exact replica-nondiagonal solutions of the saddle point equations for q = 2 for any finite replica number M.In the interacting q = 4 case we are able to construct the numerical solutions, which are in one-to-one correspondence to the analytic solutions of the quadratic model. These solutions are singular in the M → 0 limit in both quadratic and quartic interaction cases. The calculations of the on-shell action at finite integer M show that the nondiagonal replica-symmetric saddles are subleading in both quadratic and quartic cases.We also study replica-nondiagonal solutions of the SYK in the strong coupling limit. For arbitrary q we show that besides the usual solutions of the replica-diagonal saddle point equations in the conformal limit, there are also replica-nondiagonal solutions for any value of M (including zero). The specific configurations that we study, have factorized time and replica dependencies. The corresponding saddle point equations are separable at strong coupling, and can be solved using the Parisi ansatz from spin glass theory. We construct the solutions which correspond to the replica-symmetric case and to one-step replica symmetry breaking. We compute the regularizized free energy on these solutions in the limit of zero replicas. It is observed that there are nondiagonal solutions with the regularized free energy lower than that of the standard diagonal conformal solution.

Replica Symmetry Breaking in Multi-species Sherrington–Kirkpatrick Model

In the Sherrington-Kirkpatrick (SK) and related mixed $p$-spin models, there is interest in understanding replica symmetry breaking at low temperatures. For this reason, the so-called AT line proposed by de Almeida and Thouless as a sufficient (and conjecturally necessary) condition for symmetry breaking, has been a frequent object of study in spin glass theory. In this paper, we consider the analogous condition for the multi-species SK model, which concerns the eigenvectors of a Hessian matrix. The analysis is tractable in the two-species case with positive definite variance structure, for which we derive an explicit AT temperature threshold. To our knowledge, this is the first non-asymptotic symmetry breaking condition produced for a multi-species spin glass. As possible evidence that the condition is sharp, we draw further parallel with the classical SK model and show coincidence with a separate temperature inequality guaranteeing uniqueness of the replica symmetric critical point.

Finding a large submatrix of a Gaussian random matrix

We consider the problem of finding a $k\times k$ submatrix of an $n\times n$ matrix with i.i.d. standard Gaussian entries, which has a large average entry. It was shown in [Bhamidi, Dey and Nobel (2012)] using nonconstructive methods that the largest average value of a $k\times k$ submatrix is $2(1+o(1))\sqrt{\log n/k}$, with high probability (w.h.p.), when $k=O(\log n/\log\log n)$. In the same paper, evidence was provided that a natural greedy algorithm called the Largest Average Submatrix ($\mathcal{LAS}$) for a constant $k$ should produce a matrix with average entry at most $(1+o(1))\sqrt{2\log n/k}$, namely approximately $\sqrt{2}$ smaller than the global optimum, though no formal proof of this fact was provided. In this paper, we show that the average entry of the matrix produced by the $\mathcal{LAS}$ algorithm is indeed $(1+o(1))\sqrt{2\log n/k}$ w.h.p. when $k$ is constant and $n$ grows. Then, by drawing an analogy with the problem of finding cliques in random graphs, we propose a simple greedy algorithm which produces a $k\times k$ matrix with asymptotically the same average value $(1+o(1))\sqrt{2\log n/k}$ w.h.p., for $k=o(\log n)$. Since the greedy algorithm is the best known algorithm for finding cliques in random graphs, it is tempting to believe that beating the factor $\sqrt{2}$ performance gap suffered by both algorithms might be very challenging. Surprisingly, we construct a very simple algorithm which produces a $k\times k$ matrix with average value $(1+o_{k}(1)+o(1))(4/3)\sqrt{2\log n/k}$ for $k=o((\log n)^{1.5})$, that is, with the asymptotic factor $4/3$ when $k$ grows. To get an insight into the algorithmic hardness of this problem, and motivated by methods originating in the theory of spin glasses, we conduct the so-called expected overlap analysis of matrices with average value asymptotically $(1+o(1))\alpha\sqrt{2\log n/k}$ for a fixed value $\alpha\in[1,\sqrt{2}]$. The overlap corresponds to the number of common rows and the number of common columns for pairs of matrices achieving this value (see the paper for details). We discover numerically an intriguing phase transition at $\alpha^{*}\triangleq5\sqrt{2}/(3\sqrt{3})\approx1.3608\ldots\in[4/3,\sqrt{2}]$: when $\alpha \alpha^{*}$, appropriately defined. We conjecture that the OGP observed for $\alpha>\alpha^{*}$ also marks the onset of the algorithmic hardness—no polynomial time algorithm exists for finding matrices with average value at least $(1+o(1))\alpha\sqrt{2\log n/k}$, when $\alpha>\alpha^{*}$ and $k$ is a mildly growing function of $n$.

Universal behavior of the linear threshold model on weighted networks

The linear threshold model is widely adopted as a classic prototype for studying contagion processes on social networks, where nodes, representing individuals, are assumed to be in one of two states: inactive or active. Each inactive node can be activated via a threshold rule during evolution. Although both contagion mechanisms and network impacts have been well studied, very few studies paid attention on the effect of interacting strengths on the threshold rule. In this paper, a modified linear threshold model on weighted networks is proposed. On one hand, the weight of a link in the network is characterized by a power-law function of the product of endpoint degrees. On the other hand, peer influences on a node incorporate both the number of its active neighbors and associated link weights. The systematic dynamics is explored by the combination of the spin-glass theory and Monte-Carlo algorithm. In analogy to unweighted networks, a global cascade is not triggered in weighted networks when the average degree of nodes is either too small or too large, however, large cascades are realized within an intermediate range, which is referred to as the cascade window. Moreover, two regimes of the power exponent of the weight function are identified in which the system exhibits distinct behaviors: when networks are very sparse, there exist one extreme of the weight exponent making the system susceptible to large cascades; when the networks are relatively dense, on the contrary, there exists the other extreme of the weight exponent causing the system to maintain optimal robustness. All these results demonstrate the importance of both network connectivity and link weights, and offer a sophisticated description of social contagions.

Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines

Restricted Boltzmann machines (RBMs) are energy-based neural-networks which are commonly used as the building blocks for deep architectures neural architectures. In this work, we derive a deterministic framework for the training, evaluation, and use of RBMs based upon the Thouless-Anderson-Palmer (TAP) mean-field approximation of widely-connected systems with weak interactions coming from spin-glass theory. While the TAP approach has been extensively studied for fully-visible binary spin systems, our construction is generalized to latent-variable models, as well as to arbitrarily distributed real-valued spin systems with bounded support. In our numerical experiments, we demonstrate the effective deterministic training of our proposed models and are able to show interesting features of unsupervised learning which could not be directly observed with sampling. Additionally, we demonstrate how to utilize our TAP-based framework for leveraging trained RBMs as joint priors in denoising problems.

Quantum algorithm for energy matching in hard optimization problems

We consider the ability of local quantum dynamics to solve the energy matching problem: given an instance of a classical optimization problem and a low energy state, find another macroscopically distinct low energy state. Energy matching is difficult in rugged optimization landscapes, as the given state provides little information about the distant topography. Here we show that the introduction of quantum dynamics can provide a speed-up over local classical algorithms in a large class of hard optimization problems. The essential intuition is that tunneling allows the system to explore the optimization landscape while approximately conserving the classical energy, even in the presence of large barriers. In particular, we study energy matching in the random p-spin model of spin glass theory. Using perturbation theory and numerical exact diagonalization, we show that introducing a transverse field leads to three sharp dynamical phases, only one of which solves the matching problem: (1) a small-field trapped phase, in which tunneling is too weak for the system to escape the vicinity of the initial state; (2) a large-field excited phase, in which the field excites the system into high energy states, effectively forgetting the initial low energy; and (3) the intermediate tunneling phase, in which the system succeeds at energy matching. We find that in the tunneling phase, the time required to find distant states scales exponentially with system size but is nevertheless exponentially faster than simple classical Monte Carlo.

Quantum Glass of Interacting Bosons with Off-Diagonal Disorder.

We study disordered interacting bosons described by the Bose-Hubbard model with Gaussian-distributed random tunneling amplitudes. It is shown that the off-diagonal disorder induces a spin-glass-like ground state, characterized by randomly frozen quantum-mechanical U(1) phases of bosons. To access criticality, we employ the "n-replica trick," as in the spin-glass theory, and the Trotter-Suzuki method for decomposition of the statistical density operator, along with numerical calculations. The interplay between disorder, quantum, and thermal fluctuations leads to phase diagrams exhibiting a glassy state of bosons, which are studied as a function of model parameters. The considered system may be relevant for quantum simulators of optical-lattice bosons, where the randomness can be introduced in a controlled way. The latter is supported by a proposition of experimental realization of the system in question.

A comparison study of harmony search and genetic algorithm for the max-cut problem

Abstract The max-cut problem is one of well-known NP-complete problems and has applications in various fields such as the design process for VLSI (Very-Large-Scale Integration) chips and spin glass theory in statistical physics. In this paper, a harmony search algorithm for the max-cut problem is proposed. Compared to genetic algorithm, harmony search algorithm has advantages of generating a new vector after considering all of the existing vectors and requiring only a few number of parameters to be determined before the run of the algorithm. For 31 benchmark graphs of various types, the proposed harmony search algorithm is compared with a newly developed genetic algorithm and another genetic algorithm taken from the recent literature are presented. The proposed harmony search algorithm produced significantly better results than the two genetic algorithms.

On the K‐sat model with large number of clauses

AbstractWe show that in the K‐sat model with N variables and αN clauses, the expected ratio of the smallest number of unsatisfied clauses to the number of variables is up to smaller order terms as uniformly in N, where is the expected normalized maximum energy of some specific mixed p‐spin spin glass model. The formula for the limit of is well known in the theory of spin glasses.

Observation of replica symmetry breaking in disordered nonlinear wave propagation

A landmark of statistical mechanics, spin-glass theory describes critical phenomena in disordered systems that range from condensed matter to biophysics and social dynamics. The most fascinating concept is the breaking of replica symmetry: identical copies of the randomly interacting system that manifest completely different dynamics. Replica symmetry breaking has been predicted in nonlinear wave propagation, including Bose-Einstein condensates and optics, but it has never been observed. Here, we report the experimental evidence of replica symmetry breaking in optical wave propagation, a phenomenon that emerges from the interplay of disorder and nonlinearity. When mode interaction dominates light dynamics in a disordered optical waveguide, different experimental realizations are found to have an anomalous overlap intensity distribution that signals a transition to an optical glassy phase. The findings demonstrate that nonlinear propagation can manifest features typical of spin-glasses and provide a novel platform for testing so-far unexplored fundamental physical theories for complex systems.

A multispin algorithm for the Kob-Andersen stochastic dynamics on regular lattices

The aim of the paper is to propose an algorithm based on the Multispin Coding technique for the Kob-Andersen glassy dynamics. We first give motivations to speed up the numerical simulation in the context of spin glass models [M. Mezard, G. Parisi, M. Virasoro, Spin Glass Theory and Beyond (World Scientific, Singapore, 1987)], after defining the Markovian dynamics as in [W. Kob, H.C. Andersen, Phys. Rev. E 48, 4364 (1993)] as well as the related interesting observables, we extend it to the more general framework of random regular graphs, listing at the same time some known analytical results [C. Toninelli, G. Biroli, D.S. Fisher, J. Stat. Phys. 120, 167 (2005)]. The purpose of this work is a dual one, firstly, we describe how bitwise operators can be used to build up the algorithm by carefully exploiting the way data are stored on a computer. Since it was first introduced [M. Creutz, L. Jacobs, C. Rebbi, Phys. Rev. D 20, 1915 (1979), C. Rebbi, R.H. Swendsen, Phys. Rev. D 21, 4094 (1980)], this technique has been widely used to perform Monte Carlo simulations for Ising and Potts spin systems, however, it can be successfully adapted to more complex systems in which microscopic parameters may assume boolean values. Secondly, we introduce a random graph in which a characteristic para- meter allows to tune the possible transition point. A consistent part is devoted to listing the numerical results obtained by running numerical simulations.

Clustering of Nonergodic Eigenstates in Quantum Spin Glasses.

The two primary categories for eigenstate phases of matter at a finite temperature are many-body localization (MBL) and the eigenstate thermalization hypothesis (ETH). We show that, in the paradigmatic quantum p-spin models of the spin-glass theory, eigenstates violate the ETH yet are not MBL either. A mobility edge, which we locate using the forward-scattering approximation and replica techniques, separates the nonergodic phase at a small transverse field from an ergodic phase at a large transverse field. The nonergodic phase is also bounded from above in temperature, by a transition in configuration-space statistics reminiscent of the clustering transition in the spin-glass theory. We show that the nonergodic eigenstates are organized in clusters which exhibit distinct magnetization patterns, as characterized by an eigenstate variant of the Edwards-Anderson order parameter.

Sir Sam Edwards. 1 February 1928 — 7 July 2015

Sam Edwards was one of the leading physicists of the second half of the twentieth century. He was Cavendish Professor at the University of Cambridge, a Vice President of the Royal Society, a member of the Académie des Sciences and of the US National Academy, and a senior figure in the university and his college. He played a major role in public life, most notably as chairman of the Science Research Council (SRC), responsible for research funding in the UK. He was chairman of the British Association, chief government scientist to the Department of Energy, and chairman of the Defence Scientific Advisory Council. He was equally in demand to lead or to help set up bodies abroad, particularly the Max Planck Institute for Polymers in Mainz, Germany. Remarkably, Sam made some of his most celebrated scientific discoveries, for instance the theory of spin glasses and the rheology of high polymer melts, while serving as the full-time head of the SRC. Conversely, his scientific insights informed his leadership in advising the government. His later science was in highly applicable areas: he was an active advisor to Unilever, Dow, Lucas and many other companies that rely on research.

Sherrington–Kirkpatrick glassy-phase of random Josephson coupled Bose–Einstein condensates in wood-pile geometry

We study a transition to a glassy phase of neutral atoms trapped in optical lattice, where the system is realized as the array of individual Bose–Einstein condensates of N elongated vertical and horizontal N rods in a wood-pile form coupled via the random Josephson tunneling. In this geometry every horizontal (vertical) rod of a condensate is linked to its vertical (horizontal) counterpart, so that the number of nearest neighbors z of a given rod in this system is z=N, implying that the system is fully connected. This together with randomness forms a prerequisite of the Sherrington–Kirkpatrick model for N→∞ widely employed in the theory of spin glasses. For this arrangement we solve a model Hamiltonian of the Josephson array in the thermodynamic limit (N→∞) and calculate the critical temperature for the glassy phase-locking transition, caused by the Josephson tunneling of bosons in random environment.

Theory of spin glasses—commentary

Theory of spin glasses—commentary

Identifying optimal targets of network attack by belief propagation

For a network formed by nodes and undirected links between pairs of nodes, the network optimal attack problem aims at deleting a minimum number of target nodes to break the network down into many small components. This problem is intrinsically related to the feedback vertex set problem that was successfully tackled by spin-glass theory and an associated belief propagation-guided decimation (BPD) algorithm [Zhou, Eur. Phys. J. B 86, 455 (2013)EPJBFY1434-602810.1140/epjb/e2013-40690-1]. In the present work we apply the BPD algorithm (which has approximately linear time complexity) to the network optimal attack problem and demonstrate that it has much better performance than a recently proposed collective information algorithm [Morone and Makse, Nature 524, 65 (2015)NATUAS0028-083610.1038/nature14604] for different types of random networks and real-world network instances. The BPD-guided attack scheme often induces an abrupt collapse of the whole network, which may make it very difficult to defend.

Generalized minimum dominating set and application in automatic text summarization

For a graph formed by vertices and weighted edges, a generalized minimum dominating set (MDS) is a vertex set of smallest cardinality such that the summed weight of edges from each outside vertex to vertices in this set is equal to or larger than certain threshold value. This generalized MDS problem reduces to the conventional MDS problem in the limiting case of all the edge weights being equal to the threshold value. We treat the generalized MDS problem in the present paper by a replica-symmetric spin glass theory and derive a set of belief-propagation equations. As a practical application we consider the problem of extracting a set of sentences that best summarize a given input text document. We carry out a preliminary test of the statistical physics-inspired method to this automatic text summarization problem.

Composite operators in cubic field theories and link-overlap fluctuations in spin-glass models

We present a complete characterization of the fluctuations and correlations of the squared overlap in the Edwards-Anderson Spin-Glass model in zero field. The analysis reveals that the energy-energy correlations (and thus the specific heat) have a different critical behavior than the fluctuations of the link overlap in spite of the fact that the average energy and average link overlap have the same critical properties. More precisely the link-overlap fluctuations are larger than the specific heat according to a computation at first order in the $6-\epsilon$ expansion. An unexpected outcome is that the link-overlap fluctuations have a subdominant power-law contribution characterized by an anomalous logarithmic prefactor which is missing in the specific heat. In order to compute the $\epsilon$ expansion we consider the problem of the renormalization of quadratic composite operators in a generic multicomponent cubic field theory: the results obtained have a range of applicability beyond spin-glass theory.

The solution space structure of random constraint satisfaction problems with growing domains

.In this paper we study the solution space structure of model RB, a standard prototype of the constraint satisfaction problem (CSP), with growing domains. Using the first moment method and the second moment method, we rigorously show that in the satisfiable phase close to the satisfiability transition, solutions are clustered into an exponential number of well-separated clusters, with each cluster containing a sub-exponential number of solutions. As a consequence, the system has a clustering (dynamical) transition but no condensation transition. This picture of the phase diagram is different to other classic random CSPs that possess a fixed domain size, such as the K-satisfiability (K-SAT) problem and the graph colouring problem, where a condensation transition exists and is distinctly different to the satisfiability transition. Our result verifies some non-rigorous results obtained using the cavity method from spin glass theory.