Sherrington-Kirkpatrick Spin Glass Research Articles

Geometric Landscape Annealing as an Optimization Principle Underlying the Coherent Ising Machine

Given the fundamental importance of combinatorial optimization across many diverse domains, there has been widespread interest in the development of unconventional physical computing architectures that can deliver better solutions with lower resource costs. However, a theoretical understanding of their performance remains elusive. We develop such understanding for the case of the coherent Ising machine (CIM), a network of optical parametric oscillators that can be applied to any quadratic unconstrained binary optimization problem. We focus on how the CIM finds low-energy solutions of the Sherrington-Kirkpatrick spin glass. As the laser gain of this system is annealed, the CIM interpolates between gradient descent on coupled soft spins to descent on coupled binary spins. By combining the Kac-Rice formula, the replica method, and supersymmetry breaking, we develop a detailed understanding of the evolving geometry of the high-dimensional energy landscape of the CIM as the laser gain increases, finding several phase transitions in the landscape, from flat to rough to rigid. Additionally, we develop a novel cavity method that provides a geometric interpretation of supersymmetry breaking in terms of the reactivity of a rough landscape to specific external perturbations. Our energy landscape theory successfully matches numerical experiments, provides geometric insights into the principles of CIM operation, and yields optimal annealing schedules. Published by the American Physical Society 2024

Unlearning regularization for Boltzmann machines

Boltzmann machines (BMs) are graphical models with interconnected binary units, employed for the unsupervised modeling of data distributions. When trained on real data, BMs show the tendency to behave like critical systems, displaying a high susceptibility of the model under a small rescaling of the inferred parameters. This behavior is not convenient for the purpose of generating data, because it slows down the sampling process, and induces the model to overfit the training-data. In this study, we introduce a regularization method for BMs to improve the robustness of the model under rescaling of the parameters. The new technique shares formal similarities with the unlearning algorithm, an iterative procedure used to improve memory associativity in Hopfield-like neural networks. We test our unlearning regularization on synthetic data generated by two simple models, the Curie–Weiss ferromagnetic model and the Sherrington–Kirkpatrick spin glass model. We show that it outperforms Lp -norm schemes and discuss the role of parameter initialization. Eventually, the method is applied to learn the activity of real neuronal cells, confirming its efficacy at shifting the inferred model away from criticality and coming out as a powerful candidate for actual scientific implementations.

Complete replica solution for the transverse field Sherrington-Kirkpatrick spin glass model with continuous-time quantum Monte Carlo method

Complete replica solution for the transverse field Sherrington-Kirkpatrick spin glass model with continuous-time quantum Monte Carlo method

Ground-state interface exponents of the diluted Sherrington-Kirkpatrick spin glass

We present a large-scale simulation of the ground state interface properties of the diluted Sherrington-Kirkpatrick spin glass of Gaussian disorder for a broad range of the bond occupation probability $p$ using the strong disorder renormalization group and the population annealing Monte Carlo methods. We find that the interface is space-filling independent of $p$, i.e., the fractal dimension $d_s=1$. The stiffness exponent $\theta$ is likely also independent of $p$, despite that the energy finite-size correction exponent $\omega$ varies with $p$ as recently found. The energy finite-size scaling is also analyzed and compared with that of the $\pm J$ disorder, finding that the thermodynamic energy is universal in both $p$ and the disorder, and the exponent $\omega$ varies with $p$ but is universal in the disorder.

Optimized SWAP networks with equivalent circuit averaging for QAOA

The SWAP network is a qubit routing sequence that can be used to efficiently execute the Quantum Approximate Optimization Algorithm (QAOA). Even with a minimally connected topology on an $n$-qubit processor, this routing sequence enables $\mathcal{O}({n}^{2})$ operations to execute in $\mathcal{O}(n)$ steps. In this work, we optimize the execution of SWAP networks for QAOA through two techniques. First, we take advantage of an overcomplete set of native hardware operations [including 150-ns controlled-$\frac{\ensuremath{\pi}}{2}$ phase gates with up to 99.67(1)% fidelity] to decompose the relevant quantum gates and SWAP networks in a manner which minimizes circuit depth and maximizes gate cancellation. Second, we introduce equivalent circuit averaging, which randomizes over degrees of freedom in the quantum circuit compilation to reduce the impact of systematic coherent errors. Our techniques are experimentally validated at the Advanced Quantum Testbed through the execution of QAOA circuits for finding the ground state of two- and four-node Sherrington-Kirkpatrick spin-glass models with various randomly sampled parameters. We observe a $\ensuremath{\sim}60%$ average reduction in error (total variation distance) for QAOA of depth $p=1$ on four transmon qubits on a superconducting quantum processor.

A transport equation approach for deep neural networks with quenched random weights

We consider a multi-layer Sherrington–Kirkpatrick spin-glass as a model for deep restricted Boltzmann machines with quenched random weights and solve for its free energy in the thermodynamic limit by means of Guerra’s interpolating techniques under the RS and 1RSB ansatz. In particular, we recover the expression already known for the replica-symmetric case. Further, we drop the restriction constraint by introducing intra-layer connections among spins and we show that the resulting system can be mapped into a modular Hopfield network, which is also addressed via the same techniques up to the first step of replica symmetry breaking.

Dynamical Approach to the TAP Equations for the Sherrington–Kirkpatrick Model

We present a new dynamical proof of the Thouless-Anderson-Palmer (TAP) equations for the classical Sherrington-Kirkpatrick spin glass at sufficiently high temperature. In our derivation, the TAP equations are a simple consequence of the decay of the two point correlation functions. The methods can also be used to establish the decay of higher order correlation functions. We illustrate this by proving a suitable decay bound on the three point functions from which we derive an analogue of the TAP equations for the two point functions.

Spherical Spin Glass Model with External Field

We analyze the free energy and the overlaps in the 2-spin spherical Sherrington Kirkpatrick spin glass model with an external field for the purpose of understanding the transition between this model and the one without an external field. We compute the limiting values and fluctuations of the free energy as well as three types of overlaps in the setting where the strength of the external field goes to zero as the dimension of the spin variable grows. In particular, we consider overlaps with the external field, the ground state, and a replica. Our methods involve a contour integral representation of the partition function along with random matrix techniques. We also provide computations for the matching between different scaling regimes. Finally, we discuss the implications of our results for susceptibility and for the geometry of the Gibbs measure. Some of the findings of this paper are confirmed rigorously by Landon and Sosoe in their recent paper which came out independently and simultaneously.

Analysis of landscape hierarchy during coarsening and aging in Ising spin glasses

We use record dynamics (RD), a coarse-grained description of the ubiquitous relaxation phenomenology known as "aging", as a diagnostic tool to find universal features that distinguish between the energy landscapes of Ising spin models and the ferromagnet. According to RD, a non-equilibrium system after a quench relies on fluctuations that randomly generate a sequence of irreversible record-sized events (quakes or avalanches) that allow the system to escape ever-higher barriers of meta-stable states within a complex, hierarchical energy landscape. Once these record events allow the system to overcome such barriers, the system relaxes by tumbling into the following meta-stable state that is marginally more stable. Within this framework, a clear distinction can be drawn between the coarsening dynamics of an Ising ferromagnet and the aging of the spin glass, which are often put in the same category. To that end, we interpolate between the spin glass and ferromagnet by varying the admixture $p$ of ferromagnetic over anti-ferromagnetic bonds from the glassy state (at 50% each) to wherever clear ferromagnetic behavior emerges. The accumulation of record events grows logarithmic with time in the glassy regime, with a sharp transition at a specific admixture into the ferromagnetic regime where such activations saturate quickly. We show this effect both for the Edwards-Anderson model on a cubic lattice as well as the Sherrington-Kirkpatrick (mean-field) spin glass. While this transition coincides with a previously observed zero-temperature equilibrium transition in the former, that transition has not yet been described for the latter.

Ground State Properties of the Diluted Sherrington-Kirkpatrick Spin Glass

We present a numerical study of ground states of the dilute versions of the Sherrington-Kirkpatrick (SK) mean-field spin glass. In contrast to so-called "sparse" mean-field spin glasses that have been studied widely on random networks of finite (average or regular) degree, the networks studied here are randomly bond diluted to an overall density p, such that the average degree diverges as ∼pN with the system size N. Ground state energies are obtained with high accuracy for random instances over a wide range of fixed p. Since this is an NP-hard combinatorial problem, we employ the extremal optimization heuristic to that end. We find that the exponent describing the finite-size corrections ω varies continuously with p, a somewhat surprising result, as one would not expect that gradual bond dilution would change the T=0 universality class of a statistical model. For p→1, the familiar result of ω(p=1)≈2/3 for the SK model is obtained.

Annealing and Replica-Symmetry in Deep Boltzmann Machines

In this paper we study the properties of the quenched pressure of a multi-layer spin-glass model (a deep Boltzmann Machine in artificial intelligence jargon) whose pairwise interactions are allowed between spins lying in adjacent layers and not inside the same layer nor among layers at distance larger than one. We prove a theorem that bounds the quenched pressure of such a K-layer machine in terms of K Sherrington-Kirkpatrick spin glasses and use it to investigate its annealed region. The replica-symmetric approximation of the quenched pressure is identified and its relation to the annealed one is considered. The paper also presents some observation on the model's architectural structure related to machine learning. Since escaping the annealed region is mandatory for a meaningful training, by squeezing such region we obtain thermodynamical constraints on the form factors. Remarkably, its optimal escape is achieved by requiring the last layer to scale sub-linearly in the network size.

Study of longitudinal fluctuations of the Sherrington–Kirkpatrick model

We study finite-size corrections to the free energy of the Sherrington–Kirkpatrick spin glass in the low-temperature phase where replica symmetry is broken. We investigate the role of longitudinal fluctuations in these corrections, neglecting the transverse contribution. In particular, we are interested in the value of exponent that controls the finite volume corrections: is defined by the relation , N being the total number of spins. We perform both an analytical and numerical estimate of the analytical result for . From both the approaches, we get the result: .

Replica symmetry breaking in bipartite spin glasses and neural networks

Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study the mathematically tractable bipartite Sherrington-Kirkpatrick (SK) spin-glass model, which is formally similar to a restricted Boltzmann machine (RBM) neural network. The bipartite SK model has been previously studied assuming replica symmetry; here this assumption is relaxed and a replica symmetry breaking analysis is performed. The bipartite SK model is found to have many features in common with Parisi's solution of the original, unipartite SK model, including the existence of a multitude of pure states which are related in a hierarchical, ultrametric fashion. As an application of this analysis, the optimal cost for a graph partitioning problem is shown to be simply related to the ground state energy of the bipartite SK model. As a second application, empirical investigations reveal that the Gibbs sampled outputs of an RBM trained on the MNIST data set are more ultrametrically distributed than the input data themselves.

Possible ergodic-nonergodic regions in the quantum Sherrington-Kirkpatrick spin glass model and quantum annealing.

We explore the behavior of the order parameter distribution of the quantum Sherrington-Kirkpatrick model in the spin glass phase using Monte Carlo technique for the effective Suzuki-Trotter Hamiltonian at finite temperatures and that at zero temperature obtained using the exact diagonalization method. Our numerical results indicate the existence of a low- but finite-temperature quantum-fluctuation-dominated ergodic region along with the classical fluctuation-dominated high-temperature nonergodic region in the spin glass phase of the model. In the ergodic region, the order parameter distribution gets narrower around the most probable value of the order parameter as the system size increases. In the other region, the Parisi order distribution function has nonvanishing value everywhere in the thermodynamic limit, indicating nonergodicity. We also show that the average annealing time for convergence (to a low-energy level of the model, within a small error range) becomes system size independent for annealing down through the (quantum-fluctuation-dominated) ergodic region. It becomes strongly system size dependent for annealing through the nonergodic region. Possible finite-size scaling-type behavior for the extent of the ergodic region is also addressed.

The effects of the one-step replica symmetry breaking on the Sherrington–Kirkpatrick spin glass model in the presence of random field with a joint Gaussian probability density function for the exchange interactions and random fields

The Sherrington–Kirkpatrick Ising spin glass model, in the presence of a random magnetic field, is investigated within the framework of the one-step replica symmetry breaking. The two random variables (exchange integral interaction Jij and random magnetic field hi) are drawn from a joint Gaussian probability density function characterized by a correlation coefficient ρ, assuming positive and negative values. The thermodynamic properties, the three different phase diagrams and system’s parameters are computed with respect to the natural parameters of the joint Gaussian probability density function at non-zero and zero temperatures. The low temperature negative entropy controversy, a result of the replica symmetry approach, has been partly remedied in the current study, leading to a less negative result. In addition, the present system possesses two successive spin glass phase transitions with characteristic temperatures.

Stability of the quantum Sherrington-Kirkpatrick spin glass model.

I study in detail the quantum Sherrington-Kirkpatrick (SK) model, i.e., the infinite-range Ising spin glass in a transverse field, by solving numerically the effective one-dimensional model that the quantum SK model can be mapped to in the thermodynamic limit. I find that the replica symmetric solution is unstable down to zero temperature, in contrast to some previous claims, and so there is not only a line of transitions in the (longitudinal) field-temperature plane (the de Almeida-Thouless, AT, line) where replica symmetry is broken, but also a quantum de Almeida-Thouless (QuAT) line in the transverse field-longitudinal field plane at T=0. If the QuAT line also occurs in models with short-range interactions its presence might affect the performance of quantum annealers when solving spin glass-type problems with a bias (i.e., magnetic field).

Optimizing Variational Quantum Algorithms Using Pontryagin’s Minimum Principle

We use Pontryagin's minimum principle to optimize variational quantum algorithms. We show that for a fixed computation time, the optimal evolution has a bang-bang (square pulse) form, both for closed and open quantum systems with Markovian decoherence. Our findings support the choice of evolution ansatz in the recently proposed Quantum Approximate Optimization Algorithm. Focusing on the Sherrington-Kirkpatrick spin-glass as an example, we find a system-size independent distribution of the duration of pulses, with characteristic time scale set by the inverse of the coupling constants in the Hamiltonian. The optimality of the bang-bang protocols and the characteristic time scale of the pulses provide an efficient parameterization of the protocol and inform the search for effective hybrid (classical and quantum) schemes for tackling combinatorial optimization problems. For the particular systems we study, we find numerically that the optimal nonadiabatic bang-bang protocols outperform conventional quantum annealing in the presence of weak white additive external noise and weak coupling to a thermal bath modeled with the Redfield master equation.

Classical-to-quantum crossover in the critical behavior of the transverse-field Sherrington-Kirkpatrick spin glass model.

We study the critical behavior of the Sherrington-Kirkpatrick model in transverse field (at finite temperature) using Monte Carlo simulation and exact diagonalization (at zero temperature). We determine the phase diagram of the model by estimating the Binder cumulant. We also determine the correlation length exponent from the collapse of the scaled data. Our numerical studies here indicate that critical Binder cumulant (indicating the universality class of the transition behavior) and the correlation length exponent cross over from their "classical" to "quantum" values at a finite temperature (unlike the cases of pure systems, where such crossovers occur at zero temperature). We propose a qualitative argument supporting such an observation, employing a simple tunneling picture.

Dynamics and Correlations among Soft Excitations in Marginally Stable Glasses.

Marginal stability is the notion that stability is achieved, but only barely so. This property constrains the ensemble of configurations explored at low temperature in a variety of systems, including spin, electron, and structural glasses. A key feature of marginal states is a (saturated) pseudogap in the distribution of soft excitations. We examine how such pseudogaps appear dynamically by studying the Sherrington-Kirkpatrick (SK) spin glass. After revisiting and correcting the multi-spin-flip criterion for local stability, we show that stationarity along the hysteresis loop requires soft spins to be frustrated among each other, with a correlation diverging as C(λ)∼1/λ, where λ is the stability of the more stable spin. We explain how this arises spontaneously in a marginal system and develop an analogy between the spin dynamics in the SK model and random walks in two dimensions. We discuss analogous frustrations among soft excitations in short range glasses and how to detect them experimentally. We also show how these findings apply to hard sphere packings.

Quantum annealing search of Ising spin glass ground state(s) with tunable transverse and longitudinal fields

Here we first discuss briefly the quantum annealing technique. We then study the quantum annealing of Sherrington-Kirkpatrick spin glass model with the tuning of both transverse and longitudinal fields. Both the fields are time-dependent and vanish adiabatically at the same time, starting from high values. We solve, for rather small systems, the time-dependent Schrodinger equation of the total Hamiltonian by employing a numerical technique. At the end of annealing we obtain the final state having high overlap with the exact ground state(s) of classical spin glass system (obtained independently).