Replica Symmetric Research Articles

Real Space Renormalization Group Analysis with the Replica Method for the Two Dimensional Ising Edwards–Anderson Model

We apply the real space renormalization group (RG) to the replica Hamiltonian of the two dimensional Ising Edwards–Anderson model to discuss the existence of the spin glass (SG) phase. We derive the RG equations under the replica symmetric (RS) ansatz and the resulting flow diagram indicates the existence of the SG phase. The critical exponent for the SG transition has a plausible value while that of the multicritical point (MCP) is a complex number. We consider that this failure at the MCP is due to not taking the RS breaking into account. Indeed we find that the RS breaking parameter is relevant both at the SG critical point and the MCP, indicating the nontriviality of the SG phase of this model.

Finite-connectivity spin-glass phase diagrams and low-density parity check codes

We obtain phase diagrams of regular and irregular finite-connectivity spin glasses. Contact is first established between properties of the phase diagram and the performance of low-density parity check (LDPC) codes within the replica symmetric (RS) ansatz. We then study the location of the dynamical and critical transition points of these systems within the one step replica symmetry breaking theory (RSB), extending similar calculations that have been performed in the past for the Bethe spin-glass problem. We observe that the location of the dynamical transition line does change within the RSB theory, in comparison with the results obtained in the RS case. For LDPC decoding of messages transmitted over the binary erasure channel we find, at zero temperature and rate , an RS critical transition point at while the critical RSB transition point is located at , to be compared with the corresponding Shannon bound . For the binary symmetric channel we show that the low temperature reentrant behavior of the dynamical transition line, observed within the RS ansatz, changes its location when the RSB ansatz is employed; the dynamical transition point occurs at higher values of the channel noise. Possible practical implications to improve the performance of the state-of-the-art error correcting codes are discussed.

Spin glass models with ferromagnetically biased couplings on the Bethe lattice: analytic solutions and numerical simulations

We derive the zero-temperature phase diagram of spin glass models with a generic fraction of ferromagnetic interactions on the Bethe lattice. We use the cavity method at the level of one-step replica symmetry breaking (1RSB) and we find three phases: A replica-symmetric (RS) ferromagnetic one, a magnetized spin glass one (the so-called mixed phase), and an unmagnetized spin glass one. We are able to give analytic expressions for the critical point where the RS phase becomes unstable with respect to 1RSB solutions: we also clarify the mechanism inducing such a phase transition. Finally we compare our analytical results with the outcomes of a numerical algorithm especially designed for finding ground states in an efficient way, stressing weak points in the use of such numerical tools for discovering RSB effects. Some of the analytical results are given for generic connectivity.

Finite connectivity attractor neural networks

We study a family of diluted attractor neural networks with a finite average number of (symmetric) connections per neuron. As in finite connectivity spin glasses, their equilibrium properties are described by order parameter functions, for which we derive an integral equation in replica symmetric (RS) approximation. A bifurcation analysis of this equation reveals the locations of the paramagnetic to recall and paramagnetic to spin-glass transition lines in the phase diagram. The line separating the retrieval phase from the spin-glass phase is calculated at zero temperature. All phase transitions are found to be continuous.

Cluster expansions in dilute systems: applications to satisfiability problems and spin glasses.

We develop a systematic cluster expansion for dilute systems in the highly dilute phase. We first apply it to the calculation of the entropy of the K-satisfiability problem in the satisfiable phase. We derive a series expansion in the control parameter, the average connectivity, that is identical to the one obtained by using the replica approach with a replica symmetric (RS) ansatz, when the order parameter is calculated via a perturbative expansion in the control parameter. As a second application we compute the free energy of the Viana-Bray model in the paramagnetic phase. The cluster expansion allows one to compute finite-size corrections in a simple manner, and these are particularly important in optimization problems. Importantly enough, these calculations prove the exactness of the RS ansatz below the percolation threshold, and might require its revision between this and the easy-to-hard transition.

Generalization Ability of Hopfield Neural Network with Spin-S Ising Neurons

We investigate a fully connected Hopfield neural network with spin- S ( S ≥1) Ising neurons, including S =∞, when binary patterns are embedded by the Hebbian learning rule. We analyze the energy function of the neural network using the replica method. We investigate a generalization ability of the neural network within the replica symmetric (RS) solutions. We clarify that the generalization ability of the neural network with a larger value of S is enhanced when the finite number of concepts are extracted from the presented examples.

RS and RSB solutions for SK model with spin- S

We investigate the Sherrington–Kirkpatrick (SK) model with spin- S, including S=∞, in a framework of the replica method. We obtain replica symmetric (RS) solutions and one- and two-step replica symmetry breaking (RSB) solutions by using the Parisi scheme, after verifying de Almeida and Thouless (AT) stability conditions for the RS solutions. We find that the AT line exists for all the values of S, including S=∞, and therefore the SK model shows a transition from the RS solution to the RSB solution for all the values of S, including S=∞. It turns out that the properties near T=0 are common for finite S but some of them are not for S=∞.

The mutual information of a stochastic binary channel: validity of the replica symmetry ansatz

We calculate the mutual information (MI) of a two-layered neural network with noiseless, continuous inputs and binary, stochastic outputs under several assumptions on the synaptic efficiencies. The interesting regime corresponds to the limit where the number of both input and output units is large but their ratio is kept fixed at a value $\alpha$. We first present a solution for the MI using the replica technique with a replica symmetric (RS) ansatz. Then we find an exact solution for this quantity valid in a neighborhood of $\alpha = 0$. An analysis of this solution shows that the system must have a phase transition at some finite value of $\alpha$. This transition shows a singularity in the third derivative of the MI. As the RS solution turns out to be infinitely differentiable, it could be regarded as a smooth approximation to the MI. This is checked numerically in the validity domain of the exact solution.

The statistical mechanics of constructive algorithms

The storage capacity of multilayer networks with overlapping receptive fields is studied for constructive algorithms using Boolean perceptrons as their basic building block which have been investigated within a replica framework. The assumption of weak coupling between subsequently constructed perceptrons is verified within a replica symmetric (RS) ansatz and shown to be negligible in most cases in comparison with correction due to replica symmetry breaking (RSB) in individual perceptrons. The capacities of a tiling-like and variants of the upstart algorithm are then calculated within RS and one-step RSB with the quenched average taken over the individual units separately for networks with up to K = 4000 and K = 600 units respectively. Within this treatment, the storage capacity seems to exhibit a power-law behaviour in with an exponent n that may depend on the algorithm and the stability. However, due to finite size effects in K reliable estimates of n could not be extracted. Nevertheless, the results strongly indicate that n should be strictly smaller than 1 within one-step RSB, whereas within RS the Mitchison-Durbin bound is violated for finite K and n>1 may hold asymptotically.

Cluster-Mean-Field Approach to Short-Range Ising Spin Glasses in Replica Space

We construct a cluster-mean-field theory for short-range spin glasses in the replica space, in which only the effective fields which are conjugate to the order parameters are taken into account. In the one-step replica-symmetry breaking (RSB) scheme just below the critical temperature, the solutions of the self-consistent equations for general clusters are given in terms of the thermal and quenched random averages of spin products. From the comparison of the free energies we have introduced, we find the (one-step) RSB solution is more stable than the replica-symmetric (RS) solution for arbitrary clusters so long as the former exists. Analytic solutions of the Bethe cluster approximation suggest that the RSB effects are rather enhanced by short-range fluctuations.

Statistical mechanics of the multi-constraint continuous knapsack problem

We apply the replica analysis established by Gardner to the multi-constraint continuous knapsack problem,which is one of the linear programming problems and a most fundamental problem in the field of operations research (OR). For a large problem size, we analyse the space of solution and its volume, and estimate the optimal number of items to go into the knapsack as a function of the number of constraints. We study the stability of the replica symmetric (RS) solution and find that the RS calculation cannot estimate the optimal number of items in knapsack correctly if many constraints are required.In order to obtain a consistent solution in the RS region,we try the zero entropy approximation for this continuous solution space and get a stable solution within the RS ansatz.On the other hand, in replica symmetry breaking (RSB) region, the one step RSB solution is found by Parisi's scheme. It turns out that this problem is closely related to the problem of optimal storage capacity and of generalization by maximum stability rule of a spherical perceptron.

Analysis of Learning from Stochastic Rules in the Framework of Replica Symmetry Breaking

We study learning from examples by a perceptron when a return to an example is given by a stochastic relation which is represented by the conditional probability distribution P ( u ), where u is proportional to the inner product between the optimal synaptic weight vector and the example vector. The problem is analyzed by replica method in the case of spherical weights. Since the replica symmetric(RS) solution turns out to be unstable for stochastic cases, we consider the one-step replica symmetry breaking (RSB) solution. We investigate the asymptotic behavior of the learning curve as \(\alpha \equiv \frac{p}{N} \rightarrow \infty\), where p is the number of samples and N is the dimension of the synaptic weights. The average generalization error e g is expressed as (e g - e min ) ∝α - γ in the asymptotic region. For the minimum-error algorithm, we find \(\gamma=\frac{1+\delta}{1+3\delta}\) for the RS solution and \(\gamma=\frac{1+\delta}{1+2\delta}\) for the one-step RSB solution in the case of the functio...

The infinite-ranged Potts spin glass model

The s-state Potts model with a gaussian distribution of pair couplings is found to possess, in the infinite-range limit, a replica symmetric (RS) solution that exhibits a spin glass transition which appears to be second order for s 2, the stability of the PM phase breaks down above the transition temperature indicated by the RS solution suggesting a first-order transition which, however, cannot be located by the RS solution.