Swendsen-Wang Algorithm Research Articles

GPU acceleration of Swendsen–Wang dynamics

When simulating a lattice system near its critical temperature, local algorithms for modeling the system’s evolution can introduce very large autocorrelation times into sampled data. This critical slowing down places restrictions on the analysis that can be completed in a timely manner of the behavior of systems around the critical point. Because it is often desirable to study such systems around this point, a new algorithm must be introduced. Therefore, we turn to cluster algorithms, such as the Swendsen–Wang algorithm and the Wolff clustering algorithm. They incorporate global updates which generate new lattice configurations with little correlation to previous states, even near the critical point. We look to accelerate the rate at which these algorithm are capable of running by implementing and benchmarking a parallel implementation of each algorithm designed to run on GPUs under NVIDIA’s CUDA framework. A 17 and 90 fold increase in the computational rate was, respectively, experienced when measured against the equivalent algorithm implemented in serial code.

The Swendsen–Wang dynamics on trees

AbstractThe Swendsen–Wang algorithm is a sophisticated, widely‐used Markov chain for sampling from the Gibbs distribution for the ferromagnetic Ising and Potts models. This chain has proved difficult to analyze, due in part to its global nature. We present optimal bounds on the convergence rate of the Swendsen–Wang algorithm for the complete ‐ary tree. Our bounds extend to the non‐uniqueness region and apply to all boundary conditions. We show that the spatial mixing conditions known asvariance mixingandentropy mixingimply spectral gap and mixing time, respectively, for the Swendsen–Wang dynamics on the ‐ary tree. We also show that these bounds are asymptotically optimal. As a consequence, we establish mixing for the Swendsen–Wang dynamics forallboundary conditions throughout (and beyond) the tree uniqueness region. Our proofs feature a novel spectral view of the variance mixing condition and utilize recent work on block factorization of entropy.

Comparison of cluster algorithms for the bond-diluted Ising model.

Monte Carlo cluster algorithms are popular for their efficiency in studying the Ising model near its critical temperature. We might expect that this efficiency extends to the bond-diluted Ising model. We show, however, that this is not always the case by comparing how the correlation times τ_{w} and τ_{sw} of the Wolff and Swendsen-Wang cluster algorithms scale as a function of the system size L when applied to the two-dimensional bond-diluted Ising model. We demonstrate that the Wolff algorithm suffers from a much longer correlation time than in the pure Ising model, caused by isolated (groups of) spins which are infrequently visited by the algorithm. With a simple argument we prove that these cause the correlation time τ_{w} to be bounded from below by L^{z_{w}} with a dynamical exponent z_{w}=γ/ν≈1.75 for a bond concentration p<1. Furthermore, we numerically show that this lower bound is actually taken for several values of p in the range 0.5<p<1. Moreover, we show that the Swendsen-Wang algorithm does not suffer from the same problem. Consequently, it has a much shorter correlation time, shorter than in the pure Ising model even. Numerically at p=0.6, we find that its dynamical exponent is z_{sw}=0.09(4).

Feature selection and classification over the network with missing node observations.

Jointly analyzing transcriptomic data and the existing biological networks can yield more robust and informative feature selection results, as well as better understanding of the biological mechanisms. Selecting and classifying node features over genome-scale networks has become increasingly important in genomic biology and genomic medicine. Existing methods have some critical drawbacks. The first is they do not allow flexible modeling of different subtypes of selected nodes. The second is they ignore nodes with missing values, very likely to increase bias in estimation. To address these limitations, we propose a general modeling framework for Bayesian node classification (BNC) with missing values. A new prior model is developed for the class indicators incorporating the network structure. For posterior computation, we resort to the Swendsen-Wang algorithm for efficiently updating class indicators. BNC can naturally handle missing values in the Bayesian modeling framework, which improves the node classification accuracy and reduces the bias in estimating gene effects. We demonstrate the advantages of our methods via extensive simulation studies and the analysis of the cutaneous melanoma dataset from The Cancer Genome Atlas.

Loop-Cluster Coupling and Algorithm for Classical Statistical Models.

Potts spin systems play a fundamental role in statistical mechanics and quantum field theory and can be studied within the spin, the Fortuin-Kasteleyn (FK) bond or the q-flow (loop) representation. We introduce a Loop-Cluster (LC) joint model of bond-occupation variables interacting with q-flow variables and formulate an LC algorithm that is found to be in the same dynamical universality as the celebrated Swendsen-Wang algorithm. This leads to a theoretical unification for all the representations, and numerically, one can apply the most efficient algorithm in one representation and measure physical quantities in others. Moreover, by using the LC scheme, we construct a hierarchy of geometric objects that contain as special cases the q-flow clusters and the backbone of FK clusters, the exact values of whose fractal dimensions in two dimensions remain as an open question. Our work not only provides a unified framework and an efficient algorithm for the Potts model but also brings new insights into the rich geometric structures of the FK clusters.

Transforming generalized Ising models into Boltzmann machines

We find an exact mapping from the generalized Ising models with many-spin interactions to equivalent Boltzmann machines, i.e., the models with only two-spin interactions between physical and auxiliary binary variables accompanied by local external fields. More precisely, the appropriate combination of the algebraic transformations, namely the star-triangle and decoration-iteration transformations, allows one to express the model in terms of fewer-spin interactions at the expense of the degrees of freedom. Furthermore, the benefit of the mapping in Monte Carlo simulations is discussed. In particular, we demonstrate that the application of the method in conjunction with the Swendsen-Wang algorithm drastically reduces the critical slowing down in a model with two- and three-spin interactions on the Kagomé lattice.

A Bayesian Nonparametric Model for Unsupervised Joint Segmentation of a Collection of Images

Jointly segmenting a collection of images with shared classes is expected to yield better results than single-image based methods, due to the use of the shared statistical information across different images. This paper proposes a Bayesian approach for tackling this problem. As a first contribution, the proposed method relies on a new prior distribution for the class labels, which combines a hierarchical Dirichlet process (HDP) with a Potts model. The latter classically favors a spatial dependency, whereas the HDP is a Bayesian nonparametric model that allows the number of classes to be inferred automatically. The HDP also explicitly induces a sharing of classes between the images. The resulting posterior distribution of the labels is not analytically tractable and can be explored using a standard Gibbs sampler. However, such a sampling strategy is known to have poor mixing properties for high-dimensional data. To alleviate this issue, the second contribution reported in this paper consists of an adapted generalized Swendsen-Wang algorithm which is a sampling technique that improves the exploration of the posterior distribution. Finally, since the inferred segmentation depends on the values of the hyperparameters, the third contribution aims at adjusting them while sampling the posterior label distribution by resorting to an original combination of two sequential Monte Carlo samplers. The proposed methods are validated on both simulated and natural images from databases.

Random cluster dynamics for the Ising model is rapidly mixing

We show that the mixing time of Glauber (single edge update) dynamics for the random cluster model at $q=2$ is bounded by a polynomial in the size of the underlying graph. As a consequence, the Swendsen-Wang algorithm for the ferromagnetic Ising model at any temperature has the same polynomial mixing time bound.

Modeling and inferring 2.1D sketch with mixed Markov random field

This paper presents a method of computing a 2.1D sketch (i.e., layered image representation) from a single image with mixed Markov random field (MRF) under the Bayesian framework. Our model consists of three layers: the input image layer, the graphical representation layer of the computed 2D atomic regions and 3-degree junctions (such as T or arrow junctions), and the 2.1D sketch layer. There are two types of vertices in the graphical representation of the 2D entities: (i) regions, which act as the vertices found in traditional MRF, and (ii) address variables assigned to the terminators decomposed from the 3-degree junctions, which are a new type of vertices for the mixed MRF. We formulate the inference problem as computing the 2.1D sketch from the 2D graphical representation under the Bayesian framework, which consists of two components: (i) region layering/coloring based on the Swendsen-Wang cuts algorithm, which infers partial occluding order of regions, and (ii) address variable assignments based on Gibbs sampling, which completes the open bonds of the terminators of the 3-degree junctions. The proposed method is tested on the D-Order dataset, the Berkeley segmentation dataset and the Stanford 3D dataset. The experimental results show the efficiency and robustness of our approach.

Nonequilibrium behaviors of the three-dimensional Heisenberg model in the Swendsen-Wang algorithm.

Recently, it was shown [Y. Nonomura, J. Phys. Soc. Jpn. 83, 113001 (2014)JUPSAU0031-901510.7566/JPSJ.83.113001] that the nonequilibrium critical relaxation of the two-dimensional (2D) Ising model from a perfectly ordered state in the Wolff algorithm is described by stretched-exponential decay, and a universal scaling scheme was found to connect nonequilibrium and equilibrium behaviors. In the present study we extend these findings to vector spin models, and the 3D Heisenberg model could be a typical example. To evaluate the critical temperature and critical exponents precisely using the above scaling scheme, we calculate nonequilibrium ordering from the perfectly disordered state in the Swendsen-Wang algorithm, and we find that the critical ordering process is described by stretched-exponential growth with a comparable exponent to that of the 3D XY model. The critical exponents evaluated in the present study are consistent with those in previous studies.

Ferromagnetic Potts Model: Refined #BIS-hardness and Related Results

Recent results establish for 2-spin antiferromagnetic systems that the computational complexity of approximating the partition function on graphs of maximum degree D undergoes a phase transition that coincides with the uniqueness phase transition on the infinite D-regular tree. For the ferromagnetic Potts model we investigate whether analogous hardness results hold. Goldberg and Jerrum showed that approximating the partition function of the ferromagnetic Potts model is at least as hard as approximating the number of independent sets in bipartite graphs (#BIS-hardness). We improve this hardness result by establishing it for bipartite graphs of maximum degree D. We first present a detailed picture for the phase diagram for the infinite D-regular tree, giving a refined picture of its first-order phase transition and establishing the critical temperature for the coexistence of the disordered and ordered phases. We then prove for all temperatures below this critical temperature that it is #BIS-hard to approximate the partition function on bipartite graphs of maximum degree D. As a corollary, it is #BIS-hard to approximate the number of k-colorings on bipartite graphs of maximum degree D when k <= D/(2 ln D). The #BIS-hardness result for the ferromagnetic Potts model uses random bipartite regular graphs as a gadget in the reduction. The analysis of these random graphs relies on recent connections between the maxima of the expectation of their partition function, attractive fixpoints of the associated tree recursions, and induced matrix norms. We extend these connections to random regular graphs for all ferromagnetic models and establish the Bethe prediction for every ferromagnetic spin system on random regular graphs. We also prove for the ferromagnetic Potts model that the Swendsen-Wang algorithm is torpidly mixing on random D-regular graphs at the critical temperature for large q.

Critical nonequilibrium relaxation in the Swendsen-Wang algorithm in the Berezinsky-Kosterlitz-Thouless and weak first-order phase transitions.

Recently we showed that the critical nonequilibrium relaxation in the Swendsen-Wang algorithm is widely described by the stretched-exponential relaxation of physical quantities in the Ising or Heisenberg models. Here we make a similar analysis in the Berezinsky-Kosterlitz-Thouless phase transition in the two-dimensional (2D) XY model and in the first-order phase transition in the 2D q=5 Potts model and find that these phase transitions are described by the simple exponential relaxation and power-law relaxation of physical quantities, respectively. We compare the relaxation behaviors of these phase transitions with those of the second-order phase transition in the three- and four-dimensional XY models and in the 2D q-state Potts models for 2≤q≤4 and show that the species of phase transitions can be clearly characterized by the present analysis. We also compare the size dependence of relaxation behaviors of the first-order phase transition in the 2D q=5 and 6 Potts models and propose a quantitative criterion on "weakness" of the first-order phase transition.

A Localized Implementation of the Iterative Proportional Scaling Procedure for Gaussian Graphical Models

In this article, we propose localized implementations of the iterative proportional scaling (IPS) procedure by the strategy of partitioning cliques for computing maximum likelihood estimations in large Gaussian graphical models. We first divide the set of cliques into several nonoverlapping and nonempty blocks, and then adjust clique marginals in each block locally. Thus, high-order matrix operations can be avoided and the IPS procedure is accelerated. We modify the Swendsen–Wang Algorithm and apply the simulated annealing algorithm to find an approximation to the optimal partition which leads to the least complexity. This strategy of partitioning cliques can also speed up the existing IIPS and IHT procedures. Numerical experiments are presented to demonstrate the competitive performance of our new implementations and strategies.

Critical Binder cumulant and universality: Fortuin-Kasteleyn clusters and order-parameter fluctuations.

We investigate the dependence of the critical Binder cumulant of the magnetization and the largest Fortuin-Kasteleyn cluster on the boundary conditions and aspect ratio of the underlying square Ising lattices. By means of the Swendsen-Wang algorithm, we generate numerical data for large system sizes and we perform a detailed finite-size scaling analysis for several values of the aspect ratio r, for both periodic and free boundary conditions. We estimate the universal probability density functions of the largest Fortuin-Kasteleyn cluster and we compare it to those of the magnetization at criticality. It is shown that these probability density functions follow similar scaling laws, and it is found that the values of the critical Binder cumulant of the largest Fortuin-Kasteleyn cluster are upper bounds to the values of the respective order-parameter's cumulant, with a splitting behavior for large values of the aspect ratio. We also investigate the dependence of the amplitudes of the magnetization and the largest Fortuin-Kasteleyn cluster on the aspect ratio and boundary conditions. We find that the associated exponents, describing the aspect-ratio dependencies, are different for the magnetization and the largest Fortuin-Kasteleyn cluster, but in each case are independent of boundary conditions.

GPU-based Swendsen–Wang multi-cluster algorithm for the simulation of two-dimensional classical spin systems

We present the GPU calculation with the common unified device architecture (CUDA) for the Swendsen–Wang multi-cluster algorithm of two-dimensional classical spin systems. We adjust the two connected component labeling algorithms recently proposed with CUDA for the assignment of the cluster in the Swendsen–Wang algorithm. Starting with the q-state Potts model, we extend our implementation to the system of vector spins, the q-state clock model, with the idea of embedded cluster. We test the performance, and the calculation time on GTX580 is obtained as 2.51 nsec per a spin flip for the q=2 Potts model (Ising model) and 2.42 nsec per a spin flip for the q=6 clock model with the linear size L=4096 at the critical temperature, respectively. The computational speed for the q=2 Potts model on GTX580 is 12.4 times as fast as the calculation speed on a current CPU core. That for the q=6 clock model on GTX580 is 35.6 times as fast as the calculation speed on a current CPU core.

Tight bounds for mixing of the Swendsen–Wang algorithm at the Potts transition point

We study two widely used algorithms for the Potts model on rectangular subsets of the hypercubic lattice \({\mathbb{Z}^{d}}\)—heat bath dynamics and the Swendsen–Wang algorithm—and prove that, under certain circumstances, the mixing in these algorithms is torpid or slow. In particular, we show that for heat bath dynamics throughout the region of phase coexistence, and for the Swendsen–Wang algorithm at the transition point, the mixing time in a box of side length L with periodic boundary conditions has upper and lower bounds which are exponential in L d-1. This work provides the first upper bound of this form for the Swendsen–Wang algorithm, and gives lower bounds for both algorithms which significantly improve the previous lower bounds that were exponential in L/(log L)2.

A random cluster algorithm for some continuous Markov random fields

Based on a random cluster representation, the Swendsen–Wang algorithm for the Ising and Potts distributions is extended to a class of continuous Markov random fields. The algorithm can be described briefly as follows. A given configuration is decomposed into clusters. Probabilities for flipping the values of the random variables in each cluster are calculated. According to these probabilities, values of all the random variables in each cluster will be either updated or kept unchanged and this is done independently across the clusters. A new configuration is then obtained. We will show through a simulation study that, like the Swendsen–Wang algorithm in the case of Ising and Potts distributions, the cluster algorithm here also outperforms the Gibbs sampler in beating the critical slowing down for some strongly correlated Markov random fields.

THE STUDY OF PHASE TRANSITION BY PERIODIC DISTRIBUTION OF BIMODAL BONDS IN 2D POTTS MODEL

We have studied the influence of the distribution of bimodal bonds on the phase transition in two-dimensional 8-state Potts model by the recently proposed Wang–Landau (WL) and the Swendsen–Wang (SW) algorithm. All simulations and measurements are done for r = 0.5. Physical quantities such as energy density and specific heat are evaluated at all temperatures. We have also obtained the probability distributions of the energy in order to monitor the transitions. We have observed that some cases of the periodically arranged bond distributions show a single peak, and some cases show double or triple peaks in the specific heat. Besides, it seems that the appearing of these peaks in the specific heat relates to a blocking procedure for periodicity. When the number of interaction pairs between the bimodal bonds is increased on the lattice with the blocking procedure, one can observe a single peak, otherwise, one can observe a double or triple peaks in the specific heat. From the point of view of simulation methods, the WL algorithm also works efficiently in the simulation of the system for a periodically arranged bond distribution as well as the SW algorithm.

Order- N cluster Monte Carlo method for spin systems with long-range interactions

An efficient O ( N ) cluster Monte Carlo method for Ising models with long-range interactions is presented. Our novel algorithm does not introduce any cutoff for interaction range and thus it strictly fulfills the detailed balance. The realized stochastic dynamics is equivalent to that of the conventional Swendsen–Wang algorithm, which requires O ( N 2 ) operations per Monte Carlo sweep if applied to long-range interacting models. In addition, it is shown that the total energy and the specific heat can also be measured in O ( N ) time. We demonstrate the efficiency of our algorithm over the conventional method and the O ( N log N ) algorithm by Luijten and Blöte. We also apply our algorithm to the classical and quantum Ising chains with inverse-square ferromagnetic interactions, and confirm in a high accuracy that a Kosterlitz–Thouless phase transition, associated with a universal jump in the magnetization, occurs in both cases.

On the universality class of the 3d Ising model with long-range-correlated disorder

We analyze a controversial topic about the universality class of the three-dimensional Ising model with long-range-correlated disorder. Whereas both theoretical and numerical studies agree on the validity of the extended Harris criterion [A. Weinrib, B.I. Halperin, Phys. Rev. B 27 (1983) 413] and indicate the existence of a new universality class, numerical values of the critical exponents found so far differ considerably. To resolve this discrepancy we perform extensive Monte Carlo simulations of a 3d Ising model with non-magnetic impurities being arranged in a form of lines along randomly chosen axes of a lattice. The Swendsen–Wang algorithm is used alongside with a histogram reweighting technique and finite-size scaling analysis to evaluate the values of critical exponents governing magnetic phase transition. Our estimates for these exponents differ from both previous numerical simulations and are in favor of a non-trivial dependency of the critical exponents on the peculiarities of long-range correlation decay.