Mean Field Annealing Research Articles

Thermodynamic quantities of two-dimensional Ising models obtained by noisy mean-field annealing and a coherent Ising machine

Thermodynamic quantities of two-dimensional Ising models obtained by noisy mean-field annealing and a coherent Ising machine

Mean field approximation for solving QUBO problems

The Quadratic Unconstrained Binary Optimization (QUBO) problem is NP-hard. Some exact methods like the Branch-and-Bound algorithm are suitable for small problems. Some approximations like stochastic simulated annealing for discrete variables or mean-field annealing for continuous variables exist for larger ones, and quantum computers based on the quantum adiabatic annealing principle have also been developed. Here we show that the mean-field approximation of the quantum adiabatic annealing leads to equations similar to those of thermal mean-field annealing. However, a new type of sigmoid function replaces the thermal one. The new mean-field quantum adiabatic annealing can replicate the best-known cut values on some of the popular benchmark Maximum Cut problems.

Scaling advantage of chaotic amplitude control for high-performance combinatorial optimization

The development of physical simulators, called Ising machines, that sample from low energy states of the Ising Hamiltonian has the potential to transform our ability to understand and control complex systems. However, most of the physical implementations of such machines have been based on a similar concept that is closely related to relaxational dynamics such as in simulated, mean-field, chaotic, and quantum annealing. Here we show that dynamics that includes a nonrelaxational component and is associated with a finite positive Gibbs entropy production rate can accelerate the sampling of low energy states compared to that of conventional methods. By implementing such dynamics on field programmable gate array, we show that the addition of nonrelaxational dynamics that we propose, called chaotic amplitude control, exhibits exponents of the scaling with problem size of the time to find optimal solutions and its variance that are smaller than those of relaxational schemes recently implemented on Ising machines.

Multi-Robot Routing Problem with Min–Max Objective

In this paper, we study the “Multi-Robot Routing problem” with min–max objective (MRR-MM) in detail. It involves the assignment of sequentially ordered tasks to robots such that the maximum cost of the slowest robot is minimized. The problem description, the different types of formulations, and the methods used across various research communities are discussed in this paper. We propose a new problem formulation by treating this problem as a permutation matrix. A comparative study is done between three methods: Stochastic simulated annealing, deterministic mean-field annealing, and a heuristic-based graph search method. Each method is investigated in detail with several data sets (simulation and real-world), and the results are analysed and compared with respect to scalability, computational complexity, optimality, and its application to real-world scenarios. The paper shows that the heuristic method produces results very quickly with good scalability. However, the solution quality is sub-optimal. On the other hand, when optimal or near-optimal results are required with considerable computational resources, the simulated annealing method proves to be more efficient. However, the results show that the optimal choice of algorithm depends on the dataset size and the available computational budget. The contribution of the paper is three-fold: We study the MRR-MM problem in detail across various research communities. This study also shows the lack of inter-research terminology that has led to different names for the same problem. Secondly, formulating the task allocation problem as a permutation matrix formulation has opened up new approaches to solve this problem. Thirdly, we applied our problem formulation to three different methods and conducted a detailed comparative study using real-world and simulation data.

A Boltzmann machine for high-resolution prestack seismic inversion

Seismic inversion is one popular approach that aims at predicting some indicative properties to support the geologic interpretation process. Existing inversion techniques indicate weaknesses when dealing with complex geologic area, where the uncertainties arise from the guiding model, which are provided by the interpreters. We have developed a prestack seismic inversion algorithm using a machine-learning algorithm called the Boltzmann machine. Unlike common inversion approaches, this stochastic neural network does not require a starting model at the beginning of the process to guide the solution; however, low-frequency models are required to convert the inversion-derived reflectivity terms to the absolute elastic P- and S-impedance as well as density. Our algorithm incorporates a single-layer Hopfield neural network whose neurons can be treated as the desired reflectivity terms. The optimization process seeks the global minimum solution by combining the network with a stochastic model update from the mean-field annealing algorithm. Also, we use a Z-shaped sample sorting scheme and the first-order Tikhonov regularization to improve the lateral continuity of the results and to stabilize the inversion process. The algorithm is applied to a field 2D data set to invert for high-resolution indicative P- and S-impedance sections to better capture some features away from the reservoir zone. The resulting models are strongly supported by the well results and reveal some realistic features that are not clearly displayed in the model-based deterministic inversion result. In combination with well-log analyses, the new features appear to be a good prospect for hydrocarbon.

Dual mean field search for large scale linear and quadratic knapsack problems

An implementation of mean field annealing to deal with large scale linear and non linear binary optimization problems is given. Mean field annealing is based on the analogy between combinatorial optimization and interacting physical systems at thermal equilibrium. Specifically, a mean field approximation of the Boltzmann distribution given by a Lagrangian that encompass the objective function and the constraints is calculated. The original discrete task is in this way transformed into a continuous variational problem. In our version of mean field annealing, no temperature parameter is used, but a good starting point in the dual space is given by a “thermodynamic limit” argument. The method is tested in linear and quadratic knapsack problems with sizes that are considerably larger than those used in previous studies of mean field annealing. Dual mean field annealing is capable to find high quality solutions in running times that are orders of magnitude shorter than state of the art algorithms. Moreover, as may be expected for a mean field theory, the solutions tend to be more accurate as the number of variables grow.

Set-valued functional neural mapping and inverse system approximation

This work explores set-valued functional neural mapping and inverse system approximation by learning state-regulated multilayer neural networks. Multilayer neural organization is extended to recruit a discrete regulating state in addition to predictive attributes in the input layer. The network mapping regulated over a set of finite discrete states translates a predictor to many targets. Stimuli and responses clamped at visible units are assumed as mixtures of paired predictors and targets sampled from many joined elementary mappings. Unknown regulating states are related to missing exclusive memberships of paired training data to distinct sources. Learning a state-regulated neural network for set-valued mapping approximation involves retrieving unknown exclusive memberships and refining network interconnections. The learning process is realized by a hybrid of mean field annealing and Levenberg–Marquardt methods that simultaneously track expectations of unknown regulating states and optimal interconnections among consecutive layers along a physical-like annealing process. Numerical simulations show the presented learning process well reconstructing many joined elementary functions for set-valued functional mapping and inverse system approximation.

To converge more quickly and effectively—Mean field annealing based optimal path selection in WMN

Multi-constrained path selection with optimization aims at finding an optimal path that satisfies a set of QoS parameters, as an NP-hard problem, which is also a big challenge for wireless mesh networks. Heuristic algorithms with polynomial and pseudo-polynomial-time complexities are often used to deal with this problem. However, existing methods either have analyses in specific network models or cannot support the change of network topology well. Thus, multi-constrained QoSR in emerging wireless mesh networking is worth investigating further. In this paper, we propose a novel Routing Algorithm based on Mean Field Annealing (MFA_RA) to solve this problem. MFA_RA first uses a function of two QoS parameters, wireless link’s delay and transmission success rate, as the cost function, and then seeks to find a feasible path by mean field annealing. Moreover, a set of deterministic equations are utilized to replace the stochastic process in simulated annealing (SA), and a saddle point approximation is adopted for the calculation of the stationary probability distribution at equilibrium. Extensive simulation results demonstrate that MFA_RA is an effective algorithm and can outperform other heuristic algorithms.

A Multi-objective Resource-constrained Project-scheduling Problem Using Mean Field Annealing Neural Networks

The planning and scheduling activities are viewed profoundly important to generate successful plans and to maximize the utilization of scarce resources. Moreover, real life planning problems often involve several objectives that should be simultaneously optimized and real world environment is usually characterized by uncertain and incontrollable information. Thus, finding feasible and efficient plans is a considerable challenge. In this respect, the Multi-Objective Resource-Constrained Project-Scheduling problem (MRCPSP) tries to schedule activities and allocate resources in order to find an efficient course of actions to help the project manager and to optimize several optimization criteria. In this research, A Potts mean field feedback artificial neural network algorithm is developed and explored for the multi-objective resource constrained project scheduling problem. A convenient encoding of inequality constraints is achieved by means of multi-linear penalty function. An approximate energy minimum is obtained by iterating a set of Potts means field equation, is combination with annealing. Priority rule-based heuristics are the most widely used scheduling methods though their performance depends on the characteristics of the projects. To overcome this deficiency, a Potts mean field feedback artificial neural network is designed and integrated into the scheduling scheme so as to automatically select the suitable activity for each stage of project scheduling. Testing on Paterson’s classic test problems and comparison with other exact method how that the proposed Potts mean field annealing neural network based heuristic is able to improve the performance of project scheduling.

Annealed cooperative–competitive learning of Mahalanobis-NRBF neural modules for nonlinear and chaotic differential function approximation

This work explores annealed cooperative–competitive learning of multiple modules of Mahalanobis normalized radial basis functions (NRBF) with applications to nonlinear function approximation and chaotic differential function approximation. A multilayer neural network is extended to be composed of multiple Mahalanobis-NRBF modules. Each module activates normalized outputs of radial basis functions, determining Mahalanobis radial distances based on its own adaptable weight matrix. An essential cooperative scheme well decomposes learning a multi-module network to sub-tasks of learning individual modules. Adaptable network interconnections are asynchronously updated module-by-module based on annealed cooperative–competitive learning for function approximation under a physical-like mean-field annealing process. Numerical simulations show outstanding performance of annealed cooperative–competitive learning of a multi-module Mahalanobis-NRBF network for nonlinear function approximation and long term look-ahead prediction of chaotic time series.

A Meta-Learning Approach based on Mean Field Genetic Algorithms

Mean Field Genetic Algorithm (MGA) is a hybrid algorithm of Mean Field Annealing (MFA) and Simulated annealing-like Genetic Algorithm (SGA). It combines benefit of rapid convergence property of MFA and effective genetic operations of SGA. This paper presents an approach for building a multi-classifier system in a MGA-based inductive learning environment. Multiple base classifiers are combined to build a multi-classifier system. A base classifier consists of a general classifier and a meta-classifier. The general classifier performs regular classification task. The meta-classifier evaluates classification result of its general classifier and decides whether the base classifier participates into a final decisionmaking process or not. The paper discusses our approach in details and presents some empirical results that show the improvement we can achieve with our approach.

A novel and effective multi-constrained QoS routing scheme in WMNs

Multi-constrained quality of service (QoS) routing aims at finding an optimal path that satisfies a set of QoS parameters, as an NP complete problem, which is also a big challenge for wireless mesh networks (WMNs). Heuristic algorithms with polynomial and pseudo-polynomial-time complexities are often used to deal with this problem. However, existing solutions, most of which suffered either from excessive computational complexities or from low performance, were proposed only for wired networks and cannot be used directly in wireless mesh networks. In this paper, we propose a novel routing scheme based on mean field annealing (MFA-RS) to solve this problem. MFA-RS first uses a function of two QoS parameters, wireless link’s delay and transmission success rate as the cost function, and then seeks to find a feasible path by MFA. Because MFA-RS uses a set of deterministic equations to replace the stochastic process in simulated annealing (SA) and uses saddle point approximation in the calculation of the stationary probability distribution at equilibrium, the convergence time is much less than the routing scheme based on SA (SA-RS). Simulation results demonstrate that MFA-RS is an effective algorithm and is very fit for WMNs.

Optimized Local Relocation for VLSI Circuit Modification Using Mean-Field Annealing

In this paper, a fast migration method is proposed. Our method executes local relocation on a model placement where an additional module is added to it for modification with a minimum number of displacements. This method is based on mean-field annealing (MFA), which produces a solution as reliable as a previously used method called simulated annealing. The proposed method requires substantially less time and hardware, and it is less sensitive to the initial and final temperatures. In addition, the solution runtime is mostly independent of the size and complexity of the input model placement. Our proposed MFA algorithm is optimized by enabling module rotation inside an energy function called permissible distances preservation energy. This, in turn, allows more options in moving the engaged modules. Finally, a three-phase cooling process governs the convergence of problem variables called neurons or spins.

Distance‐based population classification software using mean‐field annealing

We describe a distance-based clustering method using a proximity matrix of genetic distances to partition populations into genetically similar groupings. The optimization heuristic mean-field annealing (MFA) was used to find locally optimal solutions where exhaustive search was not possible. To illustrate this method, we analysed both simulated and real data sets. Simulated data indicated that MFA successfully differentiated population groups, even with small F(ST) values, as long as there was separation of within and between group distances. Reanalysis of microsatellite data from various human populations using mean-fields found similar ethnic groups corresponding to major geographic regions reported by Rosenberg et al. (2002) who used the model-based computer program Structure. However, with MFA, the Kalash population was found to group with other Central/South Asian populations instead of being the only member of its own genetic cluster. Europe/Middle East populations formed a separate group from Central/South Asian populations instead of being a single group in the Structure analysis. The MFA analysis determined the greatest genetic distances (largest mean intracluster distance) occurred in native American populations, identifying three groups instead of only one found with Structure. For conservation purposes, it is not only important to identify genetically similar groupings but also to determine the relative level of genetic differentiation captured within these groups. To illustrate this, we compare two separate MFA analyses of Chinook salmon (Oncorhynchus tshawytscha) populations from British Columbia, Canada. The software called PORGS-MFA used in this article can be downloaded from http://www.pac.dfo-mpo.gc.ca/science/facilities-installations/pbs-sbp/mgl-lgm/apps/porgs/index-eng.htm.

Mean Field Annealing for Pattern Classification using different response functions: A Comparative Approach.

Mean Field Annealing for Pattern Classification using different response functions: A Comparative Approach.

A Distributed Stock Cutting using Mean Field Annealing and Genetic Algorithm

The composite stock cutting problem is defined as allocating rectangular and irregular patterns onto a large composite stock sheet of finite dimensions in such a way that the resulting scrap will be minimized. In this paper, we introduce a novel approach to hybrid optimization algorithm called MGA in MPI (Message Passing Interface) environments. The proposed MGA combines the benefit of rapid convergence property of Mean Field Annealing and the effective genetic operations. This paper also proposes the efficient data structures for pattern related information.

채널배선 문제에 대한 분산 평균장 유전자 알고리즘

본 논문에서는 MPI(Message Passing Interface) 환경 하에서 채널배선 문제에 대한 분산 평균장 유전자 알고리즘(MGA, Mean field Genetic Algorithm)이라는 새로운 최적화 알고리즘을 제안한다. 분산 MGA는 평균장 어닐링(MFA, Mean Field Annealing)과 시뮬레이티드 어닐링 형태의 유전자 알고리즘(SGA, Simulated annealing-like Genetic Algorithm)을 결합한 경험적 알고리즘이다. 평균장 어닐링의 빠른 평형상태 도달과 유전자 알고리즘의 다양하고 강력한 연산자를 합성하여 최적화 문제를 효율적으로 해결하였다. 제안된 분산 MGA를 VLSI 설계에서 중요한 주제인 채널 배선문제에 적용하여 실험한 결과 기존의 GA를 단독으로 사용하였을 때보다 최적해에 빠르게 도달하였다. 또한 분산 알고리즘은 순차 알고리즘에서의 최적해 수렴 특성을 해치지 않으면서 문제의 크기에 대하여 선형적인 수행시간 단축을 나타냈다. In this paper, we introduce a novel approach to optimization algorithm which is a distributed Mean field Genetic algorithm (MGA) implemented in MPI(Message Passing Interface) environments. Distributed MGA is a hybrid algorithm of Mean Field Annealing(MFA) and Simulated annealing-like Genetic Algorithm(SGA). The proposed distributed MGA combines the benefit of rapid convergence property of MFA and the effective genetic operations of SGA. The proposed distributed MGA is applied to the channel routing problem, which is an important issue in the automatic layout design of VLSI circuits. Our experimental results show that the composition of heuristic methods improves the performance over GA alone in terms of mean execution time. It is also proved that the proposed distributed algorithm maintains the convergence properties of sequential algorithm while it achieves almost linear speedup as the problem size increases.

On image reconstruction algorithms for binary electromagnetic geotomography

In this paper we discuss the selected image reconstruction methods of binary tomography in the context of their application to geophysical imaging. We restrict our considerations to a discrete version of high-frequency electromagnetic geotomography, which we label as Binary Electromagnetic Geotomography (BEG). Basically, such an imaging technique may be applied to detect subsurface anomalies (air-filled voids) whose attenuation coefficient is very low (nearly zero-value) and considerably different from that for the background. The assumption for a binary representation of the image to be reconstructed substantially relaxes image reconstruction problems related to ill-posedness that comes from an intrinsic limitation of an angular range of projections. We test two algorithms for binary tomography, where the penalty term is based on the Markov Random Field (MRF) model. The mean-field reference distribution and mean-field annealing are applied to estimate the global maximizer of the Gibbs–Boltzmann distribution associated with the objective function. We also apply the projected gradient algorithm that uses a binary steering. Very efficient implementations of the algorithms are also given. The numerical results are presented for noise-free, noisy, and real data.

Hand-written digits recognition by graph matching and annealing neural networks

Mean field annealing(MFA) is a promising tool in optimization and a neural network model based on the graph matching have attracted attention due to a number of benefits over conventional recogniti...

A hybrid simulated annealing for solving an extended model of dynamic cellular manufacturing system

This paper develops a mixed-integer programming model to design the cellular manufacturing systems (CMSs) under dynamic environment. In dynamic environment, the product mix and part demand change under a multi-period planning horizon. Thus, the best designed cells for one period may not be efficient for subsequent periods and reconfiguration of cells is required. Reconfiguration may involve adding, removing or relocating machines; it may also involve a change in processing rout of part types from a period to another. The advantages of the proposed model are as follows: considering the batch inter/intra-cell material handling by assuming the sequence of operations, considering alternative process plans for part types, and considering machine replication. The main constraints are maximal cell size and machine time-capacity. The objective is to minimize the sum of the machine constant and variable costs, inter- and intra-cell material handling, and reconfiguration costs. An efficient hybrid meta-heuristic based on mean field annealing (MFA) and simulated annealing (SA) so-called MFA–SA is used to solve the proposed model. In this case, MFA technique is applied to generate a good initial solution for SA. The obtained results show that the quality of the solutions obtained by MFA–SA is better than classical SA, especially for large-sized problems.