Continuous Estimation Of Distribution Algorithm Research Articles

Random mask-based estimation of the distribution algorithm for stacked auto-encoder one-step pre-training

The deep learning techniques have received great achievements in computer vision, natural language processing, etc. The success of deep neural networks depends on the sufficient training of parameters. The traditional way of neural network training is a gradient-based algorithm, which suffers the disadvantage of gradient disappearing, especially for the deeper neural network. Recently, a heuristic algorithm has been proposed for deeper neural network optimization. In this paper, a random mask and elitism univariate continuous estimation of distribution algorithm based on the Gaussian model is proposed to pre-train staked auto-encoder, and then a Stochastic Gradient Descent (SGD) based fine-tuning process is carried out for local searching. In the improved estimation of the distribution algorithm, two individual update strategies are defined; one group of individuals is generated according to the constructed probabilistic model, and another is updated according to the statistics of advanced individuals that aim to reduce the probability of combination explosion and time consumption according to the mask information. In the simulations, different architectures, different mask ratios and different promising individual ratios are adopted to testify the effectiveness of the improved algorithm. According to simulation results, the estimation of thr distribution algorithm has a steady optimization ability for the shallow and stacked auto-encoder by one-step pre-training combining SGD based fine-tuning for the MNIST dataset. The proposed model will achieve a state-of-the-art performance on Fashion-MNIST.

Graphical model based continuous estimation of distribution algorithm

In this paper, a new estimation of distribution algorithm is introduced. The goal is to propose a method that avoids complex approximations of learning a probabilistic graphical model and considers multivariate dependencies between continuous random variables. A parallel model of some subgraphs with a smaller number of variables is learned as the probabilistic graphical model. In each generation, the joint probability distribution of the selected solutions is estimated using a Gaussian Mixture model. Then, learning the graphical model of dependencies among random variables and sampling are done separately for each Gaussian component. In the learning step, using the selected solutions of each Gaussian mixture component, the structure of a Markov network is learned. This network is decomposed to maximal cliques and a clique graph. Then, complete Bayesian network structures are learned for these subgraphs using an optimization algorithm. The proposed optimization problem is a 0–1 constrained quadratic programming which finds the best permutation of variables. Then, sampling is done from each Bayesian network of each Gaussian component. The introduced method is compared with the other network-based estimation of distribution algorithms for optimization of continuous numerical functions.

An extended continuous estimation of distribution algorithm for solving the permutation flow-shop scheduling problem

ABSTRACTThis article proposes an extended continuous estimation of distribution algorithm (ECEDA) to solve the permutation flow-shop scheduling problem (PFSP). In ECEDA, to make a continuous estimation of distribution algorithm (EDA) suitable for the PFSP, the largest order value rule is applied to convert continuous vectors to discrete job permutations. A probabilistic model based on a mixed Gaussian and Cauchy distribution is built to maintain the exploration ability of the EDA. Two effective local search methods, i.e. revolver-based variable neighbourhood search and Hénon chaotic-based local search, are designed and incorporated into the EDA to enhance the local exploitation. The parameters of the proposed ECEDA are calibrated by means of a design of experiments approach. Simulation results and comparisons based on some benchmark instances show the efficiency of the proposed algorithm for solving the PFSP.

A continuous estimation of distribution algorithm for the online order-batching problem

In manual order-picking systems such as picker-to-parts, order pickers walk through a warehouse in order to pick up articles required by customers. Order batching consists of combining these customer orders into picking orders. In online batching, customer orders arrive throughout the scheduling. This paper considers an online order-batching problem in which the turnover time of all customer orders has to be minimized, i.e., the time period between the arrival time of the customer order and its completion time. A continuous estimation of distribution algorithm-based approach is proposed and developed to solve the problem and implement the solution. Using this approach, the warehouse performance can be noticeably improved with a substantial reduction in the average turnover time of a set of customer orders.