Univariate Marginal Distribution Algorithm Research Articles

Metaheuristic optimization algorithms comparison adopted for the profit maximization of electricity market participants

The electricity market faces numerous challenges due to the growing demand for energy, increasing penetration of renewable energy sources, and the need for grid reliability and efficiency. To address these challenges, optimization algorithms have emerged as essential tools for optimizing various aspects of the electricity market, including generation, transmission, distribution, and demand-side management. The review can be done by providing an overview of the key components and challenges of the electricity market, including generation dispatch, unit commitment, economic dispatch, transmission network optimization, and demand response management. It then systematically examines a wide range of optimization techniques employed in addressing these challenges, including linear programming, mixed-integer linear programming, nonlinear programming, dynamic programming, genetic algorithms, particle swarm optimization, simulated annealing, and machine learning-based approaches. This paper presents a comparison of optimization algorithms, RCEDUMDA (Ring-Cellular Encode-Decode Univariate Marginal Distribution Algorithm) and CL_HC2RCEDUMDA (Hill Climbing to Ring Cellular Encode-Decode Univariate Marginal Distribution Algorithm) for the profit maximization of Electricity Market consumers & prosumers.

Metaheuristic Optimization Algorithms Comparison Adopted for the Profit Maximization of Electricity Market Participants

The electricity market faces numerous challenges due to the growing demand for energy, increasing penetration of renewable energy sources, and the need for grid reliability and efficiency. To address these challenges, optimization algorithms have emerged as essential tools for optimizing various aspects of the electricity market, including generation, transmission, distribution, and demand-side management. The review can be done by providing an overview of the key components and challenges of the electricity market, including generation dispatch, unit commitment, economic dispatch, transmission network optimization, and demand response management. It then systematically examines a wide range of optimization techniques employed in addressing these challenges, including linear programming, mixed-integer linear programming, nonlinear programming, dynamic programming, genetic algorithms, particle swarm optimization, simulated annealing, and machine learning-based approaches. This paper presents a comparison of optimization algorithms, RCEDUMDA (Ring-Cellular Encode-Decode Univariate Marginal Distribution Algorithm) and CL_HC2RCEDUMDA (Hill Climbing to Ring Cellular Encode-Decode Univariate Marginal Distribution Algorithm) for the profit maximization of Electricity Market consumers & prosumers.

Automatic Classification of Coronary Stenosis Using Feature Selection and a Hybrid Evolutionary Algorithm

In this paper, a novel method for the automatic classification of coronary stenosis based on a feature selection strategy driven by a hybrid evolutionary algorithm is proposed. The main contribution is the characterization of the coronary stenosis anomaly based on the automatic selection of an efficient feature subset. The initial feature set consists of 49 features involving intensity, texture and morphology. Since the feature selection search space was O(2n), being n=49, it was treated as a high-dimensional combinatorial problem. For this reason, different single and hybrid evolutionary algorithms were compared, where the hybrid method based on the Boltzmann univariate marginal distribution algorithm (BUMDA) and simulated annealing (SA) achieved the best performance using a training set of X-ray coronary angiograms. Moreover, two different databases with 500 and 2700 stenosis images, respectively, were used for training and testing of the proposed method. In the experimental results, the proposed method for feature selection obtained a subset of 11 features, achieving a feature reduction rate of 77.5% and a classification accuracy of 0.96 using the training set. In the testing step, the proposed method was compared with different state-of-the-art classification methods in both databases, obtaining a classification accuracy and Jaccard coefficient of 0.90 and 0.81 in the first one, and 0.92 and 0.85 in the second one, respectively. In addition, based on the proposed method’s execution time for testing images (0.02 s per image), it can be highly suitable for use as part of a clinical decision support system.

A hybrid method based on estimation of distribution algorithms to train convolutional neural networks for text categorization

Convolutional Neural Networks for text categorization allows the extraction of features from the text represented through word embedding. The high dimensionality of the texts themselves implies a larger number of network parameters and a more complex optimization surface. Artificial neural network training is an NP-Hard optimization problem, which has been addressed by methods based on partial derivatives of the objective function and presents several theoretical and practical limitations, such as the probability of convergence to local minimums. In this work, we propose a hybrid method based on the Estimation of Distribution Algorithms for training a Convolutional Neural Network. For this, we train together gradient-based methods with the Estimation of Multivariate Normal Algorithm and Univariate Marginal Distribution Algorithm by dividing the training process into two stages. The different variants obtained with the proposed method are compared with gradient-based methods on public benchmark datasets and statistical differences are analyzed by nonparametric tests. The proposed method increases the accuracy of the convolutional network applied to the text categorization task and overcome in about 0.22%–24% the state-of-the-art algorithms.

The Univariate Marginal Distribution Algorithm Copes Well with Deception and Epistasis.

In their recent work, Lehre and Nguyen (2019) show that the univariate marginal distribution algorithm (UMDA) needs time exponential in the parent populations size to optimize the DeceptiveLeadingBlocks (DLB) problem. They conclude from this result that univariate EDAs have difficulties with deception and epistasis. In this work, we show that this negative finding is caused by the choice of the parameters of the UMDA. When the population sizes are chosen large enough to prevent genetic drift, then the UMDA optimizes the DLB problem with high probability with at most λ(n2+2elnn) fitness evaluations. Since an offspring population size λ of order nlogn can prevent genetic drift, the UMDA can solve the DLB problem with O(n2logn) fitness evaluations. In contrast, for classic evolutionary algorithms no better runtime guarantee than O(n3) is known (which we prove to be tight for the (1+1) EA), so our result rather suggests that the UMDA can cope well with deception and epistatis. From a broader perspective, our result shows that the UMDA can cope better with local optima than many classic evolutionary algorithms; such a result was previously known only for the compact genetic algorithm. Together with the lower bound of Lehre and Nguyen, our result for the first time rigorously proves that running EDAs in the regime with genetic drift can lead to drastic performance losses.

Automatic Feature Selection for Stenosis Detection in X-ray Coronary Angiograms

The automatic detection of coronary stenosis is a very important task in computer aided diagnosis systems in the cardiology area. The main contribution of this paper is the identification of a suitable subset of 20 features that allows for the classification of stenosis cases in X-ray coronary images with a high performance overcoming different state-of-the-art classification techniques including deep learning strategies. The automatic feature selection stage was driven by the Univariate Marginal Distribution Algorithm and carried out by statistical comparison between five metaheuristics in order to explore the search space, which is O(249) computational complexity. Moreover, the proposed method is compared with six state-of-the-art classification methods, probing its effectiveness in terms of the Accuracy and Jaccard Index evaluation metrics. All the experiments were performed using two X-ray image databases of coronary angiograms. The first database contains 500 instances and the second one 250 images. In the experimental results, the proposed method achieved an Accuracy rate of 0.89 and 0.88 and Jaccard Index of 0.80 and 0.79, respectively. Finally, the average computational time of the proposed method to classify stenosis cases was ≈0.02 s, which made it highly suitable to be used in clinical practice.

Runtime Analyses of the Population-Based Univariate Estimation of Distribution Algorithms on LeadingOnes

We perform rigorous runtime analyses for the univariate marginal distribution algorithm (UMDA) and the population-based incremental learning (PBIL) Algorithm on LeadingOnes. For the UMDA, the currently known expected runtime on the function is {mathcal {O}}left( nlambda log lambda +n^2right) under an offspring population size lambda =Omega (log n) and a parent population size mu le lambda /(e(1+delta )) for any constant delta >0 (Dang and Lehre, GECCO 2015). There is no lower bound on the expected runtime under the same parameter settings. It also remains unknown whether the algorithm can still optimise the LeadingOnes function within a polynomial runtime when mu ge lambda /(e(1+delta )). In case of the PBIL, an expected runtime of {mathcal {O}}(n^{2+c}) holds for some constant c in (0,1) (Wu, Kolonko and Möhring, IEEE TEVC 2017). Despite being a generalisation of the UMDA, this upper bound is significantly asymptotically looser than the upper bound of {mathcal {O}}left( n^2right) of the UMDA for lambda =Omega (log n)cap {mathcal {O}}left( n/log nright). Furthermore, the required population size is very large, i.e., lambda =Omega (n^{1+c}). Our contributions are then threefold: (1) we show that the UMDA with mu =Omega (log n) and lambda le mu e^{1-varepsilon }/(1+delta ) for any constants varepsilon in (0,1) and 0<delta le e^{1-varepsilon }-1 requires an expected runtime of e^{Omega (mu )} on LeadingOnes, (2) an upper bound of {mathcal {O}}left( nlambda log lambda +n^2right) is shown for the PBIL, which improves the current bound {mathcal {O}}left( n^{2+c}right) by a significant factor of Theta (n^{c}), and (3) we for the first time consider the two algorithms on the LeadingOnes function in a noisy environment and obtain an expected runtime of {mathcal {O}}left( n^2right) for appropriate parameter settings. Our results emphasise that despite the independence assumption in the probabilistic models, the UMDA and the PBIL with fine-tuned parameter choices can still cope very well with variable interactions.

Estimation of distribution algorithms for the computation of innovation estimators of diffusion processes

Innovation Method is a recognized method for the estimation of parameters in diffusion processes. It is well known that the quality of the Innovation Estimator strongly depends on an adequate selection of the initial value for the parameters when a local optimization algorithm is used in its computation. Alternatively, in this paper, we use a strategy based on a modern method for solving global optimization problems, Estimation of Distribution Algorithms (EDAs). We study the feasibility of a specific EDA - a continuous version of the Univariate Marginal Distribution Algorithm (UMDAc) - for the computation of the Innovation Estimators. Through numerical simulations, we show that the considered global optimization algorithms substantially improves the effectiveness of the Innovation Estimators for different types of diffusion processes with complex nonlinear and stochastic dynamics.

Cellular estimation of distribution algorithm designed to solve the energy resource management problem under uncertainty

The Energy Resource Management (ERM) can be modeled as a Mixed-Integer Non-Linear Problem whose aim is to maximize profits generally using smart grid capabilities more than importing energy from external markets. Due to this, many resources and customers are involved in optimization, making ERM a complex problem. Moreover, when the inherent uncertainty of weather conditions, load forecast, electric vehicles planned trips, or market prices is considered, deterministic approaches might fail in obtaining optimal solutions to the problem. In this context, evolutionary algorithms are a useful tool to find effective near-optimal solutions. In fact, to design and test evolutionary algorithms to solve the ERM problem under uncertainty, the research community has developed a simulation framework. In this paper, we propose the Cellular Univariate Marginal Distribution Algorithm with Normal-Cauchy distribution (CUMDANCauchy) to address the ERM problem in uncertain environments. CUMDANCauchy uses a univariate estimation of the product of Normal and Cauchy distributions over each feature, and produces new individuals not only by the sampling of the learned distributions but also using neighborhoods of individuals from a ring cellular structure. The experiments performed over two case studies show that: CUMDANCauchy is as competitive as the previous dominant class of algorithms in terms of the global fitness achieved; its convergence behavior is among the best in comparison with the other tested algorithms; its running time is similar to the algorithm with the best global fitness achieved in the first case study, and it is the fastest algorithm in the second one.

A Two-Stage Mono- and Multi-Objective Method for the Optimization of General UPS Parallel Manipulators

This paper introduces a two-stage method based on bio-inspired algorithms for the design optimization of a class of general Stewart platforms. The first stage performs a mono-objective optimization in order to reach, with sufficient dexterity, a regular target workspace while minimizing the elements’ lengths. For this optimization problem, we compare three bio-inspired algorithms: the Genetic Algorithm (GA), the Particle Swarm Optimization (PSO), and the Boltzman Univariate Marginal Distribution Algorithm (BUMDA). The second stage looks for the most suitable gains of a Proportional Integral Derivative (PID) control via the minimization of two conflicting objectives: one based on energy consumption and the tracking error of a target trajectory. To this effect, we compare two multi-objective algorithms: the Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D) and Non-dominated Sorting Genetic Algorithm-III (NSGA-III). The main contributions lie in the optimization model, the proposal of a two-stage optimization method, and the findings of the performance of different bio-inspired algorithms for each stage. Furthermore, we show optimized designs delivered by the proposed method and provide directions for the best-performing algorithms through performance metrics and statistical hypothesis tests.

A simplified run time analysis of the univariate marginal distribution algorithm on LeadingOnes

With elementary means, we prove a stronger run time guarantee for the univariate marginal distribution algorithm (UMDA) optimizing the LeadingOnes benchmark function in the desirable regime with low genetic drift. If the population size is at least quasilinear, then, with high probability, the UMDA samples the optimum in a number of iterations that is linear in the problem size divided by the logarithm of the UMDA's selection rate. This improves over the previous guarantee, obtained by Dang and Lehre (2015) via the deep level-based population method, both in terms of the run time and by demonstrating further run time gains from small selection rates. Under similar assumptions, we prove a lower bound that matches our upper bound up to constant factors.

The Complex Parameter Landscape of the Compact\xa0Genetic\xa0Algorithm

The compact Genetic Algorithm (cGA) evolves a probability distribution favoring optimal solutions in the underlying search space by repeatedly sampling from the distribution and updating it according to promising samples. We study the intricate dynamics of the cGA on the test function OneMax, and how its performance depends on the hypothetical population size K, which determines how quickly decisions about promising bit values are fixated in the probabilistic model. It is known that the cGA and the Univariate Marginal Distribution Algorithm (UMDA), a related algorithm whose population size is called lambda, run in expected time O(n log n) when the population size is just large enough (K = varTheta (sqrt{n}log n) and lambda = varTheta (sqrt{n}log n), respectively) to avoid wrong decisions being fixated. The UMDA also shows the same performance in a very different regime (lambda =varTheta (log n), equivalent to K = varTheta (log n) in the cGA) with much smaller population size, but for very different reasons: many wrong decisions are fixated initially, but then reverted efficiently. If the population size is even smaller (o(log n)), the time is exponential. We show that population sizes in between the two optimal regimes are worse as they yield larger runtimes: we prove a lower bound of varOmega (K^{1/3}n + n log n) for the cGA on OneMax for K = O(sqrt{n}/log ^2 n). For K = varOmega (log ^3 n) the runtime increases with growing K before dropping again to O(Ksqrt{n} + n log n) for K = varOmega (sqrt{n} log n). This suggests that the expected runtime for the cGA is a bimodal function in K with two very different optimal regions and worse performance in between.

Scheduling in Heterogeneous Distributed Computing Systems Based on Internal Structure of Parallel Tasks Graphs with Meta-Heuristics

The problem of scheduling parallel tasks graphs (PTGs) represented by directed acyclic graphs (DAGs) in heterogeneous distributed computing systems (HDCSs) is considered an nondeterministic polynomial time (NP) problem due to the diversity of characteristics and parameters, generally opposed, intended to be optimized. The PTGs are scheduled by a scheduler that determines the best location for the sub-tasks that constitute the PTGs and is responsible for allocating the resources of the HDCS to the sub-tasks of the PTGs. To optimize scheduling and allocations, the scheduler extracts characteristics from the internal structure of the PTGs. The prevailing characteristic in existing research is the critical path (CP), which is limited to providing execution paths of PTGs; considering this limitation, we extend the array method proposed in Velarde, which extracts two additional characteristics to the CP: the layering and the density of the graph for scheduling. These characteristics are represented as integer values of the PTGs to be scheduled; the values obtained from the characteristics are stored in arrays representing populations that are evaluated with the heuristic univariate marginal distribution algorithm (UMDA) and in terms of comparison with the genetic algorithm. With the best allocations produced by the algorithms, two performance parameters are evaluated: makespan and waiting time. The results indicate that when more PTGs characteristics are considered, resource allocations are optimized, and scheduling times are reduced. The results obtained with the heuristic algorithms show that UMDA provides shorter scheduling and allocation times compared with the genetic algorithm; UMDA widely distributes the sub-tasks in the clusters, whereas the genetic algorithm compacts the assignments of the PTGs in the clusters with a longer convergence time that translates into longer scheduling and allocation times. Extensive explanations of these conclusions are provided in this work, based on the conducted experiments.

Kinetic Parameter Determination for Depolymerization of Biomass by Inverse Modeling and Metaheuristics

A computational methodology based on inverse modeling and metaheuristics is presented for determining the best parameters of kinetic models aimed to predict the behavior of biomass depolymerization processes during size scaling up. The Univariate Marginal Distribution algorithm, particle swarm optimization, and Interior-Point algorithm were applied to obtain the values of the kinetic parameters (KM and Vmax) of four mathematical models based on the Michaelis–Menten equation: (i) Traditional Michaelis–Menten, (ii) non-competitive inhibition, (iii) competitive inhibition, and (iv) substrate inhibition. The kinetic data were obtained from our own experimentation in micro-scale. The parameters obtained from an optimized micro-scale experiment were compared with a bench scale experiment (0.5 L). Regarding the metaheuristic optimizers, it is concluded that the Interior-Point algorithm is effective in solving inverse modeling problems and has the best prediction power. According to the results, the Traditional model adequately describes the micro-scale experiments. It was found that the Traditional model with optimized parameters was able to predict the behavior of the depolymerization process during size scaling up. The methodology followed in this study can be adopted as a starting point for the solution of future inverse modeling problems.

Sharp Bounds for Genetic Drift in Estimation of Distribution Algorithms

Estimation of distribution algorithms (EDAs) are a successful branch of evolutionary algorithms (EAs) that evolve a probabilistic model instead of a population. Analogous to genetic drift in EAs, EDAs also encounter the phenomenon that the random sampling in the model update can move the sampling frequencies to boundary values not justified by the fitness. This can result in a considerable performance loss. This article gives the first tight quantification of this effect for three EDAs and one ant colony optimizer, namely, for the univariate marginal distribution algorithm, the compact genetic algorithm, population-based incremental learning, and the max-min ant system with iteration-best update. Our results allow to choose the parameters of these algorithms in such a way that within a desired runtime, no sampling frequency approaches the boundary values without a clear indication from the objective function.

Determinación de la parábola de la vasculatura de la retina mediante un algoritmo computacional de segmentación

El análisis cuantitativo de la arquitectura de las venas temporales superior e inferior y su monitoreo sobre el tiempo puede facilitar el diagnóstico y tratamiento oportuno de la retinopatía diabética. En este trabajo se presenta un novedoso método que consiste de dos etapas correspondientes a la segmentación automática y modelado parabólico de las venas temporales superior e inferior en imágenes de fondo de ojo. En la primera etapa, el detector lineal multiescala (DLM) es empleado para detectar estructuras de tipo arterial en imágenes de la retina. Debido a que DLM es un método de realzado arterial, es necesario aplicar una estrategia de umbralización para clasificar pixeles de tipo arterial con respecto al fondo de la imagen, donde un valor de umbral determinado de forma experimental es comparado con cinco métodos de umbralización del estado del arte. En esta etapa, el método de segmentación propuesto es comparado con seis métodos especializados del estado del arte en términos de eficiencia de segmentación. En la segunda etapa, se desempeña un modelado parabólico mediante una estrategia de optimización utilizando un Algoritmo de Distribución Marginal Univariada sobre las arterias previamente segmentadas, y los resultados son comparados con dos métodos paramétricos del estado del arte y con las delineaciones realizadas por especialistas. Los resultados de segmentación arterial utilizando el detector lineal multiescala demostraron una alta eficiencia de segmentación obteniendo un valor de 0.9618 utilizando la base de datos DRIVE de imágenes de fondo de ojo. De igual forma, los resultados de modelado parabólico entregaron una eficiencia promedio de 0.825 con respecto a las delineaciones realizadas por especialistas oftalmólogos de las venas temporales superior e inferior. En base a los resultados de eficiencia y al tiempo computacional (5.62 segundos), el método propuesto puede considerarse como altamente apropiado para desempeñar diagnóstico asistido por computadora en el área de oftalmología.

Glucose forecasting combining Markov chain based enrichment of data, random grammatical evolution and Bagging

Diabetes Mellitus is a disease affecting more and more people every year. Depending on the kind of diabetes and sometimes on the stage of the illness, diabetic patients have to inject some amount of artificial insulin, namely bolus, before the meals, to make up the absence or malfunctioning of their natural insulin. This decision is a difficult task since they need to estimate the number of carbohydrates they are going to ingest, take into account the past and future circumstances, know the past values of glucose, evaluate if the effect of previously injected insulin has already finished and any other relevant information. In this paper, we present and compare a set of methodologies to automate the decision of the insulin bolus, which reduces the number of dangerous predictions. We combine two different data enrichment techniques based on Markov chains with grammatical evolution engines to generate models of blood glucose, and univariate marginal distribution algorithms and bagging techniques to select the set of models to assemble. In particular, we propose the Random-GE procedure, an adaptation of Random Forests to Grammatical Evolution, which leads to excellent prediction models, with a simple configuration and a reduced execution time. The ensemble gives the prediction of glucose for a duple of food and insulins, helping patients in the selection of the appropriate bolus to maintain healthy glucose levels after the meals. Experimental results show that our models get more accurate and robust predictions than previous approaches.

Parameter optimization for the smoothed-particle hydrodynamics method by means of evolutionary metaheuristics

Simulation of fluid dynamics using the Smoothed-Particle Hydrodynamics (SPH) technique, has many applications in areas such as engineering, video games, animation, and graphics. In many of these cases, the values of the SPH parameters employed, such as the number of particles used in the simulation, the SPH kernel length and the values of additional parameters, are not discussed. In this work we optimize the SPH parameters by means of metaheuristic algorithms, using SPH-simulations of five very representative fluid-dynamics test problems. For this purpose, a fitness function based on the exact solution of the test problems analyzed is proposed. We provide optimized parameters that can be applied to real-world and real-time applications and suggest a method to choose these parameters based on the initial conditions in the computational domain. Furthermore, we compare the Univariate Marginal Distribution Algorithm (UMDA), the Univariate Boltzmann Marginal Distribution Algorithm (BUMDA), and Differential Evolution (DE) in terms of their performance in this optimization task. Through statistical analysis it is determined that UMDA is superior, and its convergence to the optimum parameters is examined.

Local Parameter Optimization of LSSVM for Industrial Soft Sensing With Big Data and Cloud Implementation

Due to the advantages of high prediction accuracy, least squares support vector machine (LSSVM) has been widely utilized for soft sensor developments in industrial processes. The hyper-parameters of LSSVM are often determined by minimizing the predicted error of validation set based on the intelligent optimization algorithm, which may lead to excessive optimization and model overfitting when validation set are selected improperly. Meanwhile, online parameters optimization is difficult to implement, which results in poor effect of local modeling. This paper proposes UMDA-LOS-LSSVM that is a LSSVM with parameters optimization in local objective set (LOS-LSSVM) by univariate marginal distribution algorithm (UMDA) based on the idea of local modeling. First, the local objective set is extracted in the candidate set based on the testing samples. Then, UMDA is utilized for minimize the predicted error of the objective set and provides the optimized parameters. Finally, training and testing of LSSVM are carried out based on the optimal parameters. In addition, this paper provides the distributed parallel form of the proposed method, which can be used for big data modeling and soft sensor development. The proposed method is applied in a CO <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> absorbing column unit to estimate the residual CO <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> content, which is implemented through an industrial big data distributed analytics platform. The results show a significant improvement of proposed method based soft sensor, compared to traditional methods.

Level-Based Analysis of the Univariate Marginal Distribution Algorithm

Estimation of Distribution Algorithms (EDAs) are stochastic heuristics that search for optimal solutions by learning and sampling from probabilistic models. Despite their popularity in real-world applications, there is little rigorous understanding of their performance. Even for the Univariate Marginal Distribution Algorithm (UMDA)—a simple population-based EDA assuming independence between decision variables—the optimisation time on the linear problem OneMax was until recently undetermined. The incomplete theoretical understanding of EDAs is mainly due to the lack of appropriate analytical tools. We show that the recently developed level-based theorem for non-elitist populations combined with anti-concentration results yield upper bounds on the expected optimisation time of the UMDA. This approach results in the bound mathcal {O}left( nlambda log lambda +n^2right) on the LeadingOnes and BinVal problems for population sizes lambda >mu =varOmega (log n), where mu and lambda are parameters of the algorithm. We also prove that the UMDA with population sizes mu in mathcal {O}left( sqrt{n}right) cap varOmega (log n) optimises OneMax in expected time mathcal {O}left( lambda nright) , and for larger population sizes mu =varOmega (sqrt{n}log n), in expected time mathcal {O}left( lambda sqrt{n}right) . The facility and generality of our arguments suggest that this is a promising approach to derive bounds on the expected optimisation time of EDAs.