Discrete Optimization Problems Research Articles

Pairwise Distances and the Problem of Multiple Optima.

Discrete optimization problems arise in many biological contexts and, in many cases, we seek to make inferences from the optimal solutions. However, the number of optimal solutions is frequently very large and making inferences from any single solution may result in conclusions that are not supported by other optimal solutions. We describe a general approach for efficiently (polynomial time) and exactly (without sampling) computing statistics on the space of optimal solutions. These statistics provide insights into the space of optimal solutions that can be used to support the use of a single optimum (e.g., when the optimal solutions are similar) or justify the need for selecting multiple optima (e.g., when the solution space is large and diverse) from which to make inferences. We demonstrate this approach on two well-known problems and identify the properties of these problems that make them amenable to this method.

Roulette wheel-based level learning evolutionary algorithm for feature selection of high-dimensional data

Feature selection in high-dimensional data is a large-scale sparse and discrete optimization problem. Most evolutionary algorithms are designed to tackle continuous optimization problems. However, when dealing with high-dimensional feature selection tasks, they often suffer from poor population diversity and are computationally expensive. To address these challenges, this work introduces a roulette wheel-based level learning evolutionary algorithm (RWLLEA). RWLLEA integrates two key components. Firstly, it employs a leveled population mode. Individuals from higher levels provide guidance to those at lower levels during the evolutionary process, thereby exploring the potential combinatorial effects among features. Secondly, recognizing the characteristics associated with the high-dimensional feature selection task, a roulette wheel-based update method is devised to dynamically reduce search space and harmonize the algorithm's exploitation and exploration capacities across different stages. The performance of the proposed method was evaluated by comparing it with six other feature selection techniques across a range of fifteen diverse datasets. The experimental findings demonstrate that the proposed method can achieve a reduced feature set with a shorter runtime and exhibit superior classification accuracy.

Metaheuristics for variable-size mixed optimization problems: A unified taxonomy and survey

Many real world optimization problems are formulated as mixed-variable optimization problems (MVOPs) which involve both continuous and discrete variables. MVOPs including dimensional variables are characterized by a variable-size search space. Depending on the values of dimensional variables, the number and type of the variables of the problem can vary dynamically. MVOPs and variable-size MVOPs (VMVOPs) are difficult to solve and raise a number of scientific challenges in the design of metaheuristics. Standard metaheuristics have been first designed to address continuous or discrete optimization problems, and are not able to tackle VMVOPs in an efficient way. The development of metaheuristics for solving such problems has attracted the attention of many researchers and is increasingly popular. However, to our knowledge there is no well established taxonomy or comprehensive survey for handling this important family of optimization problems. This paper presents an unified taxonomy for metaheuristic solutions for solving VMVOPs in an attempt to provide a common terminology and classification mechanisms. It provides a general mathematical formulation and concepts of VMVOPs, and identifies the various solving methodologies than can be applied in metaheuristics. The advantages, the weaknesses and the limitations of the presented methodologies are discussed. The proposed taxonomy also allows to identify some open research issues which needs further in-depth investigations.

Research on the Human–Robot Collaborative Disassembly Line Balancing of Spent Lithium Batteries with a Human Factor Load

The disassembly of spent lithium batteries is a prerequisite for efficient product recycling, the first link in remanufacturing, and its operational form has gradually changed from traditional manual disassembly to robot-assisted human–robot cooperative disassembly. Robots exhibit robust load-bearing capacity and perform stable repetitive tasks, while humans possess subjective experiences and tacit knowledge. It makes the disassembly activity more adaptable and ergonomic. However, existing human–robot collaborative disassembly studies have neglected to account for time-varying human conditions, such as safety, cognitive behavior, workload, and human pose shifts. Firstly, in order to overcome the limitations of existing research, we propose a model for balancing human–robot collaborative disassembly lines that take into consideration the load factor related to human involvement. This entails the development of a multi-objective mathematical model aimed at minimizing both the cycle time of the disassembly line and its associated costs while also aiming to reduce the integrated smoothing exponent. Secondly, we propose a modified multi-objective fruit fly optimization algorithm. The proposed algorithm combines chaos theory and the global cooperation mechanism to improve the performance of the algorithm. We add Gaussian mutation and crowding distance to efficiently solve the discrete optimization problem. Finally, we demonstrate the effectiveness and sensitivity of the improved multi-objective fruit fly optimization algorithm by solving and analyzing an example of Mercedes battery pack disassembly.

Advancing Feature Ranking with HFRWM2: A Weighted Majority Voting Strategy Enhanced by the Harris Hawks Optimizer

Abstract Feature selection (FS) is vital in improving the performance of machine learning (ML) algorithms. Despite its importance, identifying the most important features remains challenging, highlighting the need for advanced optimization techniques. In this study, we propose a novel hybrid feature ranking technique called the Hybrid Feature Ranking Weighted Majority Model (HFRWM2). HFRWM2 combines ML models with the Harris Hawks Optimizer (HHO) metaheuristic. HHO is known for its versatility in addressing various optimization challenges, thanks to its ability to handle continuous, discrete, and combinatorial optimization problems. It achieves a balance between exploration and exploitation by mimicking the cooperative hunting behavior of Harris’s hawks, thus thoroughly exploring the search space and converging towards optimal solutions. Our approach operates in two phases. First, an odd number of ML models, in conjunction with HHO, generate feature encodings along with performance metrics. These encodings are then weighted based on their metrics and vertically aggregated. This process produces feature rankings, facilitating the extraction of the top-K features. The motivation behind our research is twofold: to enhance the precision of ML algorithms through optimized FS and to improve the overall efficiency of predictive models. To evaluate the effectiveness of HFRWM2, we conducted rigorous tests on two datasets: “Australian” and “Fertility.” Our findings demonstrate the effectiveness of HFRWM2 in navigating the search space and identifying optimal solutions. We compared HFRWM2 against twelve other feature ranking techniques and found it to outperform them. This superiority was particularly evident in the graphical comparison of the “Australian” dataset, where HFRWM2 showed significant advancements in feature ranking.

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

Is Stochastic Gradient Descent (SGD) substantially different from Metropolis Monte Carlo dynamics? This is a fundamental question at the time of understanding the most used training algorithm in the field of Machine Learning, but it received no answer until now. Here we show that in discrete optimization and inference problems, the dynamics of an SGD-like algorithm resemble very closely that of Metropolis Monte Carlo with a properly chosen temperature, which depends on the mini-batch size. This quantitative matching holds both at equilibrium and in the out-of-equilibrium regime, despite the two algorithms having fundamental differences (e.g. SGD does not satisfy detailed balance). Such equivalence allows us to use results about performances and limits of Monte Carlo algorithms to optimize the mini-batch size in the SGD-like algorithm and make it efficient at recovering the signal in hard inference problems.

Mathematical Models for the Single-Channel and Multi-Channel PMU Allocation Problem and Their Solution Algorithms

Phasor measurement units (PMUs) are deployed at power grid nodes around the transmission grid, determining precise power system monitoring conditions. In real life, it is not realistic to place a PMU at every power grid node; thus, the lowest PMU number is optimally selected for the full observation of the entire network. In this study, the PMU placement model is reconsidered, taking into account single- and multi-capacity placement models rather than the well-studied PMU placement model with an unrestricted number of channels. A restricted number of channels per monitoring device is used, instead of supposing that a PMU is able to observe all incident buses through the transmission connectivity lines. The optimization models are declared closely to the power dominating set and minimum edge cover problem in graph theory. These discrete optimization problems are directly related with the minimum set covering problem. Initially, the allocation model is declared as a constrained mixed-integer linear program implemented by mathematical and stochastic algorithms. Then, the 0/1 integer linear problem is reformulated into a non-convex constraint program to find optimality. The mathematical models are solved either in binary form or in the continuous domain using specialized optimization libraries, and are all implemented in YALMIP software in conjunction with MATLAB. Mixed-integer linear solvers, nonlinear programming solvers, and heuristic algorithms are utilized in the aforementioned software packages to locate the global solution for each instance solved in this application, which considers the transformation of the existing power grids to smart grids.

Learning to sample initial solution for solving 0–1 discrete optimization problem by local search

Learning to sample initial solution for solving 0–1 discrete optimization problem by local search

Classification feature selection and dimensionality reduction based on logical binary sine-cosine function arithmetic optimization algorithm

Arithmetic optimization algorithm (AOA) is a meta-heuristic algorithm inspired by mathematical operations. AOA has been diffusely used for optimization issues on continuous domains, but few scholars have studied discrete optimization problems. In this paper, we proposed Binary AOA (BAOA) based on two strategies to handle the feature selection problem. The first strategy used S-shaped and V-shaped shift functions to map continuous variables to discrete variables. The second strategy was to combine four logical operations (AND, OR, XOR, XNOR) on the basis of the transfer function, and constructed a parameter model based on the sine and cosine function. An enhanced logic binary sine–cosine function arithmetic optimization algorithm (LBSCAOA) was proposed to realize the position update of variables. Its purpose was to improve the algorithm's global search capabilities and local exploitation capabilities. In the simulation experiments, 20 datasets were selected to testify the capability of the proposed algorithm. Since KNN had the advantages of easy understanding and low training time complexity, this classifier was selected for evaluation. The performance of the improved algorithm was comprehensively evaluated by comparing the average classification accuracy, the average number of selected features, the average fitness value and the average running time. Simulation results showed LBSCAOA with V-Shaped “V4” stood out among many improved algorithms. On the other hand, LBSCAOA with V-Shaped “V4” was used as a representative to compare with other typical feature selection algorithms to verify its competitivenes.

The Stability of Solutions of the Variable-Order Fractional Optimal Control Model for the COVID-19 Epidemic in Discrete Time

This article introduces a discrete-time fractional variable order over a SEIQR model, incorporated for COVID-19. Initially, we establish the well-possedness of solution. Further, the disease-free and the endemic equilibrium points are determined. Moreover, the local asymptotic stability of the model is analyzed. We develop a novel discrete fractional optimal control problem tailored for COVID-19, utilizing a discrete mathematical model featuring a variable order fractional derivative. Finally, we validate the reliability of these analytical findings through numerical simulations and offer insights from a biological perspective.

A Multi-Task Decomposition-Based Evolutionary Algorithm for Tackling High-Dimensional Bi-Objective Feature Selection

Evolutionary algorithms have been widely applied for solving multi-objective optimization problems, while the feature selection in classification can also be treated as a discrete bi-objective optimization problem if attempting to minimize both the classification error and the ratio of selected features. However, traditional multi-objective evolutionary algorithms (MOEAs) may have drawbacks for tackling large-scale feature selection, due to the curse of dimensionality in the decision space. Therefore, in this paper, we concentrated on designing an multi-task decomposition-based evolutionary algorithm (abbreviated as MTDEA), especially for handling high-dimensional bi-objective feature selection in classification. To be more specific, multiple subpopulations related to different evolutionary tasks are separately initialized and then adaptively merged into a single integrated population during the evolution. Moreover, the ideal points for these multi-task subpopulations are dynamically adjusted every generation, in order to achieve different search preferences and evolutionary directions. In the experiments, the proposed MTDEA was compared with seven state-of-the-art MOEAs on 20 high-dimensional classification datasets in terms of three performance indicators, along with using comprehensive Wilcoxon and Friedman tests. It was found that the MTDEA performed the best on most datasets, with a significantly better search ability and promising efficiency.

ЕФЕКТИВНЕ РОЗВ’ЯЗУВАННЯ МУЛЬТИМОДАЛЬНИХ ОПТИМІЗАЦІЙНИХ ЗАДАЧ

In this work, we consider multimodal optimization problems. Such problems contain many local extrema. We can say that most practical problems are multimodal. In particular, discrete optimization problems with Boolean and integer variables can easily be transformed into multimodal problems with continuous variables. Multimodal optimization problems can be of small or large dimensions in which the number of variables to be determined is hundreds or thousands of variables. Libraries of test and applied problems have been created to test the effectiveness of new global optimization methods. The author suggests separating problems with unknown optimal solutions in these tests. Then the best method for solving multimodal optimization problems will be the one that will allow obtaining better solutions in most such problems. Currently, this criterion is satisfied only by the exact quadratic regularization method developed by the author. This is confirmed by significant computational experiments on existing tests and applied multimodal optimization problems.

Sizing and Shape Optimization of Discrete Truss Employing a Target-Oriented Krill Herd Algorithm

Abstract The krill herd (KH) algorithm is widely used for optimizing truss structures as no gradient information is necessary, and only a few parameters require adjustment. However, when the truss structure becomes discrete and complex, KH tends to fall into a local optimum. Therefore, a novel target-oriented KH (TOKH) algorithm is proposed in this study to optimize the design of discrete truss structures. Initially, a crossover operator is established between the “best krill” and “suboptimal krill” to generate a robust “cross krill” for global exploration. Additionally, an improved local mutation and crossover (ILMC) operator is introduced to fine-tune the “center of food” and candidate solutions for local exploitation. The proposed method and other optimization approaches are experimentally compared considering 15 benchmark functions. Then, the performance of the TOKH algorithm is evaluated based on four discrete truss structure optimization problems under multiple loading conditions. The obtained optimization results indicate that the proposed method presents competitive solutions in terms of accuracy, unlike other algorithms in the literature, and avoids falling into a local minimum.

An operator-inspired framework for metaheuristics and its applications on job-shop scheduling problems

The job-shop scheduling problem (JSP) is a well-known combinatorial optimization problem in manufacturing systems. For the past two decades, real-number metaheuristics have been widely used to solve the JSP using the real-number transform methods. A limitation of the real-number metaheuristics is the premature convergence due to the stochasticity of the transform methods. To eliminate this limitation, a novel operator framework has been designed, building the bridge between the discrete optimization problem (JSP) and real-number metaheuristics (also called continuous metaheuristics). Specifically, this paper captures the core operators of the real-number metaheuristics, namely, addition, subtraction, and multiplication. Firstly, three new operators are reconstructed according to several simple neighborhood structures to solve the JSP robustly and effectively. The properties of the arithmetic (symmetry) are taken into account in the reconstruction of the proposed operators to avoid excessive redundant searches. Secondly, a positional similarity-based population diversity is presented for the JSP to demonstrate the intrinsic distinctions between the proposed operators and the transform methods during the evolutionary process. Finally, the results of five widely used benchmark test suites (185 instances) show that the proposed operators can achieve a better balance between exploration and exploitation than the current real-number transform methods.

Information theoretic clustering for coarse-grained modeling of non-equilibrium gas dynamics

We present a new framework towards the objective of learning coarse-grained models based on the maximum entropy principle. We show that existing methods for assigning clusters using the maximum entropy approach are heuristic or sub-optimal. We propose a machine learning framework informed by rate-distortion theory to learn optimal cluster assignments, aiming to improve the effectiveness of the coarse-graining process. Our approach involves transforming the discrete optimization problem into a probabilistic and continuous form. Our inverse modeling approach involves the development of a fully differentiable, adjoint-driven dynamics solver for use with gradient-based optimization techniques. The entire framework is end-to-end differentiable, facilitating backward pass gradient computation to flow through the ODE solve and probabilistic coarse-graining to train a classifier. We demonstrate application in the evolution of particle quantum states in non-equilibrium conditions. The high dimensionality of these equations poses a significant challenge for efficient and accurate computations, even for seemingly simple chemical systems. Training is performed using a loss function informed by rate-distortion theory to cluster the rovibrational states of some molecule, effectively forming a reduced order model of the master equations. We also introduce variable transformations along with several ways of improving the tractability of the training process. These allow the framework to process the extremely high-dimensional discrete optimization problem successfully. Our method is general, and to demonstrate the effectiveness in a controlled setting, we apply it to a one-dimensional Gaussian source quantization problem for which an analytical solution is known, followed by the problem of isothermal relaxation of a ▪ system with O(104) degrees of freedom.

Multi-objective optimization approach for coverage path planning of mobile robot

Abstract Coverage path planning (CPP) is a subfield of path planning problems in which free areas of a given domain must be visited by a robot at least once while avoiding obstacles. In some situations, the path may be optimized for one or more criteria such as total distance traveled, number of turns, and total area covered by the robot. Accordingly, the CPP problem has been formulated as a multi-objective optimization (MOO) problem, which turns out to be a challenging discrete optimization problem, hence conventional MOO algorithms like Non-dominated Sorting Genetic Algorithm-2 (NSGA-II) do not work as it is. This study implements a modified NSGA-II to solve the MOO problem of CPP for a mobile robot. In this paper, the proposed method adopted two objective functions: (1) the total distance traveled by the robot and (2) the number of turns taken by the robot. The two objective functions are used to calculate energy consumption. The proposed method is compared to the hybrid genetic algorithm (HGA) and the traditional genetic algorithm (TGA) in a rectilinear environment containing obstacles of various complex shapes. In addition, the results of the proposed algorithm are compared to those generated by HGA, TGA, oriented rectilinear decomposition, and spatial cell diffusion and family of spanning tree coverage in existing research papers. The results of all comparisons indicate that the proposed algorithm outperformed the existing algorithms by reducing energy consumption by 5 to 60%. This paper provides the facility to operate the robot in different modes.

A Novel Skip Orthogonal List for Dynamic Optimal Transport Problem

Optimal transport is a fundamental topic that has attracted a great amount of attention from the optimization community in the past decades. In this paper, we consider an interesting discrete dynamic optimal transport problem: can we efficiently update the optimal transport plan when the weights or the locations of the data points change? This problem is naturally motivated by several applications in machine learning. For example, we often need to compute the optimal transport cost between two different data sets; if some changes happen to a few data points, should we re-compute the high complexity cost function or update the cost by some efficient dynamic data structure? We are aware that several dynamic maximum flow algorithms have been proposed before, however, the research on dynamic minimum cost flow problem is still quite limited, to the best of our knowledge. We propose a novel 2D Skip Orthogonal List together with some dynamic tree techniques. Although our algorithm is based on the conventional simplex method, it can efficiently find the variable to pivot within expected O(1) time, and complete each pivoting operation within expected O(|V|) time where V is the set of all supply and demand nodes. Since dynamic modifications typically do not introduce significant changes, our algorithm requires only a few simplex iterations in practice. So our algorithm is more efficient than re-computing the optimal transport cost that needs at least one traversal over all |E|=O(|V|^2) variables, where |E| denotes the number of edges in the network. Our experiments demonstrate that our algorithm significantly outperforms existing algorithms in the dynamic scenarios.

Accelerating Cutting-Plane Algorithms via Reinforcement Learning Surrogates

Discrete optimization belongs to the set of N P-hard problems, spanning fields such as mixed-integer programming and combinatorial optimization. A current standard approach to solving convex discrete optimization problems is the use of cutting-plane algorithms, which reach optimal solutions by iteratively adding inequalities known as cuts to refine a feasible set. Despite the existence of a number of general-purpose cut-generating algorithms, large-scale discrete optimization problems continue to suffer from intractability. In this work, we propose a method for accelerating cutting-plane algorithms via reinforcement learning. Our approach uses learned policies as surrogates for N P-hard elements of the cut generating procedure in a way that (i) accelerates convergence, and (ii) retains guarantees of optimality. We apply our method on two types of problems where cutting-plane algorithms are commonly used: stochastic optimization, and mixed-integer quadratic programming. We observe the benefits of our method when applied to Benders decomposition (stochastic optimization) and iterative loss approximation (quadratic programming), achieving up to 45% faster average convergence when compared to modern alternative algorithms.

Using Clustering to Strengthen Decision Diagram Bounds for Discrete Optimization

Offering a generic approach to obtaining both upper and lower bounds, decision diagrams (DDs) are becoming an increasingly important tool for solving discrete optimization problems. In particular, they provide a powerful and often complementary alternative to other well-known generic bounding mechanisms such as the LP relaxation. A standard approach to employ DDs for discrete optimization is to formulate the problem as a Dynamic Program and use that formulation to compile a DD top-down in a layer-by-layer fashion. To limit the size of the resulting DD and to obtain bounds, one typically imposes a maximum width for each layer which is then enforced by either merging nodes (resulting in a so-called relaxed DD that provides a dual bound) or by dropping nodes (resulting in a so-called restricted DD that provides a primal bound). The quality of the DD bounds obtained from this top-down compilation process heavily depends on the heuristics used for the selection of the nodes to merge or drop. While it is sometimes possible to engineer problem-specific heuristics for this selection problem, the most generic approach relies on sorting the layer’s nodes based on objective function information. In this paper, we propose a generic and problem-agnostic approach that relies on clustering nodes based on the state information associated with each node. In a set of computational experiments with different knapsack and scheduling problems, we show that our approach generally outperforms the classical generic approach, and often achieves drastically better bounds both with respect to the size of the DD and the time used for compiling the DD.

Model Counting and Sampling via Semiring Extensions

Many decision and optimization problems have natural extensions as counting problems. The best known example is the Boolean satisfiability problem (SAT), where we want to count the satisfying assignments of truth values to the variables, which is known as the #SAT problem. Likewise, for discrete optimization problems, we want to count the states on which the objective function attains the optimal value. Both SAT and discrete optimization can be formulated as selective marginalize a product function (MPF) queries. Here, we show how general selective MPF queries can be extended for model counting. MPF queries are encoded as tensor hypernetworks over suitable semirings that can be solved by generic tensor hypernetwork contraction algorithms. Our model counting extension is again an MPF query, on an extended semiring, that can be solved by the same contraction algorithms. Model counting is required for uniform model sampling. We show how the counting extension can be further extended for model sampling by constructing yet another semiring. We have implemented the model counting and sampling extensions. Experiments show that our generic approach is competitive with the state of the art in model counting and model sampling.