Large Class Of Optimization Problems Research Articles

Constrained, Global Optimization of Unknown Functions with Lipschitz Continuous Gradients

We present two first-order, sequential optimization algorithms to solve constrained optimization problems. We consider a black-box setting with a priori unknown, nonconvex objective and constraint functions that have Lipschitz continuous gradients. The proposed algorithms balance the exploration of the a priori unknown feasible space with the pursuit of global optimality within a prespecified finite number of first-order oracle calls. The first algorithm accommodates an infeasible start and provides either a near-optimal global solution or establishes infeasibility. However, the algorithm may produce infeasible iterates during the search. For a strongly convex constraint function and a feasible initial solution guess, the second algorithm returns a near-optimal global solution without any constraint violation. At each iteration, the algorithms identify the next query point by solving a nonconvex, quadratically constrained, quadratic program, characterized by the Lipschitz continuous gradient property of the functions and the oracle responses. In contrast to existing methods, the algorithms compute global suboptimality bounds at every iteration. They can also satisfy user-specified tolerances in the computed solution with near-optimal complexity in oracle calls for a large class of optimization problems. We implement the proposed algorithms using GUROBI, a commercial off-the-shelf solver.

Exact Markov chain-based runtime analysis of a discrete particle swarm optimization algorithm on sorting and OneMax

Meta-heuristics are powerful tools for solving optimization problems whose structural properties are unknown or cannot be exploited algorithmically. We propose such a meta-heuristic for a large class of optimization problems over discrete domains based on the particle swarm optimization (PSO) paradigm. We provide a comprehensive formal analysis of the performance of this algorithm on certain “easy” reference problems in a black-box setting, namely the sorting problem and the problem OneMax. In our analysis we use a Markov model of the proposed algorithm to obtain upper and lower bounds on its expected optimization time. Our bounds are essentially tight with respect to the Markov model. We show that for a suitable choice of algorithm parameters the expected optimization time is comparable to that of known algorithms and, furthermore, for other parameter regimes, the algorithm behaves less greedy and more explorative, which can be desirable in practice in order to escape local optima. Our analysis provides a precise insight on the tradeoff between optimization time and exploration. To obtain our results we introduce the notion of indistinguishability of states of a Markov chain and provide bounds on the solution of a recurrence equation with non-constant coefficients by integration.

Subexponential Parameterized Algorithms and Kernelization on Almost Chordal Graphs

We study algorithmic properties of the graph class {textsc {Chordal}}{-ke}, that is, graphs that can be turned into a chordal graph by adding at most k edges or, equivalently, the class of graphs of fill-in at most k. It appears that a number of fundamental intractable optimization problems being parameterized by k admit subexponential algorithms on graphs from {textsc {Chordal}}{-ke}. More precisely, we identify a large class of optimization problems on {textsc {Chordal}}{-ke} solvable in time 2^{{mathcal{O}}(sqrt{k}log k)}cdot n^{{mathcal{O}}(1)}. Examples of the problems from this class are finding an independent set of maximum weight, finding a feedback vertex set or an odd cycle transversal of minimum weight, or the problem of finding a maximum induced planar subgraph. On the other hand, we show that for some fundamental optimization problems, like finding an optimal graph coloring or finding a maximum clique, are FPT on {textsc {Chordal}}{-ke} when parameterized by k but do not admit subexponential in k algorithms unless ETH fails. Besides subexponential time algorithms, the class of {textsc {Chordal}}{-ke} graphs appears to be appealing from the perspective of kernelization (with parameter k). While it is possible to show that most of the weighted variants of optimization problems do not admit polynomial in k kernels on {textsc {Chordal}}{-ke} graphs, this does not exclude the existence of Turing kernelization and kernelization for unweighted graphs. In particular, we construct a polynomial Turing kernel for Weighted Clique on {textsc {Chordal}}{-ke} graphs. For (unweighted) Independent Set we design polynomial kernels on two interesting subclasses of {textsc {Chordal}}{-ke}, namely, {textsc {Interval}}{-ke} and {textsc {Split}}{-ke} graphs.

Fast proximal optimization for sparse reconstruction with dictionaries based on translated waveforms

The use of compressive sensing tools has become frequent in signal processing applications when dealing with sparse signals. It has been recently shown that even in the case of very correlated dictionaries, such as dictionaries based on translated waveforms, information can be retrieved from the observations with accuracy. In a recent paper, the authors presented Convolutional Sparse Approximation (CoSA), a fast version of the Iterative Shrinkage and Thresholding Algorithm (ISTA), which is well suited for dictionaries based on this type of waveforms. In this paper, we show that CoSA is indeed a specific case of a much more general framework, and we also present three major practical improvements. Firstly, we show that the acceleration scheme can be fastened for proximal gradient methods by automatically computing the gradient’s step size in a fast and efficient way, based on the dictionary at hand, thus circumventing the manual, trial and error, setups. Secondly, we show that this method can be used on many more penalized regression problems, and illustrate this aspect on Group LASSO. Eventually, we show that this acceleration procedure can be adapted for the alternating direction method of multipliers (ADMM) optimization scheme, which is often used in the field of compressive sensing for a large class of optimization problems. Results on simulations illustrate the efficiency of the method.

Iteratively reweighted $$\ell _1$$ ℓ 1 algorithms with extrapolation

Iteratively reweighted $$\ell _1$$ algorithm is a popular algorithm for solving a large class of optimization problems whose objective is the sum of a Lipschitz differentiable loss function and a possibly nonconvex sparsity inducing regularizer. In this paper, motivated by the success of extrapolation techniques in accelerating first-order methods, we study how widely used extrapolation techniques such as those in Auslender and Teboulle (SIAM J Optim 16:697–725, 2006), Beck and Teboulle (SIAM J Imaging Sci 2:183–202, 2009), Lan et al. (Math Program 126:1–29, 2011) and Nesterov (Math Program 140:125–161, 2013) can be incorporated to possibly accelerate the iteratively reweighted $$\ell _1$$ algorithm. We consider three versions of such algorithms. For each version, we exhibit an explicitly checkable condition on the extrapolation parameters so that the sequence generated provably clusters at a stationary point of the optimization problem. We also investigate global convergence under additional Kurdyka–Łojasiewicz assumptions on certain potential functions. Our numerical experiments show that our algorithms usually outperform the general iterative shrinkage and thresholding algorithm in Gong et al. (Proc Int Conf Mach Learn 28:37–45, 2013) and an adaptation of the iteratively reweighted $$\ell _1$$ algorithm in Lu (Math Program 147:277–307, 2014, Algorithm 7) with nonmonotone line-search for solving random instances of log penalty regularized least squares problems in terms of both CPU time and solution quality.

Posterior agreement for large parameter-rich optimization problems

Most real world combinatorial optimization problems are affected by noise in the input data, thus behaving in the high noise limit like large disordered particle systems, e.g. spin glasses or random networks. Due to uncertainty in the input, optimization of such disordered instances should infer stable posterior distributions of solutions conditioned on the noisy input instance. The maximum entropy principle states that the most stable distribution given the noise influence is defined by the Gibbs distribution and it is characterized by the free energy. In this paper, we first provide rigorous asymptotics of the difficult problem to compute the free energy for two combinatorial optimization problems, namely the sparse Minimum Bisection Problem (sMBP) and Lawler's Quadratic Assignment Problem (LQAP). We prove that both problems exhibit phase transitions equivalent to the discontinuous behavior of Derrida's Random Energy Model (REM). Furthermore, the derived free energy asymptotics lead to a theoretical justification of a recently introduced concept [3] of Gibbs posterior agreement that measures stability of the Gibbs distributions when the cost function fluctuates due to randomness in the input. This relatively new stability concept may potentially provide a new method to select robust solutions for a large class of optimization problems.

A Preconditioning Technique for First-Order Primal-Dual Splitting Method in Convex Optimization

We introduce a preconditioning technique for the first-order primal-dual splitting method. The primal-dual splitting method offers a very general framework for solving a large class of optimization problems arising in image processing. The key idea of the preconditioning technique is that the constant iterative parameters are updated self-adaptively in the iteration process. We also give a simple and easy way to choose the diagonal preconditioners while the convergence of the iterative algorithm is maintained. The efficiency of the proposed method is demonstrated on an image denoising problem. Numerical results show that the preconditioned iterative algorithm performs better than the original one.

Global optimization through a stochastic perturbation of the Polak–Ribière conjugate gradient method

We develop a new modified Polak–Ribière conjugate gradient method by considering a random perturbation. Our approach is suitable for solving a large class of optimization problems on a rectangle of Rn or unconstrained problems. Theoretical results ensure that the proposed method converges to a global minimizer. Numerical experiments are achieved on some typical test problems, particularly the engineering problem of Lennard-Jones clusters. A comparison with well known methods is carried out to show the performance of our algorithm.

On a Class of Optimization Problems with No “Efficiently Computable” Solution

It is well known that large random structures may have nonrandom macroscopic properties. We give an example of nonrandom properties for a class of large optimization problems related to the computational problem MAXFLS= of calculating the maximum number of consistent equations in a given overdetermined system of linear equations. A problem of this kind is faced by a decision maker (an Agent) choosing means to protect a house from natural disasters. For this class we establish the following. There is no “efficiently computable” optimal strategy of the Agent. As the size of a random instance of the optimization problem goes to infinity, the probability that the uniform mixed strategy of the Agent is e-optimal goes to one. Moreover, there is no “efficiently computable” strategy of the Agent that is substantially better for each instance of the optimization problem. Bibliography: 13 titles.

Advances in Theory of Neural Network and Its Application

In this article we introduce a large class of optimization problems that can be approximated by neural networks. Furthermore for some large category of optimization problems the action of the corresponding neural network will be reduced to linear or quadratic programming, therefore the global optimum could be obtained immediately.

On a Class of Optimization Problems with No 'Effectively Computable' Solution

It is well-known that large random structures may have non-random macroscopic properties. We give an example of non-random properties for a class of large optimization problems related to the computational problem MAXFLS^= of calculating the maximal number of consistent equations in a given overdetermined system of linear equations. A problem of this kind is faced by a decision maker (an Agent) choosing the means to protect a house from natural disasters. For this class we establish the following. There is no “efficiently computable” optimal strategy for the Agent. When the size of a random instance of the optimization problem goes to infinity the probability that the uniform mixed strategy of the Agent is e optimal goes to one. Moreover, there is no “efficiently computable” strategy for the Agent which is substantially better for each instance of the optimization problem.

Sensor selection strategies for state estimation in energy constrained wireless sensor networks

Wireless Sensor Networks (WSNs) enable a wealth of new applications where remote estimation is essential. Individual sensors simultaneously sense a dynamic process and transmit measured information over a shared channel to a central base station. The base station computes an estimate of the process state by means of a Kalman filter. In this paper we assume that, at each time step, only a subset of all sensors are selected to send their observations to the fusion center due to channel capacity constraints or limited energy budget. We propose a multi-step sensor selection strategy to schedule sensors to transmit for the next T steps of time with the goal of minimizing an objective function related to the Kalman filter error covariance matrix. This formulation, in a relaxed convex form, defines an unified framework to solve a large class of optimization problems over energy constrained WSNs. We offer some numerical examples to further illustrate the efficiency of the algorithm.

Approximating a class of combinatorial problems with rational objective function

In the late seventies, Megiddo proposed a way to use an algorithm for the problem of minimizing a linear function a 0 + a 1 x 1 + . . . + a n x n subject to certain constraints to solve the problem of minimizing a rational function of the form (a 0 + a 1 x 1 + . . . + a n x n )/(b 0 + b 1 x 1 + . . . + b n x n ) subject to the same set of constraints, assuming that the denominator is always positive. Using a rather strong assumption, Hashizume et al. extended Megiddo’s result to include approximation algorithms. Their assumption essentially asks for the existence of good approximation algorithms for optimization problems with possibly negative coefficients in the (linear) objective function, which is rather unusual for most combinatorial problems. In this paper, we present an alternative extension of Megiddo’s result for approximations that avoids this issue and applies to a large class of optimization problems. Specifically, we show that, if there is an α-approximation for the problem of minimizing a nonnegative linear function subject to constraints satisfying a certain increasing property then there is an α-approximation (1/α-approximation) for the problem of minimizing (maximizing) a nonnegative rational function subject to the same constraints. Our framework applies to covering problems and network design problems, among others.

Design is as Easy as Optimization

We consider the class of max-min and min-max optimization problems subject to a global budget constraint. We undertake a systematic algorithmic and complexity-theoretic study of such problems, which we call design problems. Every optimization problem leads to a natural design problem. Our main result uses techniques of Freund and Schapire [Games Econom. Behav., 29 (1999), pp. 79-103] from learning theory, and its generalizations, to show that for a large class of optimization problems, the design version is as easy as the optimization version. We also observe the relationship between max-min design problems and fractional packing problems. In particular, we obtain in a systematic fashion results about the fractional packing number of Steiner trees.

A note on an approximate lagrange multiplier rule

In this note we show that for a large class of optimization problems, the Lagrange multiplier rule can be derived from the approximate multiplier rule.

Primal-dual approach for directed vertex connectivity augmentation and generalizations

In their seminal paper, Frank and Jordán [1995] show that a large class of optimization problems, including certain directed graph augmentation, fall into the class of covering supermodular functions over pairs of sets. They also give an algorithm for such problems, however, it relies on the ellipsoid method. Prior to our result, combinatorial algorithms existed only for the 0--1 valued problem. Our key result is a combinatorial algorithm for the general problem that includes directed vertex or S − T connectivity augmentation. The algorithm is based on Benczúr's previous algorithm for the 0--1 valued case [Benczúr 2003]. Our algorithm uses a primal-dual scheme for finding covers of partially ordered sets that satisfy natural abstract properties as in Frank and Jordán. For an initial (possibly greedy) cover, the algorithm searches for witnesses for the necessity of each element in the cover. If no two (weighted) witnesses have a common cover, the solution is optimal. As long as this is not the case, the witnesses are gradually exchanged for smaller ones. Each witness change defines an appropriate change in the solution; these changes are finally unwound in a shortest-path manner to obtain a solution of size one less.

Group-strategyproof cost sharing mechanisms for makespan and other scheduling problems

Classical results in economics show that no truthful mechanism can achieve budget balance and efficiency simultaneously. Roughgarden and Sundararajan recently proposed an alternative efficiency measure, which was subsequently used to exhibit that many previously known cost sharing mechanisms approximate both budget balance and efficiency. In this work, we investigate cost sharing mechanisms for combinatorial optimization problems using this novel efficiency measure, with a particular focus on scheduling problems. Our contribution is threefold: First, for a large class of optimization problems that satisfy a certain cost-stability property, we prove that no budget balanced Moulin mechanism can approximate efficiency better than Ω ( log n ) , where n denotes the number of players in the universe. Second, we present a group-strategyproof cost sharing mechanism for the minimum makespan scheduling problem that is tight with respect to budget balance and efficiency. Finally, we show a general lower bound on the budget balance factor for cost sharing methods, which can be used to prove a lower bound of Ω ( n ) on the budget balance factor for completion and flow time scheduling objectives.

Oil Production Optimization in Petroleum Reservoirs by Ant Algorithm

The objective of the research presented in this paper is to investigate the effectiveness of a new stochastic algorithm to obtain maximum hydrocarbon production rate in petroleum reservoirs. The hydrocarbon production rate is a function of several variables. Some of these variables are production well location, production well flow pressure, injection well location and fluid injection rate. Changing any of the variables will change the hydrocarbon production rate value. By using stochastic optimization techniques, there is no limit to the number of decision variables that can be optimized simultaneously. The presented method in this paper is a new metaheuristic algorithm which enables us to analyze a system of mathematical equations containing a large number of decision variables and determine the optimum values of them that should give the most economic result. This is referred to as the Ant Colony Optimization Algorithm. Ant Colony Optimization technique leads us to lower computational cost to optimize complicated problems too. Although applied to discrete domains, this algorithm with some modifications has been applied to continuous optimization. Here, a numerical simulation is initially applied for modeling the hydrocarbon reservoir and then the results are analyzed and furthermore optimized to enhance profitability using Ant Colony Optimization algorithm. The developed code in MATLAB environment, based on Ant Colony Optimization, is able to estimate production and injection wells location, fluid injection rate and well flow pressure which would result in improved hydrocarbon production. This algorithm is simple to implement and the results of case studies show its ability to provide fast and accurate solutions. Results prove the reliability of algorithm for solving a large class of optimization problems in petroleum engineering.

Multiscale multiresolution genetic algorithm with a golden sectioned population composition

AbstractA new genetic algorithm (GA) strategy called the multiscale multiresolution GA is proposed for expediting solution convergence by orders of magnitude. The motivation for this development was to apply GAs to a certain class of large optimization problems, which are otherwise nearly impossible to solve. For the algorithm, standard binary design variables are binary wavelet transformed to multiscale design variables. By working with the multiscale variables, evolution can proceed in multiresolution; converged solutions at a low resolution are reused as a part of individuals of the initial population for the next resolution evolution. It is shown that the best solution convergence can be achieved if three initial population groups having different fitness levels are mixed at the golden section ratio. An analogy between cell division and the proposed multiscale multiresolution strategy is made. The specific applications of the developed method are made in topology optimization problems. Copyright © 2007 John Wiley & Sons, Ltd.

Polynomial transformations and data-independent neighborhood functions

A neighborhood function that is polynomial in size and independent of the problem data, except that it may depend on the maximum absolute value of a number in an instance, is termed semi-reasonable. A semi-data-independent order transformation (SDIOT) is introduced such that if problem A SDIOT to problem B and B has a semi-reasonable neighborhood function, where the number of local optima is polynomial, then problem A has a semi-reasonable neighborhood function such that the number of local optima is polynomial. A large class of optimization problems is shown to SDIOT to Maximum Clause Weighted Satisfiability.