R-algorithm for Solving Quadratic Programming Problems

The paper is devoted to the description of a new approach to the construction of algorithms for solving linear programming problems (LP-problems), in which the number of constraints is much greater than the number of variables. It is based on the use of a modification of the r-algorithm to solve the problem of minimizing a nonsmooth function, which is equivalent to LP problem. The advantages of the approach are demonstrated on the linear robust optimization problem and the robust parameters estimation problem using the least moduli method. The developed octave programs are designed to solve LP problems with a very large number of constraints, for which the use of standard software from linear programming is either impossible or impractical, because it requires significant computing resources. The material of the paper is presented in three sections. In the first section for the problem of minimizing a convex function we describe a modification of the r-algorithm with a constant coefficient of space dilation in the direction of the difference of two successive subgradients and an adaptive method for step size adjustment in the direction of the antisubgradient in the transformed space of variables. The software implementation of this modification is presented in the form of Octave function ralgb5a, which allows to find or approximation of the minimum point of a convex function, or approximation of the maximum point of the concave function. The code of the ralgb5a function is given with a brief description of its input and output parameters. In the second section, a method for solving the LP problem is presented using a nonsmooth penalty function in the form of maximum function and the construction of an auxiliary problem of unconstrained minimization of a convex piecewise linear function. The choice of the finite penalty coefficient ensures equivalence between the LP-problem and the auxiliary problem, and the latter is solved using the ralgb5a program. The results of computational experiments in GNU Octave for solving test LP-problems with the number of constraints from two hundred thousand to fifty million and the number of variables from ten to fifty are presented. The third section presents least moduli method that is robust to abnormal observations or "outliers". The method uses the problem of unconstrained minimization of a convex piecewise linear function, and is solved using the ralgb5a program. The results of computational experiments in GNU Octave for solving test problems with a large number of observations (from two hundred thousand to five million) and a small number of unknown parameters (from ten to one hundred) are presented. They demonstrate the superiority of the developed programs over well-known linear programming software such as the GLPK package. Keywords: robust optimization, linear programming problem, nonsmooth penalty function, r-algorithm, least modulus method, GNU Octave.

Although quadratic programming problems are a special class of nonlinear programming, they can also be seen as general linear programming problems. These quadratic problems are of the utmost importance in an increasing variety of practical fields. As, in addition, ambiguity and vagueness are natural and ever-present in real-life situations requiring operative solutions, it makes perfect sense to address them using fuzzy concepts formulated as quadratic programming problems with uncertainty, i.e., as Fuzzy Quadratic Programming problems. This work proposes two novel fuzzy-sets-based methods to solve a particular class of Fuzzy Quadratic Programming problems which have vagueness coefficients in the objective function. Moreover, two other linear approaches are extended to solve the quadratic case. Finally, it is shown that the solutions reached from the extended approaches may be obtained from two proposed parametric multiobjective approaches.

Robust optimization (RO) is a tractable method to address uncertainty in optimization problems where uncertain parameters are modeled as belonging to uncertainty sets that are commonly polyhedral or ellipsoidal. The two most frequently described methods in the literature for solving RO problems are reformulation to a deterministic optimization problem or an iterative cutting-plane method. There has been limited comparison of the two methods in the literature, and there is no guidance for when one method should be selected over the other. In this paper we perform a comprehensive computational study on a variety of problem instances for both robust linear optimization (RLO) and robust mixed-integer optimization (RMIO) problems using both methods and both polyhedral and ellipsoidal uncertainty sets. We consider multiple variants of the methods and characterize the various implementation decisions that must be made. We measure performance with multiple metrics and use statistical techniques to quantify certainty in the results. We find for polyhedral uncertainty sets that neither method dominates the other, in contrast to previous results in the literature. For ellipsoidal uncertainty sets we find that the reformulation is better for RLO problems, but there is no dominant method for RMIO problems. Given that there is no clearly dominant method, we describe a hybrid method that solves, in parallel, an instance with both the reformulation method and the cutting-plane method. We find that this hybrid approach can reduce runtimes to 50–75 % of the runtime for any one method and suggest ways that this result can be achieved and further improved on.

Decomposition methods are well-known techniques for solving quadratic programming (QP) problems arising in support vector machines (SVMs). In each iteration of a decomposition method, a small number of variables are selected and a QP problem with only the selected variables is solved. Since large matrix computations are not required, decomposition methods are applicable to large QP problems. In this paper, we will make a rigorous analysis of the global convergence of general decomposition methods for SVMs. We first introduce a relaxed version of the optimality condition for the QP problems and then prove that a decomposition method reaches a solution satisfying this relaxed optimality condition within a finite number of iterations under a very mild condition on how to select variables.

Interior Point methods for linear and nonlinear optimization problems have received an increasing attention in the last years. A crucial issue in the development of efficient Interior Point software is the solution of the linear system, named KKT system, that arises at each iteration of the method. Iterative solvers appear to be very promising, however the use of effective preconditioners is mandatory because of the increasing ill-conditioning of the system when the iterates generated by the Interior Point method approach the solution. The aim of the research activity described in this thesis has been the analysis, the development and the implementation of iterative methods for the efficient solution of the KKT systems arising at each iteration of Interior Point methods for large-scale convex Quadratic Programming problems. We focus on KKT systems reduced to the well known augmented system and normal equations forms and we consider a preconditioned Conjugate Gradient method for their solution. Specifically we consider an incomplete Cholesky factorization with limited memory for the normal equations approach and a constraint preconditioner for the augmented system approach. In the last case, we analyze the behaviour of the constraint preconditioner with the Conjugate Gradient algorithm and we show, for KKT systems deriving from linear inequality constraints and nonnegativity bounds on the variables, the equivalence with a suitable preconditioned Conjugate Gradient applied to the positive definite normal equations. The Interior Point framework is given by the Potential Reduction method. Due to our interest on the iterative solution of the linear systems, we extend convergence results for the Potential Reduction method in the case of inexact solution of the KKT system arising at each iteration. We describe two software packages that have been developed for solving large-scale convex quadratic problems with the considered algorithms. We discuss some implementation issues, mainly focusing on those related to the solution of the KKT systems with iterative methods. We describe a computational study of stopping criteria of the preconditioned Conjugate Gradient method for solving the KKT systems. We present results of numerical experiments carried out to verify the effectiveness of the proposed approaches on a set of large-scale quadratic problems. We also compare our software packages to a well-estabilished software for nonlinear optimization problems.

Running time analysis of the (1+1)-EA for robust linear optimization

Robust counterpart optimization techniques for linear optimization and mixed integer linear optimization problems are studied in this paper. Different uncertainty sets, including those studied in literature (i.e., interval set; combined interval and ellipsoidal set; combined interval and polyhedral set) and new ones (i.e., adjustable box; pure ellipsoidal; pure polyhedral; combined interval, ellipsoidal, and polyhedral set) are studied in this work and their geometric relationship is discussed. For uncertainty in the left hand side, right hand side, and objective function of the optimization problems, robust counterpart optimization formulations induced by those different uncertainty sets are derived. Numerical studies are performed to compare the solutions of the robust counterpart optimization models and applications in refinery production planning and batch process scheduling problem are presented.

ABSTRACTThe rapid development of renewable energy has promoted the research of the integrated energy system. In particular, the joint optimal scheduling of renewable energy and traditional thermal power units is the key technology to solve the current renewable energy integration and the source network load balance. At present, the total installed capacity of cogeneration units is huge, which hinders the integration of renewable energy and the flexibility of the power grid regulation. Therefore, this paper proposes a robust optimal scheduling strategy for the plant‐level integrated energy system considering electricity‐heat balance. The wind power and solar energy are connected in parallel with the heating network of cogeneration units to decouple heat and power in a more economical and flexible way, while improving the integration of renewable energy. Based on the principle of Kang's robust optimization, robust optimization, a linear robust optimization probability density bias prediction optimization method for the plant‐level integrated energy system considering the electric‐heat balance is proposed in this paper. The dynamic nonlinear constraints in the output process of thermal and electrical loads are transformed into linear, which takes into account the conservatism of robust optimization and the economy of the objective function. The proposed optimization algorithm facilitates the integration of a high proportion of renewable energy into the power grid and is also applicable to other integrated energy systems with nonlinear constraint characteristics.

Graph layout methodologies often solve difficult subproblems in order to satisfy the aesthetic constraints: finding the maximal planar subgraph, minimizing crossings, minimizing area, maximizing symmetries, etc. One standard approach is to translate the subproblem into a linear optimization problem (LOP) and to use a standard mathematical tool to find the solution. The standard solving tool for such LOPs is ILOG CPLEX. ILOG CPLEX delivers high-performance, robust, flexible optimizers for solving linear, mixed-integer and quadratic programming problems (including mixed integer quadratic constrained problems). It is a component that includes C, C++ and Java API, and it is integrated via the Concert Technology into the ILOG Optimization Suite. With ILOG JViews, we also offer a visualization suite that includes sophisticated graph layout algorithms. Our recent investigations focused on cross-product development and the question, how ILOG CPLEX can be used to help graph layout and how ILOG JViews can be used to help the LOP solving.

Compressive sampling matching pursuit (CoSaMP), as a conventional algorithm requiring system sparsity and sensitive to step size, was improved in this paper by approximating the sparsity with adaptive variable step size. In the proposed algorithm (CoSaMP with variable step size abbreviated as Vss-CoSaMP), the idea of approximating sparsity with adaptive step size was borrowed from the sparsity adaptive matching pursuit (SAMP) algorithm to determine the sparsity for the CoSaMP algorithm. The applicability of the CoSaMP algorithm was therefore expanded considerably. On this basis, a step size reduction was added as the iteration termination condition of an orthogonal frequency division multiplexing (OFDM) system. An adaptive variable step size algorithm was then put forward to address the CoSaMP algorithm’s sensitivity to step size. It could realize the required precision at different initial step sizes. A simulation was carried out to analyze the influence of pilot number and step size in an OFDM system on the algorithm. The algorithms, including SAMP, CoSaMP, and Vss-CoSaMP, were compared with two sparse channels, revealing that the Vss-CoSaMP algorithm overcame the problem of the CoSaMP algorithm, that is, the impossibility to forecast the channel sparsity. With the adaptive step size, the proposed algorithm could reach and achieve better accuracy than the CoSaMP algorithm. Additionally, the proposed algorithm was superior over the SAMP algorithm in terms of reconstruction, mean square error (MSE), and bit error ratio (BER).

Introduction. When formulating the classical two-stage transportation problem, it is assumed that the product is transported from suppliers to consumers through intermediate points. Intermediary firms and various kinds of storage facilities (warehouses) can act as intermediate points. The article discusses two mathematical models for two-stage transportation problem (linear programming problem and quadratic programming problem) and a fairly universal way to solve them using modern software. It uses the description of the problem in the modeling language AMPL (A Mathematical Programming Language) and depends on which of the known programs is chosen to solve the problem of linear or quadratic programming. The purpose of the article is to propose the use of AMPL code for solving a linear programming two-stage transportation problem using modern software for linear programming problems, to formulate a mathematical model of a quadratic programming two-stage transportation problem and to investigate its properties. Results. The properties of two variants of a two-stage transportation problem are described: a linear programming problem and a quadratic programming problem. An AMPL code for solving a linear programming two-stage transportation problem using modern software for linear programming problems is given. The results of the calculation using Gurobi program for a linear programming two-stage transportation problem, which has many solutions, are presented and analyzed. A quadratic programming two-stage transportation problem was formulated and conditions were found under which it has unique solution. Conclusions. The developed AMPL-code for a linear programming two-stage transportation problem and its modification for a quadratic programming two-stage transportation problem can be used to solve various logistics transportation problems using modern software for solving mathematical programming problems. The developed AMPL code can be easily adapted to take into account the lower and upper bounds for the quantity of products transported from suppliers to intermediate points and from intermediate points to consumers. Keywords: transportation problem, linear programming problem, AMPL modeling language, Gurobi program, quadratic programming problem.

Several robust counterparts of linear optimization problems with uncertain data were proposed since 1970 and have been extensively studied and extended.In these approaches, the uncertainty set plays an important role since it determines the level of protection of the solution; the solution might be too conservative in order to ensure that the solution remains feasible if the disturbance of data is relatively large.In this paper, we propose a new robust counterpart under a new distance measure.The new approach can ensure that all uncertain data can be mapped to a bounded neighborhood of nominal value regardless of the data from the nominal value either near or far.So the new approach succeeds in reducing the price of robustness; on the other hand, the new robust formulation is also a linear optimization problem.Numerical results for the problems of AFIRO and ADLITTLE from the Net Lib library shown that the effectiveness of the new formulation.

Recently, Ghanbari and Mahdavi-Amiri (Appl Math Model 34:3363–3375, 2010) gave the general compromised solution of an LR fuzzy linear system using ABS algorithm. Here, using this general solution, we solve quadratic programming problems with fuzzy LR variables. We convert fuzzy quadratic programming problem to a crisp quadratic problem by using general solution of fuzzy linear system. By using this method, the crisp optimization problem has fewer variables in comparison with other methods, specially when rank of the coefficient matrix is full. Thus, solving the fuzzy quadratic programming problem by using our proposed method is computationally easier than the solving fuzzy quadratic programming problem by using ranking function. Also, we study the fuzzy quadratic programming problem with symmetric variables. We show that, in this case, the associate quadratic programming problem is a convex problem, and thus, we able to find the global optimal.

We investigate the concept of adjustability—the difference in objective values between two types of dynamic robust optimization formulations: one where (static) decisions are made before uncertainty realization, and one where uncertainty is resolved before (adjustable) decisions. This difference reflects the value of information and decision timing in optimization under uncertainty, and is related to several other concepts such as the optimality of decision rules in robust optimization. We develop a theoretical framework to quantify adjustability based on the input data of a robust optimization problem with a linear objective, linear constraints, and fixed recourse. We make very few additional assumptions. In particular, we do not assume constraint-wise separability or parameter nonnegativity that are commonly imposed in the literature for the study of adjustability. This allows us to study important but previously under-investigated problems, such as formulations with equality constraints and problems with both upper and lower bound constraints. Based on the discovery of an interesting connection between the reformulations of the static and fully adjustable problems, our analysis gives a necessary and sufficient condition—in the form of a theorem-of-the-alternatives—for adjustability to be zero when the uncertainty set is polyhedral. Based on this sharp characterization, we provide two efficient mixed-integer optimization formulations to verify zero adjustability. Then, we develop a constructive approach to quantify adjustability when the uncertainty set is general, which results in an efficient and tight poly-time algorithm to bound adjustability. We demonstrate the efficiency and tightness via both theoretical and numerical analyses.

This paper investigates the general quadratic programming problem, i.e., the problem of finding the minimum of a quadratic function subject to linear constraints. In the case where, over the set of feasible points, the objective function is bounded from below, this problem can be solved by the minimization of a linear function, subject to the solution set of a linear complementarity problem, representing the Kuhn-Tucker conditions of the quadratic problem. To detect in the quadratic problem the unboundedness from below of the objective function, necessary and sufficient conditions are derived. It is shown that, when these conditions are applied, the general quadratic programming problem becomes equivalent to the investigation of an appropriately formulated linear complementarity problem.