Infeasibility Detection Research Articles

Coordinate-update algorithms can efficiently detect infeasible optimization problems

Coordinate update/descent algorithms are widely used in large-scale optimization due to their low per-iteration cost and scalability, but their behavior on infeasible or misspecified problems has not been much studied compared to the algorithms that use full updates. For coordinate-update methods to be as widely adopted to the extent so that they can be used as engines of general-purpose solvers, it is necessary to also understand their behavior under pathological problem instances. In this work, we show that the normalized iterates of randomized coordinate-update fixed-point iterations (RC-FPI) converge to the infimal displacement vector and use this result to design an efficient infeasibility detection method. We then extend the analysis to the setup where the coordinates are defined by non-orthonormal basis using the Friedrichs angle and then apply the machinery to decentralized optimization problems.

Infeasibility Detection with Primal-Dual Hybrid Gradient for Large-Scale Linear Programming

Infeasibility Detection with Primal-Dual Hybrid Gradient for Large-Scale Linear Programming

No-regret constrained Bayesian optimization of noisy and expensive hybrid models using differentiable quantile function approximations

This paper investigates the problem of efficient constrained global optimization of hybrid models that are a composition of a known white-box function and an expensive multi-output black-box function subject to noisy observations, which often arises in real-world science, engineering, manufacturing, and control applications. We propose a novel method, Constrained Upper Quantile Bound (CUQB), to solve such problems that directly exploits the composite structure of the objective and constraint functions that we show leads substantially improved sampling efficiency. CUQB is a conceptually simple, deterministic approach that avoids the constraint approximations used by previous methods. Although the CUQB acquisition function is not available in closed form, we propose a novel differentiable sample average approximation that enables it to be efficiently maximized. We further derive bounds on the cumulative regret and constraint violation under a non-parametric Bayesian representation of the black-box function. Since these bounds depend sublinearly on the number of iterations under some regularity assumptions, we establish explicit bounds on the convergence rate to the optimal solution of the original constrained problem. In contrast to most existing methods, CUQB further incorporates a simple infeasibility detection scheme, which we prove triggers in a finite number of iterations when the original problem is infeasible (with high probability given the Bayesian model). Numerical experiments on several test problems, including environmental model calibration and real-time optimization of a reactor system, show that CUQB significantly outperforms traditional Bayesian optimization in both constrained and unconstrained cases. Furthermore, compared to other state-of-the-art methods that exploit composite structure, CUQB achieves competitive empirical performance while also providing substantially improved theoretical guarantees.

SEQUOIA: A sequential algorithm providing feasibility guarantees for constrained optimization*

Real-time optimal control applications rely on quickly solving nonlinear programs (NLPs) with many constraints. It is usually challenging to design a safe controller that guarantees feasibility of the returned points even in the case of early termination. We present a novel algorithm called SEQUOIA that allows users to choose the maximum optimality gap tolerated and implement sub-optimal, feasible decisions faster in time-critical scenarios. Typical constrained optimization methods approach the feasible set from its exterior, hence the point returned in the case of early termination is not guaranteed to be fit for purpose because critical constraints might be violated. Our algorithm guarantees a feasible point is returned after any iteration and an incrementally better point can be easily constructed. Additionally, we developed the theoretical framework behind our algorithm and show some key mathematical properties. The core idea is to store exact upper and lower bounds for the optimal cost and iteratively solve feasibility problems in order to tighten these bounds. To solve the feasibility problems we use a residual-based approach and we can provide an infeasibility detection feature during the first iteration. We illustrate our implementation on a constrained optimization benchmark as well as a maximum radius orbit transfer optimal control problem. Solving this type of dynamic optimization problems is difficult because of the high number of equality constraints generated from the discretization of dynamic and algebraic path constraints. The numerical experiments show that one can reduce the computational time by more than 40% if the allowed optimality gap is increased from 10−15 to 10−10 and the overall solve time can be decreased by up to two orders of magnitude when compared to a quadratic penalty method (QPM).

Maximum weight perfect matching problem with additional disjunctive conflict constraints

AbstractWe focus on an extension of the maximum weight perfect matching problem with additional disjunctive conflict constraints in conjunction with the degree and binary restrictions. Given a simple graph with a nonnegative weight associated with each edge and a set of conflicting edges, the perfect matching problem with conflict constraints consists of finding a maximum weight perfect matching without any conflicting edge pair. Unlike the well‐known ordinary maximum weight perfect matching problem this one is strongly ‐hard. We propose two branch‐and‐bound algorithms for the exact solution of the problem. The first one is based on an equivalent maximum weight stable set formulation with an additional cardinality restriction obtained on the graph representing conflict relations and uses the information coming from its maximal stable sets. The second one is essentially a recursive depth first search scheme that benefits from simple upper bounds incorporated with a fast infeasibility detection procedure to prune the branch‐and‐bound tree. According to the extensive computational tests it is possible to say that they are both very efficient.

A primal–dual penalty-interior-point method for solving the reactive optimal power flow problem with discrete control variables

The Optimal Reactive Power Flow (ORPF) problem has been used as an important computational tool for power system planning and operation. Its mixed-discrete version (DORPF) is formulated as a non-convex, non-linear optimization problem with discrete and continuous variables, which is aimed at minimizing the transmission losses while meeting the power demand and enforcing operational and physical limits of the system. Although the DORPF problem has been solved by a myriad of methods, some of them present regularization problems associated with poor matrix-conditioning in the optimal solution, some do not provide for rapid infeasibility detection and some do not provide effective ways for handling the discrete nature of the control variables. Although some of these complicating issues have been tackled separately in the literature by various studies, a method that addresses all these issues has not yet been proposed for solving the DORPF problem. In this paper, an integration of optimization approaches is proposed for handling all such complicating issues. The basis of this approach is a primal–dual penalty-interior-point method, which integrates the good properties of penalty methods (e.g. regularization effects on the constraints and rapid infeasibility detection) and interior-point methods (e.g. scalability and good convergence behavior), without suffering from their disadvantages. In the proposed approach, this method is integrated with a sinusoidal penalty function method for handling the discrete nature of the control variables of the DORPF problem, together with a specific inertia correction strategy designed to avoid local maximizers associated with such a penalty function. Numerical tests carried out with the IEEE 30-, 57-, 118- and 300-bus systems focus on showing that all the complicating issues have been addressed by the proposed method. Comparisons with the results obtained by interior-point methods are also provided.

On a primal-dual Newton proximal method for convex quadratic programs

This paper introduces QPDO, a primal-dual method for convex quadratic programs which builds upon and weaves together the proximal point algorithm and a damped semismooth Newton method. The outer proximal regularization yields a numerically stable method, and we interpret the proximal operator as the unconstrained minimization of the primal-dual proximal augmented Lagrangian function. This allows the inner Newton scheme to exploit sparse symmetric linear solvers and multi-rank factorization updates. Moreover, the linear systems are always solvable independently from the problem data and exact linesearch can be performed. The proposed method can handle degenerate problems, provides a mechanism for infeasibility detection, and can exploit warm starting, while requiring only convexity. We present details of our open-source C implementation and report on numerical results against state-of-the-art solvers. QPDO proves to be a simple, robust, and efficient numerical method for convex quadratic programming.

Efficient Semidefinite Programming with Approximate ADMM

Tenfold improvements in computation speed can be brought to the alternating direction method of multipliers (ADMM) for Semidefinite Programming with virtually no decrease in robustness and provable convergence simply by projecting approximately to the Semidefinite cone. Instead of computing the projections via “exact” eigendecompositions that scale cubically with the matrix size and cannot be warm-started, we suggest using state-of-the-art factorization-free, approximate eigensolvers, thus achieving almost quadratic scaling and the crucial ability of warm-starting. Using a recent result from Goulart et al. (Linear Algebra Appl 594:177–192, 2020. https://doi.org/10.1016/j.laa.2020.02.014), we are able to circumvent the numerical instability of the eigendecomposition and thus maintain tight control on the projection accuracy. This in turn guarantees convergence, either to a solution or a certificate of infeasibility, of the ADMM algorithm. To achieve this, we extend recent results from Banjac et al. (J Optim Theory Appl 183(2):490–519, 2019. https://doi.org/10.1007/s10957-019-01575-y) to prove that reliable infeasibility detection can be performed with ADMM even in the presence of approximation errors. In all of the considered problems of SDPLIB that “exact” ADMM can solve in a few thousand iterations, our approach brings a significant, up to 20x, speedup without a noticeable increase in ADMM’s iterations.

Early Termination of Convex QP Solvers in Mixed-Integer Programming for Real-Time Decision Making

The branch-and-bound optimization algorithm for mixed-integer model predictive control (MI-MPC) solves several convex quadratic program relaxations, but often the solutions are discarded based on already known integer feasible solutions. This letter presents a projection and early termination strategy for infeasible interior point methods to reduce the computational effort of finding a globally optimal solution for MI-MPC. The method is shown to be also effective for infeasibility detection of the convex relaxations. We present numerical simulation results with a reduction of the total number of solver iterations by 42% for an MI-MPC example of decision making for automated driving with obstacle avoidance constraints.

An interior point-proximal method of multipliers for convex quadratic programming

In this paper we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM). The resulting algorithm (IP-PMM) is interpreted as a primal-dual regularized IPM, suitable for solving linearly constrained convex quadratic programming problems. We apply few iterations of the interior point method to each sub-problem of the proximal method of multipliers. Once a satisfactory solution of the PMM sub-problem is found, we update the PMM parameters, form a new IPM neighbourhood and repeat this process. Given this framework, we prove polynomial complexity of the algorithm, under standard assumptions. To our knowledge, this is the first polynomial complexity result for a primal-dual regularized IPM. The algorithm is guided by the use of a single penalty parameter; that of the logarithmic barrier. In other words, we show that IP-PMM inherits the polynomial complexity of IPMs, as well as the strict convexity of the PMM sub-problems. The updates of the penalty parameter are controlled by IPM, and hence are well-tuned, and do not depend on the problem solved. Furthermore, we study the behavior of the method when it is applied to an infeasible problem, and identify a necessary condition for infeasibility. The latter is used to construct an infeasibility detection mechanism. Subsequently, we provide a robust implementation of the presented algorithm and test it over a set of small to large scale linear and convex quadratic programming problems. The numerical results demonstrate the benefits of using regularization in IPMs as well as the reliability of the method.

Infeasibility Detection in the Alternating Direction Method of Multipliers for Convex Optimization

The alternating direction method of multipliers is a powerful operator splitting technique for solving structured optimization problems. For convex optimization problems, it is well known that the algorithm generates iterates that converge to a solution, provided that it exists. If a solution does not exist, then the iterates diverge. Nevertheless, we show that they yield conclusive information regarding problem infeasibility for optimization problems with linear or quadratic objective functions and conic constraints, which includes quadratic, second-order cone, and semidefinite programs. In particular, we show that in the limit the iterates either satisfy a set of first-order optimality conditions or produce a certificate of either primal or dual infeasibility. Based on these results, we propose termination criteria for detecting primal and dual infeasibility.

Rapid infeasibility detection in a mixed logarithmic barrier-augmented Lagrangian method for nonlinear optimization

ABSTRACTWe present a modification of a primal-dual algorithm based on a mixed augmented Lagrangian and a log-barrier penalty function. The goal of this new feature is to quickly detect infeasibility. An additional parameter is introduced to balance the minimization of the objective function and the realization of the constraints. The global convergence of the modified algorithm is analysed under mild assumptions. We also show that under a suitable choice of the parameters along the iterations, the rate of convergence of the algorithm to an infeasible stationary point is superlinear. This is the first local convergence result for the class of interior point methods in the infeasible case. We finally report some numerical experiments to show that this new algorithm is quite efficient to detect infeasibility and does not deteriorate the overall behavior in the general case.

An Augmented Lagrangian Method for Equality Constrained Optimization with Rapid Infeasibility Detection Capabilities

We present a primal-dual augmented Lagrangian method for solving an equality constrained minimization problem, which is able to rapidly detect infeasibility. The method is based on a modification of the algorithm proposed in Armand and Omheni (Optim Methods Softw 32(1):1–21, 2017). A new parameter is introduced to scale the objective function and, in case of infeasibility, to force the convergence of the iterates to an infeasible stationary point. It is shown, under mild assumptions, that whenever the algorithm converges to an infeasible stationary point, the rate of convergence is quadratic. This is a new convergence result for the class of augmented Lagrangian methods. The global convergence of the algorithm is also analyzed. It is also proved that, when the algorithm converges to a stationary point, the properties of the original algorithm are preserved. The numerical experiments show that our new approach is as good as the original one when the algorithm converges to a local minimum, but much more efficient in case of infeasibility.

Application of constraint infeasibility detection methods in engineering design problems

PurposeThe purpose of this paper is to present numerical experimentation of three constraint detection methods to explore their main features and drawbacks in infeasibility detection during the design process.Design/methodology/approachThree detection methods (deletion filter, additive method and elasticity method) are used to find the minimum intractable subsystem of constraints in conflict. These methods are tested with four enhanced NLP solvers (sequential quadratic program, multi-start sequential quadratic programing, global optimization solver and genetic algorithm method).FindingsThe additive filtering method with both the multistart sequential quadratic programming and the genetic algorithm solvers is the most efficient method in terms of computation time and accuracy of detecting infeasibility. Meanwhile, the elasticity method has the worst performance.Research limitations/implicationsThe research has been carried out for only inequality constraints and continuous design variables. This research work could be extended to develop computer-aided graphical user interface with the capability of including equality constraints and discrete variables.Practical implicationsThese proposed methods have great potential for finding and guiding the designer to detect the infeasibility for ill-posed complex design problems.Originality/valueThe application of the proposed infeasibility detection methods with their four enhanced solvers on several mechanical design problems reduces the number of constraints to be checked from full set to a much smaller subset.

An interior point method for nonlinear programming with infeasibility detection capabilities

This paper describes an interior point method for nonlinear programming endowed with infeasibility detection capabilities. The method is composed of two phases, a main phase whose goal is to seek optimality, and a feasibility phase that aims exclusively at improving feasibility. An important feature of the algorithm is the use of a step-decomposition interior-point approach in which the step is the sum of a normal component and a tangential component. The normal component of the step provides detailed information that allows the algorithm to determine whether it should transition from the main phase to the feasibility phase. We give particular attention to the reliability of the switching mechanism between the two phases. The algorithm proposed in this paper has been implemented in the knitro package as extensions of the knitro/cg method. Numerical results illustrate the performance of our method on both feasible and infeasible problems.

A Sequential Quadratic Optimization Algorithm with Rapid Infeasibility Detection

We present a sequential quadratic optimization (SQO) algorithm for nonlinear constrained optimization. The method attains all of the strong global and fast local convergence guarantees of classical SQO methods, but has the important additional feature that fast local convergence is guaranteed when the algorithm is employed to solve infeasible instances. A two-phase strategy, carefully constructed parameter updates, and a line search are employed to promote such convergence. The first phase subproblem determines the reduction that can be obtained in a local model of an infeasibility measure when the objective function is ignored. The second phase subproblem then seeks to minimize a local model of the objective while ensuring that the resulting search direction attains a reduction in the local model of the infeasibility measure that is proportional to that attained in the first phase. The subproblem formulations and parameter updates ensure that, near an optimal solution, the algorithm reduces to a classica...

A penalty-interior-point algorithm for nonlinear constrained optimization

Penalty and interior-point methods for nonlinear optimization problems have enjoyed great successes for decades. Penalty methods have proved to be effective for a variety of problem classes due to their regularization effects on the constraints. They have also been shown to allow for rapid infeasibility detection. Interior-point methods have become the workhorse in large-scale optimization due to their Newton-like qualities, both in terms of their scalability and convergence behavior. Each of these two strategies, however, have certain disadvantages that make their use either impractical or inefficient for certain classes of problems. The goal of this paper is to present a penalty-interior-point method that possesses the advantages of penalty and interior-point techniques, but does not suffer from their disadvantages. Numerous attempts have been made along these lines in recent years, each with varying degrees of success. The novel feature of the algorithm in this paper is that our focus is not only on the formulation of the penalty-interior-point subproblem itself, but on the design of updates for the penalty and interior-point parameters. The updates we propose are designed so that rapid convergence to a solution of the nonlinear optimization problem or an infeasible stationary point is attained. We motivate the convergence properties of our algorithm and illustrate its practical performance on large sets of problems, including sets of problems that exhibit degeneracy or are infeasible.

Infeasibility Detection and SQP Methods for Nonlinear Optimization

This paper addresses the need for nonlinear programming algorithms that provide fast local convergence guarantees regardless of whether a problem is feasible or infeasible. We present a sequential quadratic programming method derived from an exact penalty approach that adjusts the penalty parameter automatically, when appropriate, to emphasize feasibility over optimality. The superlinear convergence of such an algorithm to an optimal solution is well known when a problem is feasible. The main contribution of this paper, however, is a set of conditions under which the superlinear convergence of the same type of algorithm to an infeasible stationary point can be guaranteed when a problem is infeasible. Numerical experiments illustrate the practical behavior of the method on feasible and infeasible problems.

Global Optimization of Nonconvex MINLP by a Hybrid Branch-and-Bound and Revised General Benders Decomposition Approach

Mixed-integer nonlinear programming, MINLP, has played a crucial role in chemical process design via superstructures that always involve discrete and continuous variables. In this paper, a global optimization algorithm for nonconvex MINLP problems is developed by addressing the nonconvexity caused by the nonconvex continuous functions with a convex quadratic underestimator within a branch-and-bound framework, as well as the joint problem caused by the mixed natures of integer and continuous variables through a revised general Benders decomposition (GBD) method, where the latter is designed mainly for three favorable structures, i.e., separable, bilinear, and partly linear, between the two domains of continuous and binary variables. The convergence of the revised GBD method to the global solution of the relaxed MINLP subprobelm over each subregion generated in the above framework is guaranteed by the convex underestimation functions in terms of the twice-differentiable assumptions of the continuous functions and the above three favorable joint structures. Then, the convergence of the proposed hybrid algorithm can be established by the exhaustive partition of the constrained region, the monotonicity of the lower bound, and the reliability of the infeasibility detection. Finally, a very simple example for process design is used to verify the different implementation aspects of the proposed approach, especially the unique underestimator construction and the infeasibility detection in each lower-bounding problem.