Equality Constrained Optimization Problems Research Articles

A sequential quadratic programming algorithm for equality-constrained optimization without derivatives

In this paper, we present a new model-based trust-region derivative-free optimization algorithm which can handle nonlinear equality constraints by applying a sequential quadratic programming (SQP) approach. The SQP methodology is one of the best known and most efficient frameworks to solve equality-constrained optimization problems in gradient-based optimization [see e.g. Lalee et al. (SIAM J Optim 8:682–706, 1998), Schittkowski (Optim Lett 5:283–296, 2011), Schittkowski and Yuan (Wiley encyclopedia of operations research and management science, Wiley, New York, 2010)]. Our derivative-free optimization (DFO) algorithm constructs local polynomial interpolation-based models of the objective and constraint functions and computes steps by solving QP sub-problems inside a region using the standard trust-region methodology. As it is crucial for such model-based methods to maintain a good geometry of the set of interpolation points, our algorithm exploits a self-correcting property of the interpolation set geometry. To deal with the trust-region constraint which is intrinsic to the approach of self-correcting geometry, the method of Byrd and Omojokun is applied. Moreover, we will show how the implementation of such a method can be enhanced to outperform well-known DFO packages on smooth equality-constrained optimization problems. Numerical experiments are carried out on a set of test problems from the CUTEst library and on a simulation-based engineering design problem.

Piecewise constant level set method for topology optimization of unilateral contact problems

The paper deals with the structural optimization of the elastic body in unilateral contact with a rigid foundation using the level set approach. A piecewise constant level set method is used to represent the evolution of interfaces rather than the standard method. The piecewise constant level set function takes distinct constant values in each subdomain of a whole design domain. Using a two-phase approximation the original structural optimization problem is reformulated as an equivalent constrained optimization problem in terms of the piecewise constant level set function. Necessary optimality condition is formulated. Finite difference and finite element methods are applied as the approximation methods. Numerical examples are provided and discussed.

A Matrix-Free Trust-Region SQP Method for Equality Constrained Optimization

We develop and analyze a trust-region sequential quadratic programming (SQP) method for the solution of smooth equality constrained optimization problems, which allows the inexact and hence iterative solution of linear systems. Iterative solution of linear systems is important in large-scale applications, such as optimization problems with partial differential equation constraints, where direct solves are either too expensive or not applicable. Our trust-region SQP algorithm is based on a composite-step approach that decouples the step into a quasi-normal and a tangential step. The algorithm includes critical modifications of substep computations needed to cope with the inexact solution of linear systems. The global convergence of our algorithm is guaranteed under rather general conditions on the substeps. We propose algorithms to compute the substeps and prove that these algorithms satisfy global convergence conditions. All components of the resulting algorithm are specified in such a way that they can b...

A Subspace Version of the Powell–Yuan Trust-Region Algorithm for Equality Constrained Optimization

This paper studied subspace properties of the Celis–Dennis–Tapia (CDT) subproblem that arises in some trust-region algorithms for equality constrained optimization. The analysis is an extension of that presented by Wang and Yuan (Numer. Math. 104:241–269, 2006) for the standard trust-region subproblem. Under suitable conditions, it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration. Based on this observation, a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results are also reported.

Smoothly Clipped Absolute Deviation (SCAD) regularization for compressed sensing MRI Using an augmented Lagrangian scheme

PurposeCompressed sensing (CS) provides a promising framework for MR image reconstruction from highly undersampled data, thus reducing data acquisition time. In this context, sparsity-promoting regularization techniques exploit the prior knowledge that MR images are sparse or compressible in a given transform domain. In this work, a new regularization technique was introduced by iterative linearization of the non-convex smoothly clipped absolute deviation (SCAD) norm with the aim of reducing the sampling rate even lower than it is required by the conventional l1 norm while approaching an l0 norm. Materials and MethodsThe CS-MR image reconstruction was formulated as an equality-constrained optimization problem using a variable splitting technique and solved using an augmented Lagrangian (AL) method developed to accelerate the optimization of constrained problems. The performance of the resulting SCAD-based algorithm was evaluated for discrete gradients and wavelet sparsifying transforms and compared with its l1-based counterpart using phantom and clinical studies. The k-spaces of the datasets were retrospectively undersampled using different sampling trajectories. In the AL framework, the CS-MRI problem was decomposed into two simpler sub-problems, wherein the linearization of the SCAD norm resulted in an adaptively weighted soft thresholding rule with a sparsity enhancing effect. ResultsIt was demonstrated that the proposed regularization technique adaptively assigns lower weights on the thresholding of gradient fields and wavelet coefficients, and as such, is more efficient in reducing aliasing artifacts arising from k-space undersampling, when compared to its l1-based counterpart. ConclusionThe SCAD regularization improves the performance of l1-based regularization technique, especially at reduced sampling rates, and thus might be a good candidate for some applications in CS-MRI.

Optimality conditions for 2-regular problems with nonsmooth objective functions

For equality-constrained optimization problems with locally Lipschitzian objective functions, we derive meaningful first-order necessary conditions for local optimality without assuming conventional regularity of constraints. In the case of a smooth objective function, theories of optimality conditions of this kind have been developed in the last three decades. This work extends these results to the nonsmooth case, employing the generalized differentiation concepts of modern nonsmooth analysis. As a by-product of this development, we establish the upper estimate of the Mordukhovich subdifferential of the lower directional derivative. Some applications of these results to the problem of minimization of the maximum function and to the constrained version of a Steiner-type problem are discussed.

A penalty-free method with line search for nonlinear equality constrained optimization

A new line search method is introduced for solving nonlinear equality constrained optimization problems. It does not use any penalty function or a filter. At each iteration, the trial step is determined such that either the value of the objective function or the measure of the constraint violation is sufficiently reduced. Under usual assumptions, it is shown that every limit point of the sequence of iterates generated by the algorithm is feasible, and there exists at least one limit point that is a stationary point for the problem. A simple modification of the algorithm by introducing second order correction steps is presented. It is shown that the modified method does not suffer from the Maratos’ effect, so that it converges superlinearly. The preliminary numerical results are reported.

An extension of the Fletcher–Reeves method to linear equality constrained optimization problem

In this paper, combining the feasible direction method with the conjugate gradient method, we propose a feasible Fletcher–Reeves conjugate gradient method for solving linear equality constrained optimization problem. The directions generated by the method are feasible and descent for the objective function. Under mild conditions, we show that the proposed method with exact line search is globally convergent. Numerical experiments are also given to show the efficiency of the method.

A Nonmonotone Trust Region Algorithm Based on the Average of the Successive Penalty Function Values for Nonlinear Optimization

We present a nonmonotone trust region algorithm for nonlinear equality constrained optimization problems. In our algorithm, we use the average of the successive penalty function values to rectify the ratio of predicted reduction and the actual reduction. Compared with the existing nonmonotone trust region methods, our method is independent of the nonmonotone parameter. We establish the global convergence of the proposed algorithm and give the numerical tests to show the efficiency of the algorithm.

Properties of a class of multilevel optimization algorithms for equality-constrained problems

A multilevel optimization approach (termed MG/Opt) is presented for the solution of equality-constrained optimization problems. The approach assumes that one has a hierarchy of models, ordered from fine to coarse, of an underlying optimization problem and that one is interested in finding solutions at the finest level of detail. In this hierarchy of models, calculations on coarser levels are less expensive, but also are of less fidelity, than calculations on finer levels. The intent of MG/Opt is to use calculations on coarser levels to accelerate the progress of the optimization on the finest level. Global convergence is ensured by requiring a single step of a convergent method on the finest level, plus a line search for incorporating the coarse-level corrections. The convergence results apply to a broad class of algorithms with minimal assumptions about the properties of the coarse models. Under additional assumptions, the coarse-level correction is guaranteed to result in improvement of the fine-level solution. Computational results demonstrate that the multilevel algorithm can dramatically accelerate convergence to the solution.

A Prediction-Correction Dynamic Method for Large-Scale Generalized Eigenvalue Problems

This paper gives a new prediction-correction method based on the dynamical system of differential-algebraic equations for the smallest generalized eigenvalue problem. First, the smallest generalized eigenvalue problem is converted into an equivalent-constrained optimization problem. Second, according to the Karush-Kuhn-Tucker conditions of this special equality-constrained problem, a special continuous dynamical system of differential-algebraic equations is obtained. Third, based on the implicit Euler method and an analogous trust-region technique, a prediction-correction method is constructed to follow this system of differential-algebraic equations to compute its steady-state solution. Consequently, the smallest generalized eigenvalue of the original problem is obtained. The local superlinear convergence property for this new algorithm is also established. Finally, in comparison with other methods, some promising numerical experiments are presented.

A new convergence theory for trust-region algorithm for solving constrained optimization problems

In this paper, we propose a new trust-region algorithm for solving a constrained optimization problem with equality and inequality constraints. In this algorithm, an active-set technique is used to convert the constrained optimization problem with equality and inequality constraints to equality constrained optimization problem. A projected Hessian technique is used together with a conjugate gradient method to compute the trial step. Global convergence results are established under important assumptions and it is shown that a subsequence of the iteration sequence is not bounded away from KKT points. Preliminary numerical experiment on the algorithm is presented. The performance of the algorithm is reported. The numerical results show that our approach is of value and merit further investigation.

Efficient sparse reconstruction algorithm for bioluminescence tomography based on duality and variable splitting

Bioluminescence tomography (BLT) can three-dimensionally and quantitatively resolve the molecular processes in small animals in vivo. In this paper, we propose a BLT reconstruction algorithm based on duality and variable splitting. By using duality and variable splitting to obtain a new equivalent constrained optimization problem and updating the primal variable as the Lagrangian multiplier in the dual augmented Lagrangian problem, the proposed method can obtain fast and stable source reconstruction even without the permissible source region and multispectral measurements. Numerical simulations on a mouse atlas and in vivo mouse experiments were conducted to validate the effectiveness and potential of the method.

A dynamical method of DAEs for the smallest eigenvalue problem

This article gives a new method based on the dynamical system of differential-algebraic equations for the smallest eigenvalue problem of a symmetric matrix. First, the smallest eigenvalue problem is converted into an equivalent constrained optimization problem. Second, from the Karush–Kuhn–Tucker conditions for this special equality-constrained problem, a special continuous dynamical system of differential-algebraic equations is obtained. Lastly, based on the implicit Euler method and an analogous trust-region technique, we obtain a prediction-correction method to compute a steady-state solution of this special system of differential-algebraic equations, and consequently obtain the smallest eigenvalue of the original problem. We also analyze the local superlinear property for this new method, and present the promising numerical results, in comparison with other methods.

On the reduction of the normality conditions in equality-constrained optimization problems in mechanics

A large class of problems in mechanics leads to the minimization of an objective function under equality constraints. In fact, inequality constraints can always be transformed into equality constraints by means of slack variables. The classical approach to solve equality-constrained problems relies on Lagrange multipliers, whose first-order normality conditions (FONC) lead to a system of nonlinear algebraic equations. This system of equations involves as many equations as unknowns, composed of the design variables and Lagrange multipliers, and hence, is amenable to a host of solution methods. In this paper, two methods to eliminate the Lagrange multipliers are reported, by which a reduced system of normality conditions is obtained. Reduction is conducted here either symbolically or numerically using an isotropic orthogonal complementL of the Jacobian matrix of the equality constraints. The relations thus resulting are cast into what is termed the dual form of the FONC. When the problem allows for symbolic calculations, a semi-graphical approach is applied, which leads to the global optimum of the problem at hand. However, the main novelty of the paper lies in an algorithm that returns the stationary points of a constrained optimization problem without requiring the closed-form expressions of the dual form of the FONC. Moreover, numerically efficient and stable procedures are given for the intermediate computational steps. The application of this algorithm is demonstrated with three examples from mechanics.

Semismooth SQP method for equality-constrained optimization problems with an application to the lifted reformulation of mathematical programs with complementarity constraints

We consider the sequential quadratic programming (SQP) algorithm applied to equality-constrained optimization problems, where the problem data is differentiable with Lipschitz-continuous first derivatives. For this setting, Dennis–Moré-type analysis of primal superlinear convergence is presented. Our main motivation is a special modification of SQP tailored to the structure of the lifted reformulation of mathematical programs with complementarity constraints (MPCC). For this problem, we propose a special positive definite modification of the matrices in the generalized Hessian, which is suitable for globalization of SQP based on the penalty function, and at the same time can be expected to satisfy our general Dennis–Moré-type conditions, thus preserving local superlinear convergence. (Standard quasi-Newton updates in the SQP framework require twice differentiability of the problem data at the solution for superlinear convergence.) Preliminary numerical results comparing a number of quasi-Newton versions of semismooth SQP applied to MPCC are also reported.

Set covering-based surrogate approach for solving sup- $${\mathcal{T}}$$ equation constrained optimization problems

This work considers solving the sup- $${\mathcal{T}}$$ equation constrained optimization problems from the integer programming viewpoint. A set covering-based surrogate approach is proposed to solv...

A feasible directions method for nonsmooth convex optimization

We propose a new technique for minimization of convex functions not necessarily smooth. Our approach employs an equivalent constrained optimization problem and approximated linear programs obtained...

High order stability conditions for degenerate optimization problems; elements of [formula omitted]-regularity theory

This paper is devoted to sensitivity analysis of degenerate equality-constrained optimization problems, when the customary regularity of constraints may be violated. Under the p -regularity assumptions we develop p -order stability theory for this class of problems subject to small perturbations and present the estimations of asymptotic behavior of the solutions to the perturbed problems. We derive conditions that guarantee the existence of a solution for perturbed optimization problems and substantiate the assertion of natural character of these conditions. All results are illustrated by classical calculus of variations problems.

On the Kantorovich Theorem and the Regularization of Total Variation Denoising Problems

AbstractTotal variation methods are an optimization-based approach for solving im-age restoration problems. The mathematical formulation results in an equality con-strained optimization problem, a solution which can be obtained using Newton’smethod. This note is motivated by the numerical results of an augmented Lagrangianhomotopy method for the regularization of total variation problems. The numeri-cal technique uses the regularization parameter as a homotopy parameter which isreduced. As a result, a sequence of equality constrained optimization problems issolved using Newton’s method. In this report, the convergence of an augmented La-grangian homotopy method for total variation minimization is addressed. We presenta relationship between the homotopy parameter and the radius of the Kantorovichball. Assistant Professor, Department of Mathematics, Shippensburg University, Shippensburg, PA 17257USAy Member of Technical Staff, National Institute of Standards and Technology, Gaithersburg, MD 20899-8910 USA