Exact Penalty Approach Research Articles

Optimality and KKT conditions for interval valued optimization problems on Hadamard manifolds

Recently, a new type of optimization problems, the so-called interval optimization problems on Hadamard manifolds, is introduced by the authors in Nguyen et al. [Interval optimization problems on Hadamard manifolds. J Nonlinear Convex Anal. 2023;24(11):2489–2511]. In this follow-up, we further offer the algorithmic bricks for these problems. More specifically, we characterize the optimality and KKT conditions for the interval valued optimization problems on Hadamard manifolds. For unconstrained problems, the existence of efficient points and the steepest descent algorithm are investigated. To the contrast, the KKT conditions and exact penalty approach are explored in the ones involving inequality constraints. These results pave the foundations for the solvability of interval valued optimization problems on Hadamard manifolds.

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

Abstract This paper is concerned with a class of optimization problems with the non-negative orthogonal constraint, in which the objective function is $L$-smooth on an open set containing the Stiefel manifold $\textrm {St}(n,r)$. We derive a locally Lipschitzian error bound for the feasible points without zero rows when $n>r>1$, and when $n>r=1$ or $n=r$ achieve a global Lipschitzian error bound. Then, we show that the penalty problem induced by the elementwise $\ell _1$-norm distance to the non-negative cone is a global exact penalty, and so is the one induced by its Moreau envelope under a lower second-order calmness of the objective function. A practical penalty algorithm is developed by solving approximately a series of smooth penalty problems with a retraction-based nonmonotone line-search proximal gradient method, and any cluster point of the generated sequence is shown to be a stationary point of the original problem. Numerical comparisons with the ALM [Wen, Z. W. & Yin, W. T. (2013, A feasible method for optimization with orthogonality constraints. Math. Programming, 142, 397–434),] and the exact penalty method [Jiang, B., Meng, X., Wen, Z. W. & Chen, X. J. (2022, An exact penalty approach for optimization with nonnegative orthogonality constraints. Math. Programming. https://doi.org/10.1007/s10107-022-01794-8)] indicate that our penalty method has an advantage in terms of the quality of solutions despite taking a little more time.

An Adaptive Continuous-Time Algorithm for Nonsmooth Convex Resource Allocation Optimization

This article develops a novel continuous-time algorithm based on the idea of adaptive strategy for solving a resource allocation optimization with nonsmooth objective functions and constraints over multiagent network. It is proved that the state solution is globally bounded and finally converges to an optimal solution to the nonsmooth convex resource allocation problem. Compared with the existing algorithms, the strong/strict convexity of the objective function is relaxed and only convexity is required. Moreover, by employing an exact penalty approach for the distributed optimization, the primal-dual variables is avoided to introduce. Therefore, the proposed algorithm has a simple structure with low dimensionality of state variables. To show the effectiveness and practicability of the presented algorithm, a numerical example and an application in power system are presented.

Derivative-free methods for mixed-integer nonsmooth constrained optimization

In this paper, mixed-integer nonsmooth constrained optimization problems are considered, where objective/constraint functions are available only as the output of a black-box zeroth-order oracle that does not provide derivative information. A new derivative-free linesearch-based algorithmic framework is proposed to suitably handle those problems. First, a scheme for bound constrained problems that combines a dense sequence of directions to handle the nonsmoothness of the objective function with primitive directions to handle discrete variables is described. Then, an exact penalty approach is embedded in the scheme to suitably manage nonlinear (possibly nonsmooth) constraints. Global convergence properties of the proposed algorithms toward stationary points are analyzed and results of an extensive numerical experience on a set of mixed-integer test problems are reported.

An exact penalty approach for optimization with nonnegative orthogonality constraints

Optimization with nonnegative orthogonality constraints has wide applications in machine learning and data sciences. It is NP-hard due to some combinatorial properties of the constraints. We first propose an equivalent optimization formulation with nonnegative and multiple spherical constraints and an additional single nonlinear constraint. Various constraint qualifications, the first- and second-order optimality conditions of the equivalent formulation are discussed. By establishing a local error bound of the feasible set, we design a class of (smooth) exact penalty models via keeping the nonnegative and multiple spherical constraints. The penalty models are exact if the penalty parameter is sufficiently large but finite. A practical penalty algorithm with postprocessing is then developed to approximately solve a series of subproblems with nonnegative and multiple spherical constraints. We study the asymptotic convergence and establish that any limit point is a weakly stationary point of the original problem and becomes a stationary point under some additional mild conditions. Extensive numerical results on the problem of computing the orthogonal projection onto nonnegative orthogonality constraints, the orthogonal nonnegative matrix factorization problems and the K-indicators model show the effectiveness of our proposed approach.

G-penalty approach for multi-dimensional control optimisation problem with nonlinear dynamical system

This article studies the exact G-penalty function method to solve a multi-dimensional control optimization problem with non-linear dynamical system involving G-convex functionals. Particularly, the exact G-penalized unconstrained problem associated with the aforementioned problem is constructed using the exact penalty approach. Further, the equivalence relation between an optimal solution of the constrained problem and the minimizer of its related unconstrained problem is established imposing the suitable assumptions on the involved functionals. In addition, several numerical examples are formulated to better illustrate the main results, and an informative algorithm is also constructed to specify the certain steps of the solution procedure.

Finite-Alphabet Symbol-Level Multiuser Precoding for Massive MU-MIMO Downlink

We propose a finite-alphabet symbol-level precoding technique for massive multiuser multiple-input multiple-output (MU-MIMO) downlink systems which are equipped with finite-resolution digital-to-analog converters (DACs) of any precision. Using the idea of constructive interference (CI), we adopt a max-min fair design criterion which aims to maximize the minimum instantaneous received signal-to-noise ratio (SNR) among the user equipments (UEs) while ensuring a CI constraint for each UE under the restriction that the output of the precoder is a vector with finite-alphabet discrete elements. Due to this latter constraint, the design problem is an NP-hard quadratic program with discrete variables, and hence, is difficult to solve. In this paper, we tackle this difficulty by reformulating the problem in several steps into an equivalent continuous-domain biconvex form, including equivalent representations for discrete and binary constraints. Our final biconvex reformulation is obtained via an exact penalty approach and can efficiently be solved using a standard cyclic block coordinate descent algorithm. We evaluate the performance of the proposed finite-alphabet precoding design for DACs with different resolutions, where it is shown that employing low-resolution DACs can lead to higher power efficiencies. In particular, we focus on a setup with one-bit DACs and show through simulation results that compared to the existing schemes, the proposed design can achieve SNR gains of up to 2 dB. We further provide analytic and numerical analyses of complexity and show that our proposed algorithm is computationally efficient as it typically needs only a few tens of iterations to converge.

On the finite element solution of frictionless contact problems using an exact penalty approach

In this article, an exact penalty method is introduced for the enforcement of the impenetrability constraint in problems of frictionless two-body contact solved by the finite element method. A complete algorithmic implementation is presented, including an automated scheme for the selection of the penalty parameter. Numerical examples are employed to assess the accuracy and robustness of the exact penalty method in comparison to classical penalty and Lagrange multiplier alternatives.

Robust identification of nonlinear state-dependent impulsive switched system with switching duration constraints

In this paper, we will study the problem in fed-batch culture of glycerol producing 1,3-propanediol induced by Klebsiella pneumonia. In practice, the concentrations of intracellular substances are difficult to be measured. To address this problem, quantitative biological robustness is introduced to identify the unknown parameters related to the concentrations of intracellular substances. The problem is formulated as an optimization problem governed by a nonlinear state-dependent impulsive switched (NSDIS) system with unknown switching times and a set of system parameters. The formulated optimization problem is subject to parameter constraints and continuous state inequality constraints. The time scaling transformation, an exact penalty method and binary relaxation approach are introduced to solve the problem. Due to the complexity of the formulated problem, a hybrid parallel algorithm through integrating artificial bee colony algorithm and gradient-based exact penalty approach is proposed. The effectiveness of our proposed method is validated through numerical experiments.

Точные штрафы и сопряженная двойственность для обобщенных задач равновесия Нэша со связанными и общими ограничениями

Точные штрафы и сопряженная двойственность для обобщенных задач равновесия Нэша со связанными и общими ограничениями

Using a spectral scaling structured BFGS method for constrained nonlinear least squares

In this paper, we use a spectral scaled structured BFGS formula for approximating projected Hessian matrices in an exact penalty approach for solving constrained nonlinear least-squares problems. We show this spectral scaling formula has a good self-correcting property. The reported numerical results show that the use of the spectral scaling structured BFGS method outperforms the standard structured BFGS method.

Feature selection in machine learning: an exact penalty approach using a Difference of Convex function Algorithm

We develop an exact penalty approach for feature selection in machine learning via the zero-norm $$\ell _{0}$$l0-regularization problem. Using a new result on exact penalty techniques we reformulate equivalently the original problem as a Difference of Convex (DC) functions program. This approach permits us to consider all the existing convex and nonconvex approximation approaches to treat the zero-norm in a unified view within DC programming and DCA framework. An efficient DCA scheme is investigated for the resulting DC program. The algorithm is implemented for feature selection in SVM, that requires solving one linear program at each iteration and enjoys interesting convergence properties. We perform an empirical comparison with some nonconvex approximation approaches, and show using several datasets from the UCI database/Challenging NIPS 2003 that the proposed algorithm is efficient in both feature selection and classification.

Optimal feedback control for dynamic systems with state constraints: An exact penalty approach

In this paper, we consider a class of nonlinear dynamic systems with terminal state and continuous inequality constraints. Our aim is to design an optimal feedback controller that minimizes total system cost and ensures satisfaction of all constraints. We first formulate this problem as a semi-infinite optimization problem. We then show that by using a new exact penalty approach, this semi-infinite optimization problem can be converted into a sequence of nonlinear programming problems, each of which can be solved using standard gradient-based optimization methods. We conclude the paper by discussing applications of our work to glider control.

A two-step superlinearly convergent projected structured BFGS method for constrained nonlinear least squares

We present an exact penalty approach for solving constrained non-linear least-squares problems, when the projected structured Hessian is approximated by a projected version of the structured BFGS formula and prove its local two-step Q-superlinear convergence. The numerical results obtained in an extensive comparative testing experiment confirm the approach to be reliable and efficient.

An exact penalty approach to constrained minimization problems on metric spaces

In this paper we use the penalty approach in order to study a class of constrained minimization problems on complete metric spaces. A penalty function is said to have the generalized exact penalty property if there is a penalty coefficient for which approximate solutions of the unconstrained penalized problem are close enough to approximate solutions of the corresponding constrained problem. For our class of problems we establish the generalized exact penalty property and obtain an estimation of the exact penalty.

The exact penalty principle

The exact penalty approach aims at replacing a constrained optimization problem by an equivalent unconstrained optimization problem. Most results in the literature of exact penalization are mainly concerned with finding conditions under which a solution of the constrained optimization problem is a solution of an unconstrained penalized optimization problem, and the reverse property is rarely studied. In this paper, we study the reverse property. We give the conditions under which the original constrained (single and/or multiobjective) optimization problem and the unconstrained exact penalized problem are exactly equivalent. The main conditions to ensure the exact penalty principle for optimization problems include the global and local error bound conditions. By using variational analysis, these conditions may be characterized by using generalized differentiation.

An Exact Penalty Approach for Mixed Integer Nonlinear Programming Problems

We propose an exact penalty approach for solving mixed integer nonlinear programming (MINLP) problems by converting a general MINLP problem to a finite sequence of nonlinear programming (NLP) problems with only continuous variables. We express conditions of exactness for MINLP problems and show how the exact penalty approach can be extended to constrained problems.

An exact penalty-Lagrangian approach for large-scale nonlinear programming

Nonlinear programming problems with equality constraints and bound constraints on the variables are considered. The presence of bound constraints in the definition of the problem is exploited as much as possible. To this aim, an efficient search direction is defined which is able to produce a locally and superlinearly convergent algorithm and that can be computed in an efficient way by using a truncated scheme suitable for large scale problems. Then, an exact merit function is considered whose analytical expression again exploits the particular structure of the problem by using an exact augmented Lagrangian approach for equality constraints and an exact penalty approach for the bound constraints. It is proved that the search direction and the merit function have some strong connections which can be the basis to define a globally convergent algorithm with superlinear convergence rate for the solution of the constrained problem.

An approach to constrained global optimization based on exact penalty functions

In the field of global optimization many efforts have been devoted to solve unconstrained global optimization problems. The aim of this paper is to show that unconstrained global optimization methods can be used also for solving constrained optimization problems, by resorting to an exact penalty approach. In particular, we make use of a non-differentiable exact penalty function $${P_q(x;\varepsilon)}$$ . We show that, under weak assumptions, there exists a threshold value $${\bar \varepsilon >0 }$$ of the penalty parameter $${\varepsilon}$$ such that, for any $${\varepsilon \in (0, \bar \varepsilon]}$$ , any global minimizer of P q is a global solution of the related constrained problem and conversely. On these bases, we describe an algorithm that, by combining an unconstrained global minimization technique for minimizing P q for given values of the penalty parameter $${\varepsilon}$$ and an automatic updating of $${\varepsilon}$$ that occurs only a finite number of times, produces a sequence {x k } such that any limit point of the sequence is a global solution of the related constrained problem. In the algorithm any efficient unconstrained global minimization technique can be used. In particular, we adopt an improved version of the DIRECT algorithm. Some numerical experimentation confirms the effectiveness of the approach.

Infeasibility Detection and SQP Methods for Nonlinear Optimization

This paper addresses the need for nonlinear programming algorithms that provide fast local convergence guarantees regardless of whether a problem is feasible or infeasible. We present a sequential quadratic programming method derived from an exact penalty approach that adjusts the penalty parameter automatically, when appropriate, to emphasize feasibility over optimality. The superlinear convergence of such an algorithm to an optimal solution is well known when a problem is feasible. The main contribution of this paper, however, is a set of conditions under which the superlinear convergence of the same type of algorithm to an infeasible stationary point can be guaranteed when a problem is infeasible. Numerical experiments illustrate the practical behavior of the method on feasible and infeasible problems.