Exact Penalty Function Research Articles

Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions

Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions

Extending oscars-ii to generally constrained global optimization

AbstractA derivative free method for generally constrained global optimization is described. A non-smooth merit function with one parameter is used. When this parameter equals the optimal objective function value $$f^*$$ f ∗ , the merit function becomes an exact penalty function. The method estimates $$f^*$$ f ∗ , avoiding the need for it to be supplied. The method randomly samples the region satisfying the simple bounds from time to time, ensuring convergence almost surely. Other samples are drawn randomly from smaller regions considered promising. Numerical testing is done using a variety of bound constrained problems and generally constrained problems from the G-suite and elsewhere. Results show the method is competitive in practice. They also show that the method performs better when it estimates the optimal objective function value than when the actual value is used.

Portfolio reshaping under 1st-order stochastic dominance constraints by the exact penalty function methods

The paper studies a financial portfolio selection problem under 1st-order stochastic dominance constraints. These constraints constitute lower bounds on the return profile of the portfolio. In particular, they allow searching for a better portfolio than some reference portfolio by comparing their cumulative distribution functions. Candidate objective functions are the average return, a value at risk, or the average value at risk. The optimization problems obtained are computationally hard because of possibly non-convex constraints and possibly discontinuous objective functions. In the case of a discrete distribution of the return, we develop numerical procedures to solve the problem. The proposed approach uses new exact penalty functions to tackle with the 1st-order stochastic dominance constraints. The resulting penalized objective function is further optimized be the stochastic successive smoothing method as a local optimizer within some branch and bound global optimization scheme. The approach is numerically and graphically illustrated on small portfolio selection problems up to dimension 10.

A Nonconvex Proximal Bundle Method for Nonsmooth Constrained Optimization

An implementable algorithm for solving nonsmooth nonconvex constrained optimization is proposed by combining bundle ideas, proximity control, and the exact penalty function. We construct two kinds of approximations to nonconvex objective function; these two approximations correspond to the convex and concave behaviors of the objective function at the current point, which captures precisely the characteristic of the objective function. The penalty coefficients are increased only a finite number of times under the conditions of Slater constraint qualification and the boundedness of the constrained set, which limit the unnecessary penalty growth. The given algorithm converges to an approximate stationary point of the exact penalty function for constrained nonconvex optimization with weakly semismooth objective function. We also provide the results of some preliminary numerical testing to show the validity and efficiency of the proposed method.

Koopman modeling and optimal control for microbial fed-batch fermentation with switching operators

The modeling of microbial fed-batch fermentation with switching operators to product 1,3-propanediol (1,3-PD) still maintains a challenge because it is strongly of nonlinearity and uncertainty. Machine learning methods for learning such models have become a hot research topic, but the interpretability of existing techniques remains a challenging problem. Recently, the Koopman operator, which is a linear operator governing the eigenfunction evolution along trajectories of a nonlinear dynamical system with switching operators, has been studied for modeling complex dynamics. In this paper, we propose a Koopman modeling method based on an interpretable Koopman operator. The predominant merit of using the Koopman operator is to offer a linear infinite dimensional description of a nonlinear dynamical system with switching operators. In the proposed method, an enhanced learning-based extended dynamic mode decomposition (enhanced-EDMD) algorithm based on a novel eigenfunction construction method is proposed to obtain a finite-dimensional approximation of the Koopman operator. The convergence analysis of the enhanced-EDMD algorithm is also studied. Furthermore, to maximize the productivity of 1,3-PD and minimize the total variation in the optimal control within a time frame, an algorithm combining the model predictive control method with the enhanced learning-based EDMD (denoted by MPC-Enhanced-EDMD), based on gradient-based optimization and exact penalty function method, is proposed for devising optimal feeding rate of glycerol evolving with time. Numerical simulations are conducted by demonstrating the effectiveness of the enhanced-EDMD algorithm on the dynamics prediction and the MPC-Enhanced-EDMD method on the optimal feeding rates of glycerol.

An exact penalty function optimization method and its application in stress constrained topology optimization and scenario based reliability design problems

A smooth penalty function method which does not involve dual and slack implicit variables to formulate constrained optimization conditions is proposed. New active and loss functions are devised to independently and adaptively handle the violation of each constraint function. A single unconstrained minimization of the proposed penalty function produces a solution to the original optimization problem. The constrained optimization conditions depending only on the primal variable are derived and reduce to the canonical Karush-Kuhn-Tucker(KKT) conditions in a much more general framework. The derivative of the proposed loss functions with respect to constraint function can be interpreted as the Lagrange multiplier of the traditional Lagrangian function. The appearance of the first-order Hessian information is unveiled. This is in contrast to the existing works where the classical Lagrange Hessian contains only second order derivatives. Finally, some numerical examples including medium-scale stress constrained topology optimization and scenario based reliability design problems are presented to demonstrate the effectiveness of the proposed methodology.

On Shor's r-Algorithm for Problems with Constraints

Introduction. Nonsmooth optimization problems arise in a wide range of applications, including engineering, finance, and deep learning, where activation functions often have discontinuous derivatives, such as ReLU. Conventional optimization algorithms developed primarily for smooth problems face difficulties when applied to nonsmooth contexts due to discontinuities and other associated irregularities. Possible approaches to overcome these problems include smoothing of functions and applying non-smooth optimization techniques. In particular, Shor's r-algorithm (Shor, Zhurbenko (1971), Shor (1979)) with space stretching in the direction of the difference of two subsequent subgradients is a competitive non-smooth optimization method (Bagirov et al. (2014)). However, the original r-algorithm is designed to minimize unconstrained convex functions. The goal of the work. The standard technique for applying this algorithm to problems with constraints consists in the use of exact non-smooth penalty functions (Eremin (1967), Zangwill (1967)). At the same time, it is necessary to correctly choose (quite large) the penalty coefficient of the penalty functions. Norkin (2020, 2022), Galvan et al. (2021) propose the so-called projective exact penalty functions method, which theoretically does not require choice of the penalty coefficient. The purpose of the present work is to study an applicability of the new exact projective non-smooth penalty functions method for solving conditional problems of non-smooth optimization by Shor's r-algorithm. The results. In this paper, the original optimization problem with convex constraints is first transformed into an unconstrained problem by the projective penalty function method, and then the r-algorithm is used to solve the transformed problem. The results of testing this approach on problems with linear constraints using a program implemented in Matlab are presented. The results of the present study show that the standard method of non-smooth penalties combined with Shor's r-algorithm is fast, due to the use of the provided program to calculate the subgradients, but it requires the correct selection of the penalty parameter. The projective penalty method is slow because in this study it uses finite differences to calculate the gradients, but it is quite stable with respect to the choice of the penalty parameter. Further research will be aimed at investigating the differential properties of the projection mapping and reducing the time of computing subgradients for account of parallel calculations. Keywords: Subgradient descent, constrained optimization, r-algorithm, exact projective penalty.

Dealing with infeasibility in multi-parametric programming for application to explicit model predictive control

Motivated by explicit model predictive control, we address infeasibility in multi-parametric quadratic programming according to the exact penalty function approach, where some user-chosen parameter-dependent constraints are relaxed and the 1-norm of their violation is penalized in the cost function. We characterize the relation between the resulting multi-parametric quadratic program and the original one and show that, as the penalty coefficient grows to infinity, the solution to the former provides a piecewise affine continuous function, which is an optimal solution for the latter over the feasibility region, while it minimizes the 1-norm of the relaxed constraints violation over the infeasibility region.

Solving invex multitime control problems with first‐order PDE constraints via the absolute value exact penalty method

SummaryIn this paper, a nonconvex multitime control problem with first‐order PDE constraints is considered. Then, we investigate the absolute value exact penalty function method which is used for solving the aforesaid control problem. Namely, in order to ensure the effective use of the absolute value exact penalty function method in the considered case, the most important property of any exact penalty function method, that is, exactness of the penalization, is analyzed in the case when the aforementioned method is applied for solving the considered multitime control problem with first‐order PDE constraints in which the functionals involved are nonconvex. Thus, the equivalence between an optimal solution of the aforementioned control problem and a minimizer of its associated unconstrained multitime control problem constructed in the used absolute value exact penalty function method is proved under appropriate invexity hypotheses.

On the exactness and the convergence of the l_{1} exact penalty E-function method for E-differentiable optimization problems

This paper is devoted to introduce and investigate a new exact penalty function method which is called the l_{1} exact penalty E-function method. Namely, we use the aforesaid exact penalty function method to solve a completely new class of nonconvex (not necessarily) differentiable mathematical programming problems, that is, E-differentiable minimization problems. Then, we analyze the most important from a practical point of view property of all exact penalty function methods, that is, exactness of the penalization. Thus, under appropriate E-convexity hypotheses, we prove the equivalence between the original E-differentiable extremum problem and its corresponding penalized optimization problem created in the introduced l_{1} exact penalty E-function method. Further, we also present and investigate the algorithm for this exact penalty function method which minimizes the l_{1} exact penalty E-function. The convergence theorem for the aforesaid algorithm is also established.

An exact penalty function approach for inequality constrained optimization problems based on a new smoothing technique

Exact penalty methods are one of the effective tools to solve nonlinear programming problems with inequality constraints. In this study, a new class of exact penalty functions is defined and a new family of smoothing techniques to exact penalty functions is introduced. Error estimations are presented among the original, non-smooth exact penalty and smoothed exact penalty problems. It is proved that an optimal solution of smoothed penalty problem is an optimal solution of original problem. A smoothing penalty algorithm based on the the new smoothing technique is proposed and the convergence of the algorithm is discussed. Finally, the efficiency of the algorithm on some numerical examples is illustrated.

A hybrid level-based learning swarm algorithm with mutation operator for solving large-scale cardinality-constrained portfolio optimization problems

We propose a hybrid variant of the level-based learning swarm optimizer (LLSO) for solving large-scale portfolio optimization problems. This solver fills the gap due to the inadequacy of the particle swarm optimization algorithm for high-dimensional instances. We aim to extend the classical mean-variance formulation by maximizing a modified version of the Sharpe ratio subject to cardinality, box, and budget constraints. The algorithm involves a projection operator to deal with these three constraints simultaneously. Further, we implicitly control transaction costs thanks to a rebalancing constraint handled by a suitable exact penalty function. In addition, we develop an ad hoc mutation operator to modify candidate exemplars in the highest level of the swarm. The experimental results, using three large-scale data sets, show that including this procedure improves the accuracy of the solutions. Then, a comparison with other variants of the LLSO algorithm and two state-of-the-art swarm optimizers points out the outstanding performance of the proposed solver in terms of exploration capabilities and solution quality. Finally, we assess the profitability of the portfolio allocation strategy in the last five years using an investable pool of 1119 constituents from the MSCI World Index.

Dissolving Constraints for Riemannian Optimization

In this paper, we consider optimization problems over closed embedded submanifolds of [Formula: see text], which are defined by the constraints c(x) = 0. We propose a class of constraint-dissolving approaches for these Riemannian optimization problems. In these proposed approaches, solving a Riemannian optimization problem is transferred into the unconstrained minimization of a constraint-dissolving function ( CDF ). Different from existing exact penalty functions, the exact gradient and Hessian of CDF are easy to compute. We study the theoretical properties of CDF and prove that the original problem and CDF have the same first-order and second-order stationary points, local minimizers, and Łojasiewicz exponents in a neighborhood of the feasible region. Remarkably, the convergence properties of our proposed constraint-dissolving approaches can be directly inherited from the existing rich results in unconstrained optimization. Therefore, the proposed constraint-dissolving approaches build up short cuts from unconstrained optimization to Riemannian optimization. Several illustrative examples further demonstrate the potential of our proposed constraint-dissolving approaches. Funding: The research of N. Xiao and K.-C. Toh is supported by the Ministry of Education of Singapore Academic Research Fund Tier 3 [Grant MOE-2019-T3-1-010]. The research of X. Liu is supported in part by the National Natural Science Foundation of China [Grants 12125108, 11971466, 12288201, 12021001, and 11991021]; the National Key R&D Program of China [Grants 2020YFA0711900 and 2020YFA0711904]; and the Key Research Program of Frontier Sciences, Chinese Academy of Sciences [Grant ZDBS-LY-7022].

Adaptive Exact Penalty Design for Optimal Resource Allocation.

In this article, a distributed adaptive continuous-time optimization algorithm based on the Laplacian-gradient method and adaptive control is designed for resource allocation problem with the resource constraint and the local convex set constraints. In order to deal with local convex sets, a distance-based exact penalty function method is adopted to reformulate the resource allocation problem instead of the widely used projection operator method. By using the nonsmooth analysis and set-valued LaSalle invariance principle, it is proven that the proposed algorithm is capable of solving the nonsmooth resource allocation problem. Finally, two simulation examples are presented to substantiate the theoretical results.

An approach to solve local and global optimization problems based on exact objective filled penalty functions

<p style='text-indent:20px;'>In this work, an exact objective penalty function and an exact objective filled penalty function are proposed to solve constrained optimization problems. There are two main innovations of these two functions: using the exact objective penalty function to find a locally optimal point; a better locally optimal point than the current one can be found by exact objective filled penalty function. A local search method and a global search method for constrained optimization problems are proposed. We can use this new global search method to compute an approximately globally optimal point for constrained optimization problems. The convergence of the sequences obtained by the new algorithms are also analyzed respectively. Finally, the rationality of these two search methods are illustrated by numerical experiments.</p>

On nondifferentiable semi-infinite multiobjective programming with interval-valued functions

In the paper, we investigate a class of nondifferentiable semi-infinite multiobjective programming problems such that all functions constituting them are interval-valued. We derive both Fritz John necessary optimality conditions and, under a constraint qualification, Karush-Kuhn-Tucker necessary optimality conditions for (weak) $ LU $-Pareto solutions in the considered nondifferentiable semi-infinite vector interval-valued optimization problem. Under appropriate invexity hypotheses, we also prove sufficient optimality conditions for this semi-infinite interval-valued vector optimization problem. Further, we also use the $ l_{1} $ exact function method for solving the aforesaid nondifferentiable interval-valued multicriteria optimization problem. Then, we analyze the property of exactness of the penalization for the absolute value exact penalty function method under assumption that the functions involved in the considered semi-infinite multiobjective programming problem are locally Lipschitz interval-valued invex functions. The conditions guaranteeing the equivalence of the sets of (weak) $ LU $-Pareto solutions in the original semi-infinite interval-valued multiobjective programming problem and its associated vector penalized optimization problem with the multiple interval-valued $ l_{1} $ exact penalty function are given.

An optimal PID tuning method for a two-link manipulator via an exact penalty function method

In this paper, an optimal PID tuning method is proposed for a class of two-link manipulators. In particular, the control specifications of the manipulators are considered. By modeling the control specifications into state constraints, the optimal PID tuning problem is converted to an optimal parameters selection problem with state constraints. An exact penalty function based method is then utilized to handle the state constraints. The superior of the proposed method over an existing method is verified by carrying out an numerical example.

Exact augmented Lagrangians for constrained optimization problems in Hilbert spaces I: theory

In this two-part study, we develop a general theory of the so-called exact augmented Lagrangians for constrained optimization problems in Hilbert spaces. In contrast to traditional nonsmooth exact penalty functions, these augmented Lagrangians are continuously differentiable for smooth problems and do not suffer from the Maratos effect, which makes them especially appealing for applications in numerical optimization. Our aim is to present a detailed study of various theoretical properties of exact augmented Lagrangians and discuss several applications of these functions to constrained variational problems, problems with PDE constraints, and optimal control problems. The first paper is devoted to a theoretical analysis of an exact augmented Lagrangian for optimization problems in Hilbert spaces. We obtain several useful estimates of this augmented Lagrangian and its gradient, and present several types of sufficient conditions for KKT-points of a constrained problem corresponding to locally/globally optimal solutions to be local/global minimizers of the exact augmented Lagrangian.

A new projective exact penalty function for a general constrained optimization

A new projective exact penalty function method is proposed for the equivalent reduction of constrained optimization problems to unconstrained ones. In the method, the original objective function is extended to infeasible points by summing its value at the projection of an infeasible point on the feasible set with the distance to the set. The equivalence means that local and global minimums of the problems coincide. Nonconvex sets with multivalued projections are admitted, and the objective function may be lower semicontinuous. The particular case of convex problems is included. So the method does not assume the existence of the objective function outside the allowable area and does not require the selection of the penalty coefficient.

Dynamic flow optimization for a three-loop fluid heat dissipation system in spacecraft

Dynamic optimization of the fluid loop is critical for the active thermal control system (ATCS) for future spacecraft. In this paper, the dynamic heat transfer model of a three-loop fluid heat dissipation system is constructed by the transient heat current modeling method to analyze the optimal control problem of dynamic flow allocation. A sequential quadratic programming (SQP) algorithm combined with the exact external penalty function method is designed to solve the difficulty of temperature path constraints. Simulation results show that the proposed method effectively improves the optimization effect of temperature path constraints and significantly reduces the computational time. Compared with the results of mean allocation flow (unoptimized) and steady flow optimization, the dynamic flow allocation reduced the residual heat by 6.9% and 21.5%, respectively, while meeting all the temperature constraints. In addition, the total flow rate needs to be increased at least by 48% to meet all temperature constraints and achieve similar heat dissipation capacity, when the flow allocation was designed as steady variables. The comparison results indicate that the dynamic flow allocation effectively improves the heat dissipation efficiency of the parallel fluid loop system and ensures the payload temperatures within the constraints.