Exact Penalty Research Articles

Optimality analysis on ℓ0/1 -sparsity regularized nonsmooth optimization

ABSTRACT We consider the problem of minimizing the sum of a nonsmooth function and an ℓ 0 / 1 -sparsity function, which is called the ℓ 0 / 1 -sparsity regularized nonsmooth optimization ( ℓ 0 / 1 -RNO). In this paper, by defining and analyzing two classes of stationary points, we present the first-order necessary and sufficient optimality conditions for ℓ 0 / 1 -RNO. Moreover, applying these optimality results on ℓ 0 / 1 -RNO, we derive the first-order necessary and sufficient optimality conditions for both the nonsmooth M-estimation of linear regression with positive variable sparsity, and ℓ 1 -penalty minimum of 0/1 composite optimization. Especially, we obtain that the given ℓ 1 -penalty is strongly exact penalty of 0/1 composite optimization.

Entry trajectory optimization of lifting-body vehicle by successive difference-of-convex programming

The complexity of the three-dimensional entry trajectory optimization problem has escalated due to the need to liberalize the angle of attack and bank angle as control variables, thereby enhancing the inherent maneuverability and control capabilities of lifting-body vehicles. The difference-of-convex (DC) properties inherent in the constraints of the problem are exploited in this paper. A DC decomposition approach is utilized to address the nonlinear auxiliary control equations, and the DC relaxation technique is applied to resolve iteration infeasibilities arising from Taylor expansion. The dependence on the initial trajectory is diminished by the implementation of an exact penalty method, thus improving the applicability of the methods. Furthermore, a control variable oscillation suppression mechanism has been constructed to tackle the control variable oscillation issues arising from the relaxation of the angle of attack and bank angle. This mechanism effectively suppresses large jumps in the angle of attack and high-frequency oscillations in the bank angle. Two novel successive DC programming methods are proposed: the successive concave-convex procedure and the successive proximal bundle method, functioning independently of trust-region constraints. Numerical experiments have demonstrated that the two proposed successive DC optimization methods exhibit exceptional performance in accuracy, feasibility, adaptability, and low sensitivity to initial values when applied to solving the three-dimensional entry trajectory optimization problem.

Optimality and KKT conditions for interval valued optimization problems on Hadamard manifolds

Recently, a new type of optimization problems, the so-called interval optimization problems on Hadamard manifolds, is introduced by the authors in Nguyen et al. [Interval optimization problems on Hadamard manifolds. J Nonlinear Convex Anal. 2023;24(11):2489–2511]. In this follow-up, we further offer the algorithmic bricks for these problems. More specifically, we characterize the optimality and KKT conditions for the interval valued optimization problems on Hadamard manifolds. For unconstrained problems, the existence of efficient points and the steepest descent algorithm are investigated. To the contrast, the KKT conditions and exact penalty approach are explored in the ones involving inequality constraints. These results pave the foundations for the solvability of interval valued optimization problems on Hadamard manifolds.

Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions

Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functions

The Importance of Analysis of Judicial Circumstances in the Sentence Dosimetry Process

The Constitution plays an important role as a point of reference for the critical analysis of the criminal system, not only because it derives from the legal assets susceptible to punitive protection, but also and among other reasons, because the principles enshrined as a government norm in the Penal Code governing The imposition of sanctions is a clear development of the constitutional text and doctrine that the Constitutional Court has been building with its pronouncements, in the form that criminal law must acquire. The dosimetry of punishment in Brazil takes a three-step approach, pushing the system of fixed sanctions implemented by the legislature or the full discretion of the judge, promoting, on the other hand, a distribution of functions between the two, as he chooses the minimum and maximum sentence levels and dosage parameters, and this determines the sanction in the specific case from the legal criteria. In this sense, the present research aims to analyze the importance of analyzing judicial circumstances in the process of penalty dosimetry. The specific objectives are: to present a brief history of criminal legislation in Brazil, to review the most recent literature on the application of the penalty and its principles, to analyze the relationship between the dosimetry of the penalty and judicial circumstances and to point out cases in the jurisprudence that materialize the revised theory. Thus, given the relevance of the theme in the current legal scenario, the research is justified by discussing a contemporary theme. Through research on the principles of exceptionality, state of innocence, motivation of decisions and proportionality and the dosimetry of the penalty, possible judgments of the judge that relate the dosimetry of the penalty with the principles of human rights will be discussed. The judge should take charge of the evidence produced in relation to the facts that, in his opinion, influence the minor or major measure of the exact penalty to be imposed. The lack of motivation currently constitutes a violation of the constitutional guarantee of due process and absolute grounds for invalidity under the Criminal Procedure Code.

Adaptive penalty-based neurodynamic approach for nonsmooth interval-valued optimization problem

The complex and diverse practical background drives this paper to explore a new neurodynamic approach (NA) to solve nonsmooth interval-valued optimization problems (IVOPs) constrained by interval partial order and more general sets. On the one hand, to deal with the uncertainty of interval-valued information, the LU-optimality condition of IVOPs is established through a deterministic form. On the other hand, according to the penalty method and adaptive controller, the interval partial order constraint and set constraint are punished by one adaptive parameter, which is a key enabler for the feasibility of states while having a lower solution space dimension and avoiding estimating exact penalty parameters. Through nonsmooth analysis and Lyapunov theory, the proposed adaptive penalty-based neurodynamic approach (APNA) is proven to converge to an LU-solution of the considered IVOPs. Finally, the feasibility of the proposed APNA is illustrated by numerical simulations and an investment decision-making problem.

Extending oscars-ii to generally constrained global optimization

AbstractA derivative free method for generally constrained global optimization is described. A non-smooth merit function with one parameter is used. When this parameter equals the optimal objective function value $$f^*$$ f ∗ , the merit function becomes an exact penalty function. The method estimates $$f^*$$ f ∗ , avoiding the need for it to be supplied. The method randomly samples the region satisfying the simple bounds from time to time, ensuring convergence almost surely. Other samples are drawn randomly from smaller regions considered promising. Numerical testing is done using a variety of bound constrained problems and generally constrained problems from the G-suite and elsewhere. Results show the method is competitive in practice. They also show that the method performs better when it estimates the optimal objective function value than when the actual value is used.

Bayesian Optimization via Exact Penalty

Constrained optimization problems pose challenges when the objective function and constraints are nonconvex and their evaluation requires expensive black-box simulations. Recently, hybrid optimization methods that integrate statistical surrogate modeling with numerical optimization algorithms have shown great promise, as they inherit the properties of global convergence from statistical surrogate modeling and fast local convergence from numerical optimization algorithms. However, the computational efficiency is not satisfied by practical needs under limited budgets and in the presence of equality constraints. In this article, we propose a novel hybrid optimization method, called exact penalty Bayesian optimization (EPBO), which employs Bayesian optimization within the exact penalty framework. We model the composite penalty function by a weighted sum of Gaussian processes, where the qualitative components of the constraint violations are smoothed by their predictive means. The proposed method features (i) closed-form acquisition functions, (ii) robustness to initial designs, (iii) the capability to start from infeasible points, and (iv) effective handling of equality constraints. We demonstrate the superiority of EPBO to state-of-the-art competitors using a suite of benchmark synthetic test problems and two real-world engineering design problems.

Portfolio reshaping under 1st-order stochastic dominance constraints by the exact penalty function methods

The paper studies a financial portfolio selection problem under 1st-order stochastic dominance constraints. These constraints constitute lower bounds on the return profile of the portfolio. In particular, they allow searching for a better portfolio than some reference portfolio by comparing their cumulative distribution functions. Candidate objective functions are the average return, a value at risk, or the average value at risk. The optimization problems obtained are computationally hard because of possibly non-convex constraints and possibly discontinuous objective functions. In the case of a discrete distribution of the return, we develop numerical procedures to solve the problem. The proposed approach uses new exact penalty functions to tackle with the 1st-order stochastic dominance constraints. The resulting penalized objective function is further optimized be the stochastic successive smoothing method as a local optimizer within some branch and bound global optimization scheme. The approach is numerically and graphically illustrated on small portfolio selection problems up to dimension 10.

A Nonconvex Proximal Bundle Method for Nonsmooth Constrained Optimization

An implementable algorithm for solving nonsmooth nonconvex constrained optimization is proposed by combining bundle ideas, proximity control, and the exact penalty function. We construct two kinds of approximations to nonconvex objective function; these two approximations correspond to the convex and concave behaviors of the objective function at the current point, which captures precisely the characteristic of the objective function. The penalty coefficients are increased only a finite number of times under the conditions of Slater constraint qualification and the boundedness of the constrained set, which limit the unnecessary penalty growth. The given algorithm converges to an approximate stationary point of the exact penalty function for constrained nonconvex optimization with weakly semismooth objective function. We also provide the results of some preliminary numerical testing to show the validity and efficiency of the proposed method.

The exact projective penalty method for constrained optimization

The exact projective penalty method for constrained optimization

Riemannian Conjugate Gradient Method Based on Exact Penalty for Phase-Only Beamforming Synthesis

Riemannian Conjugate Gradient Method Based on Exact Penalty for Phase-Only Beamforming Synthesis

The minimax exact penalty fuzzy function method for solving convex nonsmooth optimization problems with fuzzy objective functions

The minimax exact penalty fuzzy function method for solving convex nonsmooth optimization problems with fuzzy objective functions

Optimality conditions and constraint qualifications for cardinality constrained optimization problems

The cardinality constrained optimization problem (CCOP) is an optimization problem where the maximum number of nonzero components of any feasible point is bounded. In this paper, by rewriting the cardinality constraint as a constraint requiring that any feasible point must lie in the union of certain subspaces, we consider CCOP as a mathematical program with disjunctive subspaces constraints (MPDSC). Since a subspace is a special case of a convex polyhedral set, MPDSC is a special case of the mathematical program with disjunctive constraints (MPDC). Using the special structure of subspaces, we are able to obtain more precise formulas for the tangent and (directional) normal cones for the disjunctive set of subspaces. We then obtain first and second order optimality conditions by using the corresponding results from MPDC. Thanks to the special structure of the subspace, we are able to obtain some results for MPDSC that do not hold in general for MPDC. In particular we show that the relaxed constant positive linear dependence (RCPLD) is a sufficient condition for the metric subregularity/error bound property for MPDSC which is not true for MPDC in general. Finally we show that under all constraint qualifications presented in this paper, certain exact penalization holds for CCOP.

Zero-One Composite Optimization: Lyapunov Exact Penalty and a Globally Convergent Inexact Augmented Lagrangian Method

We consider the problem of minimizing the sum of a smooth function and a composition of a zero-one loss function with a linear operator, namely the zero-one composite optimization problem (0/1-COP). It has a vast body of applications, including the support vector machine (SVM), calcium dynamics fitting (CDF), one-bit compressive sensing (1-bCS), and so on. However, it remains challenging to design a globally convergent algorithm for the original model of 0/1-COP because of the nonconvex and discontinuous zero-one loss function. This paper aims to develop an inexact augmented Lagrangian method (IALM), in which the generated whole sequence converges to a local minimizer of 0/1-COP under reasonable assumptions. In the iteration process, IALM performs minimization on a Lyapunov function with an adaptively adjusted multiplier. The involved Lyapunov penalty subproblem is shown to admit the exact penalty theorem for 0/1-COP, provided that the multiplier is optimal in the sense of the proximal-type stationarity. An efficient zero-one Bregman alternating linearized minimization algorithm is also designed to achieve an approximate solution of the underlying subproblem in finite steps. Numerical experiments for handling SVM, CDF, and 1-bCS demonstrate the satisfactory performance of the proposed method in terms of solution accuracy and time efficiency. Funding: This work was supported by the Fundamental Research Funds for the Central Universities [Grant 2022YJS099] and the National Natural Science Foundation of China [Grants 12131004 and 12271022].

Koopman modeling and optimal control for microbial fed-batch fermentation with switching operators

The modeling of microbial fed-batch fermentation with switching operators to product 1,3-propanediol (1,3-PD) still maintains a challenge because it is strongly of nonlinearity and uncertainty. Machine learning methods for learning such models have become a hot research topic, but the interpretability of existing techniques remains a challenging problem. Recently, the Koopman operator, which is a linear operator governing the eigenfunction evolution along trajectories of a nonlinear dynamical system with switching operators, has been studied for modeling complex dynamics. In this paper, we propose a Koopman modeling method based on an interpretable Koopman operator. The predominant merit of using the Koopman operator is to offer a linear infinite dimensional description of a nonlinear dynamical system with switching operators. In the proposed method, an enhanced learning-based extended dynamic mode decomposition (enhanced-EDMD) algorithm based on a novel eigenfunction construction method is proposed to obtain a finite-dimensional approximation of the Koopman operator. The convergence analysis of the enhanced-EDMD algorithm is also studied. Furthermore, to maximize the productivity of 1,3-PD and minimize the total variation in the optimal control within a time frame, an algorithm combining the model predictive control method with the enhanced learning-based EDMD (denoted by MPC-Enhanced-EDMD), based on gradient-based optimization and exact penalty function method, is proposed for devising optimal feeding rate of glycerol evolving with time. Numerical simulations are conducted by demonstrating the effectiveness of the enhanced-EDMD algorithm on the dynamics prediction and the MPC-Enhanced-EDMD method on the optimal feeding rates of glycerol.

Distributed time-varying optimization control protocol for multi-agent systems via finite-time consensus approach

This paper addresses a distributed time-varying optimization problem with inequality constraints based on multi-agent systems over switching communication graphs. To reduce the influence of time-varying inequality constraints, an exact penalty method and smoothing technique are employed. Then, a Hessian-based distributed control protocol is presented to seek the time-varying optimal solution of the distributed time-varying optimization problem by virtue of only local information and interaction. It is shown that all agents not only achieve finite-time consensus but also track the time-varying global optimal target eventually. Compared with the existing distributed optimization protocols, the proposed control protocol is suitable for more general distributed time-varying optimization problems and enjoys high-efficiency convergence. Finally, numerical examples and experiment on moving target tracking of Unmanned Aircraft Vehicle (UAV) are performed to illustrate the effectiveness of the proposed control protocol.

Exact penalty method for knot selection of B-spline regression

Exact penalty method for knot selection of B-spline regression

Accelerated Projection Algorithm Based on Smoothing Approximation for Distributed Nonsmooth Optimization

In this paper, a distributed smoothing accelerated projection algorithm (DSAPA) is proposed to address constrained non-smooth convex optimization problems over undirected multi-agent networks in a distributed manner, where the objective function is free of the assumption of Lipschitz gradient or strong convexity. First, based on a distributed exact penalty method, the original optimization problem is translated to a problem of standard assignment without consensus constraints. Then, a novel DSAPA by combining the smoothing approximation with Nesterov's accelerated schemes is proposed. In addition, we provide a systematic analysis to derive an upper bound on the convergence rate in terms of the objective function based on penalty function and to choose the optimal step size accordingly. Our results demonstrate that the proposed DSAPA can reach <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(\frac{\log (k)}{k})$</tex-math></inline-formula> when the optimal step size is chosen. Finally, the effectiveness and correctness of the proposed algorithm are verified by numerical and practical application examples.

An exact penalty function optimization method and its application in stress constrained topology optimization and scenario based reliability design problems

A smooth penalty function method which does not involve dual and slack implicit variables to formulate constrained optimization conditions is proposed. New active and loss functions are devised to independently and adaptively handle the violation of each constraint function. A single unconstrained minimization of the proposed penalty function produces a solution to the original optimization problem. The constrained optimization conditions depending only on the primal variable are derived and reduce to the canonical Karush-Kuhn-Tucker(KKT) conditions in a much more general framework. The derivative of the proposed loss functions with respect to constraint function can be interpreted as the Lagrange multiplier of the traditional Lagrangian function. The appearance of the first-order Hessian information is unveiled. This is in contrast to the existing works where the classical Lagrange Hessian contains only second order derivatives. Finally, some numerical examples including medium-scale stress constrained topology optimization and scenario based reliability design problems are presented to demonstrate the effectiveness of the proposed methodology.