Penalty Algorithms Research Articles

A genetic algorithm improvement by tour constraint violation penalty discount for maritime cargo delivery

The problem of minimizing the cost of maritime cargo delivery is considered. The cost is equivalent to the sum of the tour lengths of feeders used for the delivery. The problem is formulated as a multiple traveling salesman problem. In order to find its solution as the shortest route of the tours of feeders, a genetic algorithm is used where we present two inequalities constraining the tour length of every feeder to lie between the shortest and longest lengths. Apart from the constant tour constraint violation penalty in the genetic algorithm, we suggest a changeable penalty as an exponential function of the algorithm iteration, where we maintain the possibility of the penalty rate to be either increasing or decreasing, whose steepness is controlled by a positive parameter. Our tests show that the changeable penalty algorithm may return shorter routes, although the constant penalty algorithms cannot be neglected. As the longest possible tour of the feeder is shortened, the changeable penalty becomes more useful owing to a penalty discount required either at the beginning or at the end of the algorithm run to improve the selectivity of the best feeder tours. In optimizing maritime cargo delivery, we propose to run the genetic algorithm by the low and constant penalties along with the increasing and decreasing penalties. The solution is the minimal value of the four route lengths. In addition, we recommend that four algorithm versions be initialized by four different pseudorandom number generator states. The expected gain is a few percent, by which the route length is shortened, but it substantially reduces expenses for maritime cargo delivery.

Improved penalty algorithm for mixed integer PDE constrained optimization problems

Optimal control problems including partial differential equation (PDE) as well as integer constraints merge the combinatorial difficulties of integer programming and the challenges related to large-scale systems resulting from discretized PDEs. So far, the branch-and-bound framework has been the most common solution strategy for such problems. In order to provide an alternative solution approach, especially in a large-scale context, this article investigates penalization techniques. Taking inspiration from a well-known family of existing exact penalty algorithms, a novel improved penalty algorithm is derived, whose key ingredients are a basin hopping strategy and an interior point method, both of which are specialized for the problem class. A thorough numerical investigation is carried out for a standard stationary test problem. Extensions to a convection-diffusion as well as a nonlinear test problem finally demonstrate the versatility of the approach.

A class of exact penalty functions and penalty algorithms for nonsmooth constrained optimization problems

In this paper, a class of smoothing penalty functions is proposed for optimization problems with equality, inequality and bound constraints. It is proved exact, under the condition of weakly generalized Mangasarian–Fromovitz constraint qualification, in the sense that each local optimizer of the penalty function corresponds to a local optimizer of the original problem. Furthermore, necessary and sufficient conditions are discussed for the inverse proposition of exact penalization. Based on the theoretical results in this paper, a class of smoothing penalty algorithms with feasibility verification is presented. Theories on the penalty exactness, feasibility verification and global convergence of the proposed algorithm are presented. Numerical results show that this algorithm is effective for nonsmooth nonconvex constrained optimization problems.

Proximal alternating penalty algorithms for nonsmooth constrained convex optimization

We develop two new proximal alternating penalty algorithms to solve a wide range class of constrained convex optimization problems. Our approach mainly relies on a novel combination of the classical quadratic penalty, alternating minimization, Nesterov’s acceleration, adaptive strategy for parameters. The first algorithm is designed to solve generic and possibly nonsmooth constrained convex problems without requiring any Lipschitz gradient continuity or strong convexity, while achieving the best-known $$\mathcal {O}\left( \frac{1}{k}\right) $$ -convergence rate in a non-ergodic sense, where k is the iteration counter. The second algorithm is also designed to solve non-strongly convex, but semi-strongly convex problems. This algorithm can achieve the best-known $$\mathcal {O}\left( \frac{1}{k^2}\right) $$ -convergence rate on the primal constrained problem. Such a rate is obtained in two cases: (1) averaging only on the iterate sequence of the strongly convex term, or (2) using two proximal operators of this term without averaging. In both algorithms, we allow one to linearize the second subproblem to use the proximal operator of the corresponding objective term. Then, we customize our methods to solve different convex problems, and lead to new variants. As a byproduct, these algorithms preserve the same convergence guarantees as in our main algorithms. We verify our theoretical development via different numerical examples and compare our methods with some existing state-of-the-art algorithms.

The Robustness of Proximal penalty algorithms in Restoration of noisy Image

The nondifferentiable convex optimization has an importance crucial in the image restoration for this and in this article we present the performance of the Prox method adapted to the restoration of noisy images. Following of our article ([12]), we illustrate in this work thesuperior efficacy of this algorithm “Prox” ([12]) then we are comparing the obtained numerical results with the algorithms of Wiener filtering ([7], [16]), total variation ([5]) and wavelet soft-thresholding denoising ([1], [12], [13]), in terms of image quality and convergence. Our first experiments showed that by applying of Prox algorithm for restoration of noised image by the white Gaussian noise we obtain a top results of denosed image with high quality (net, not rehearsed and unsmoothed; textures are preserved) in addition to the convergence of the algorithm is ensured whatever the values of SNR.

New active set identification for general constrained optimization and minimax problems

The purpose of this paper is to discuss the problem of identifying the active constraints for general constrained nonlinear programming and constrained minimax problems at an isolated local solution. Facchinei, Fischer and Kanzow [4] proposed an effective technique which can identify the active set in a neighborhood of an isolated local solution for nonlinear programming, and Han, Jian and Li [5] improved and extended this to minimax problems. In this work, a new active constraint identification set is constructed, not only is it tighter than the previous two identification sets, but also it can be used effectively in penalty algorithms. Without strict complementarity and linear independence, it is shown that the new identification technique can accurately identify the active constraints of nonlinear programming and constrained minimax problems. Some numerical examples illustrate the identification technique.

A new class of exact penalty functions and penalty algorithms

For nonlinear programming problems, we propose a new class of smooth exact penalty functions, which includes both barrier-type and exterior-type penalty functions as special cases. We develop neces...

Dual Convergence for Penalty Algorithms in Convex Programming

Algorithms for convex programming, based on penalty methods, can be designed to have good primal convergence properties even without uniqueness of optimal solutions. Taking primal convergence for granted, in this paper we investigate the asymptotic behavior of an appropriate dual sequence obtained directly from primal iterates. First, under mild hypotheses, which include the standard Slater condition but neither strict complementarity nor second-order conditions, we show that this dual sequence is bounded and also, each cluster point belongs to the set of Karush–Kuhn–Tucker multipliers. Then we identify a general condition on the behavior of the generated primal objective values that ensures the full convergence of the dual sequence to a specific multiplier. This dual limit depends only on the particular penalty scheme used by the algorithm. Finally, we apply this approach to prove the first general dual convergence result of this kind for penalty-proximal algorithms in a nonlinear setting.

Nonsmooth penalty and subgradient algorithms to solve the problem of projection onto a polytope

The least distance problem is considered for a convex hull of a finite family of vectors in a finite-dimensional Euclidian space. It is reduced to an equivalent nonsmooth optimization problem with a directly estimated penalty parameter for which subgradient algorithms with space dilation are proposed.

Penalty Algorithm Based on Conjugate Gradient Method for Solving Portfolio Management Problem

A new approach was proposed to reformulate the biobjectives optimization model of portfolio management into an unconstrained minimization problem, where the objective function is a piecewise quadratic polynomial. We presented some properties of such an objective function. Then, a class of penalty algorithms based on the well-known conjugate gradient methods was developed to find the solution of portfolio management problem. By implementing the proposed algorithm to solve the real problems from the stock market in China, it was shown that this algorithm is promising.

Time Warp Edit Distance with Stiffness Adjustment for Time Series Matching

In a way similar to the string-to-string correction problem, we address discrete time series similarity in light of a time-series-to-time-series-correction problem for which the similarity between two time series is measured as the minimum cost sequence of edit operations needed to transform one time series into another. To define the edit operations, we use the paradigm of a graphical editing process and end up with a dynamic programming algorithm that we call Time Warp Edit Distance (TWED). TWED is slightly different in form from Dynamic Time Warping (DTW), Longest Common Subsequence (LCSS), or Edit Distance with Real Penalty (ERP) algorithms. In particular, it highlights a parameter that controls a kind of stiffness of the elastic measure along the time axis. We show that the similarity provided by TWED is a potentially useful metric in time series retrieval applications since it could benefit from the triangular inequality property to speed up the retrieval process while tuning the parameters of the elastic measure. In that context, a lower bound is derived to link the matching of time series into downsampled representation spaces to the matching into the original space. The empiric quality of the TWED distance is evaluated on a simple classification task. Compared to Edit Distance, DTW, LCSS, and ERP, TWED has proved to be quite effective on the considered experimental task.

Improved convergence order for augmented penalty algorithms

We refine the speed of convergence analysis for the quadratic augmented penalty algorithm. We improve the convergence order from 4/3 to 3/2 for the first order multiplier iteration. For the second order iteration, we generalize the analysis, and consider a primal---dual variant which asymptotically reduces to a Newton step for the optimality conditions.

Leakage Control Through Fine-Grained Placement and Sizing of Sleep Transistors

Multithreshold CMOS (MTCMOS) technology has become a popular technique for standby power reduction. Sleep transistor insertion in circuits is an effective application of MTCMOS technology for reducing leakage power. In this paper, we present a fine-grained approach where each gate in the circuit is provided with an independent sleep transistor. Key advantages of this approach include better circuit slack utilization and improvements in ground-bounce-related signal integrity (which is a major disadvantage in clustering-based approaches). To this end, we propose an optimal polynomial-time fine-grained sleep transistor sizing algorithm. We also prove the selective sleep transistor placement problem as NP-complete and propose an effective heuristic. Finally, in order to reduce the sleep transistor area penalty, we propose a placement-area-constrained sleep transistor sizing formulation. Our experiments show that, on average, the sleep transistor placement and optimal sizing algorithms gave 50.9% and 46.5% savings in leakage power as compared with the conventional fixed-delay penalty algorithms for 5% and 7% circuit slowdown, respectively. Moreover, the postplacement area penalty was less than 5%, which is comparable to clustering schemes.

Smoothed penalty algorithms for optimization of nonlinear models

We introduce an algorithm for solving nonlinear optimization problems with general equality and box constraints. The proposed algorithm is based on smoothing of the exact l 1-penalty function and solving the resulting problem by any box-constraint optimization method. We introduce a general algorithm and present theoretical results for updating the penalty and smoothing parameter. We apply the algorithm to optimization problems for nonlinear traffic network models and report on numerical results for a variety of network problems and different solvers for the subproblems.

Penalty Algorithms in Hilbert Spaces

We analyze the classical penalty algorithm for nonlinear programming in Hilbert spaces and obtain global convergence results, as well as asymptotic superlinear convergence order. These convergence results generalize similar results obtained for finite-dimensional problems. Moreover, the nature of the algorithms allows us to solve the unconstrained subproblems in finite-dimensional spaces.

Augmented penalty algorithms based on BFGS secant approximations and trust regions

An iterative method for solving the nonlinear minimization problem with equality constraints is presented. The method is based on the sequential minimization of the augmented Lagrangian function. The unconstrained minimization subproblems are solved by using a technique based on the conjugate gradients combined with the trust region approach as globalization strategy. The Hessian matrix of the augmented Lagrangian is updated by using a BFGS-like structured secant approximation. Global convergence results are shown. The method is applied to solve the equality constrained nonlinear least squares problem. Numerical results are presented not only with well-known test problems but also with ill-posed problems involving a regularization. Even though the tested problems are small and medium, features like the conjugate gradient/trust region strategy and the structured secant approximation make the proposed algorithm specially efficient for large scale problems.

Primal-Dual-Based Algorithms for a Directed Network Design Problem

We present efficient algorithms for a special case of network design problems, the strong-connectivity problem. Given a directed graph G, the strong-connectivity problem seeks a minimum cost strongly connected spanning subgraph of G. Our algorithms include the primal-dual method, penalty algorithms, and drop algorithms. Primal-dual methods have been quite successful for developing algorithms for undirected network design problems. However, no results are known for extending them to the directed counterparts. We apply the primal-dual method to the strong-connectivity problem, and show that the new algorithm has an approximation guarantee of three. Our computational results over randomly created instances show that the primal-dual is efficient. We develop two improved algorithms, penalty and drop, building on primal-dual as a subroutine.

Augmented non-quadratic penalty algorithms

Auslender, Cominetti and Haddou have studied, in the convex case, a new family of penalty/barrier functions. In this paper, we analyze the asymptotic behavior of augmented penalty algorithms using those penalty functions under the usual second order sufficient optimality conditions, and present order of convergence results (superlinear convergence with order of convergence 4/3). Those results are related to the analysis of ‘‘pure’’ penalty algorithms, as well as ‘‘augmented’’ penalty using a quadratic penalty function. Limited numerical examples are presented to appreciate the practical impact of this local asymptotic analysis.

Numerical Implementation of Dynamic Contact Analysis by a Complementarity Formulation and Application to a Valve-Cotter System in Motorcycle Engines1*

A numerical approach of dynamic contact analysis is presented and applied to the simulation of an axisymmetric valve-cotter system in a motorcycle engine. The contact and the Coulomb friction conditions are described exactly by linear complementarity relations, and discretized dynamic equations are derived. A Newton-type method is used to solve the resulting complementarity problems. This implementation has the advantage of not requiring any user-supplied parameter values to satisfy the contact and friction conditions, compared to penalty or other function-smoothing algorithms. Detailed histories of contact forces and contact status obtained show the applicability of this approach to complex frictional contact problems.

Augmented penalty algorithms

We coin the term augmented penalty method to refer to an augmented Lagrangian-like method in which the penalty parameter is driven to zero instead of being kept bounded away from zero. The spirit of the algorithm is thus that of a classical penalty in which an estimate of the Lagrange multipliers is updated. For the classical augmented Lagrangian function (using the quadratic loss penalty term) applied to equality constrained optimization, Gould (1989 Siam J. Numer. Anal. 26, 107) has obtained a two-step superlinear local convergence result using a constant Lagrange multiplier estimate. Dussault et al have generalized those results to inequality constrained optimization using other penalty terms. On the other hand, convergence and rate of convergence results for augmented Lagrangian methods concern the convergence of the dual variables, usually without relation to the actual effort required to obtain the approximate solutions of the unconstrained primal minimizations. In this paper we consider several updating rules for the Lagrange multiplier estimates and we obtain rate of convergence results for both primal and dual variables, a two-step superlinear convergence of order α, with α < 2. In this result, each iteration uses the solution of a single primal-dual linear system. While this does not improve on the rate of convergence of simple penalty methods, it may alleviate some cancellation errors in internal computations.