Penalty Function Approaches Research Articles

Fast Compressed Sensing of 3D Radial T1 Mapping with Different Sparse and Low-Rank Models.

Knowledge of the relative performance of the well-known sparse and low-rank compressed sensing models with 3D radial quantitative magnetic resonance imaging acquisitions is limited. We use 3D radial T1 relaxation time mapping data to compare the total variation, low-rank, and Huber penalty function approaches to regularization to provide insights into the relative performance of these image reconstruction models. Simulation and ex vivo specimen data were used to determine the best compressed sensing model as measured by normalized root mean squared error and structural similarity index. The large-scale compressed sensing models were solved by combining a GPU implementation of a preconditioned primal-dual proximal splitting algorithm to provide high-quality T1 maps within a feasible computation time. The model combining spatial total variation and locally low-rank regularization yielded the best performance, followed closely by the model combining spatial and contrast dimension total variation. Computation times ranged from 2 to 113 min, with the low-rank approaches taking the most time. The differences between the compressed sensing models are not necessarily large, but the overall performance is heavily dependent on the imaged object.

On decision-making strategies for improved-reliability size reduction of microwave passives: Intermittent correction of equality constraints and adaptive handling of inequality constraints

Design optimization of passive microwave components is an intricate process, especially if the primary objective is a reduction of the physical size of the structure. The latter has become an important design consideration for a growing number of modern applications (mobile communications, wearable/implantable devices, internet of things), where miniaturization is imperative due to a limited space allocated for the electronic circuitry. Optimization-based size reduction is a heavily constrained task, with several acceptance thresholds imposed on electrical characteristics of the system. The challenges are pronounced whenever equality constraints are involved (e.g., related to power split ratio requirements), in which case the feasible space is a thin set, thereby difficult to be explored throughout the optimization process. This feature makes conventional methods, such as penalty function approaches or algorithms with explicit constraint handling, of limited reliability. In this paper, we introduce a novel technique for reliable control of equality constraints in simulation-driven size reduction of microwave components. Our methodology involves an intermittent optimization-based correction of equality constraints. This is essentially a knowledge-based decision-making strategy implemented as a supplementary optimization stage, and launched before each iteration of the core algorithm. Constraint violation is reduced without being detrimental to the remaining figures of merit, in particular, the circuit size and inequality constraints. Meanwhile, inequality constraints are handled using a penalty function approach with adaptive adjustment of penalty coefficients. The proposed technique facilitates exploration of the feasible space, and allows for achieving reduced miniaturization rates in comparison to the benchmark methods, while ensuring a reliable control of the design constraints. These advantages have been demonstrated using four microstrip couplers, with consistent results obtained for all considered circuits.

Improved contact approaches for irregular polygonal or polyhedral blocks and their applications

Correct mathematical description and mechanical solution towards contact of polygonal or polyhedral blocks are fundamental issues in discontinuous deformation analysis of fractured rock masses. Encountering irregular polygonal or polyhedral shapes (such as blocks with small edges or tiny faces), traditional contact detection and solution approaches may predict incorrect or inaccurate results due to the inappropriate choice of contact parameters or algorithmic robustness problems. Aimed to solve the above disadvantages of conventional contact detection and solution approaches in treating irregular polygonal or polyhedral block shapes, we developed several improved approaches, including (1) the new cover-based contact detection approach to establish inequality constraints for irregular polygons, and (2) the improved potential-based penalty function approaches to compute contact forces for irregular blocks. These improved approaches were proposed based on the contact theory proposed by Shi and the penalty function approach proposed by Munjiza. Several new algorithms, such as the contact trace tracking algorithm, block overlap examination algorithm, intersection block construction algorithm, contact force integration algorithm, were developed. The correctness of proposed approaches was first verified by several benchmarking tests. Several specially designed cases showed that these improved approaches can correctly treat the contact scenarios involving irregular blocks while previous approaches may fail or produce inaccurate results. The proposed approaches were further applied to several application cases which involves a large number of blocks. The proposed approaches and several embedded algorithms enriched the algorithm database for discontinuum-based computational methods.

Cohort intelligence with self-adaptive penalty function approach hybridized with colliding bodies optimization algorithm for discrete and mixed variable constrained problems

Recently, several socio-/bio-inspired algorithms have been proposed for solving a variety of problems. Generally, they perform well when applied for solving unconstrained problems; however, their performance degenerates when applied for solving constrained problems. Several types of penalty function approaches have been proposed so far for handling linear and non-linear constraints. Even though the approach is quite easy to understand, the precise choice of penalty parameter is very much important. It may further necessitate significant number of preliminary trials. To overcome this limitation, a new self-adaptive penalty function (SAPF) approach is proposed and incorporated into socio-inspired Cohort Intelligence (CI) algorithm. This approach is referred to as CI–SAPF. Furthermore, CI–SAPF approach is hybridized with Colliding Bodies Optimization (CBO) algorithm referred to as CI–SAPF–CBO algorithm. The performance of the CI–SAPF and CI–SAPF–CBO algorithms is validated by solving discrete and mixed variable problems from truss structure domain, design engineering domain, and several problems of linear and nonlinear in nature. Furthermore, the applicability of the proposed techniques is validated by solving two real-world applications from manufacturing engineering domain. The results obtained from CI–SAPF and CI–SAPF–CBO are promising and computationally efficient when compared with other nature inspired optimization algorithms. A non-parametric Wilcoxon’s rank sum test is performed on the obtained statistical solutions to examine the significance of CI–SAPF–CBO. In addition, the effect of the penalty parameter on pseudo-objective function, penalty function and constrained violations is analyzed and discussed along with the advantages over other algorithms.

Constrained Nonnegative Matrix Factorization Based on Particle Swarm Optimization for Hyperspectral Unmixing

Spectral unmixing is an important part of hyperspectral image processing. In recent years, constrained nonnegative matrix factorization (CNMF) has been successfully applied for unmixing without the pure-pixel assumption and the result is physically meaningful. However, traditional CNMF algorithms always have two limitations: 1) Most of them are based on gradient methods and usually get trapped in a local optimum. 2) As they adopt static penalty function as the constraint handling method, it's difficult to choose a proper regularization parameter that can balance the tradeoff between reconstruction error and constraint well, which leads to the decreased accuracy. In this paper, we introduce particle swarm optimization (PSO) combined with two types of progressive constraint handling approaches for spectral unmixing in the framework of CNMF. A basic method called high-dimensional double-swarm PSO (HDPSO) algorithm is first proposed. It divides the original high-dimension problem into a series of easier subproblems and adopts two interactive swarms to search endmembers and abundances, respectively. Then, adaptive PSO (APSO) and multiobjective PSO algorithms are proposed by respectively incorporating adaptive penalty function and multiobjective optimization approaches into HDPSO. Experiments with both simulated data and real hyperspectral images are used to compare these methods with traditional algorithms and results validate that the proposed methods give better performance for spectral unmixing.

Quantile-Based Simulation Optimization With Inequality Constraints: Methodology and Applications

Many automation or manufacturing systems are too complex to be modeled by analytical approaches and can only resort to fast-running simulation. Stochastic Nelder–Mead simplex method (SNM) is a newly developed methodology for simulation optimization with expected-value-based objective functions. Quantile, as an important alternative to the usual expected value, provides additional information about the distribution of system performance. In particular, it is useful in describing the tail behavior of the distribution. In this paper, we exploit the structure of SNM and extend it to solve simulation optimization problems with quantile-based objective functions and inequality constraints. The proposed method, called SNM-QC, utilizes the same search strategy as SNM but further incorporates effective quantile estimation techniques and penalty function approaches to solve the problem. We prove that SNM-QC has the desirable global convergence guarantee, i.e., the algorithm is guaranteed to converge to the true optima with probability one. One advantage of SNM-QC is that it is a direct search method that determines the moving direction simply by comparing a set of solutions rather than estimating gradient, thus it can handle many practical problems where gradient does not exist or is difficult to estimate. An extensive numerical study shows that the performance of SNM-QC is promising compared to the existing heuristics. Two illustrative applications are provided in the end to demonstrate the viability of SNM-QC in practical settings.

Integrating Goal Programming, Taylor Series, Kuhn-Tucker Conditions, and Penalty Function Approaches to Solve Linear Fractional Bi-level Programming Problems

In this paper, we integrate goal programming (GP), Tay- lor Series, Kuhn-Tucker conditions and Penalty Function approaches to solve linear fractional bi-level programming (LFBLP)problems. As we know, the Taylor Series is having the property of transforming fractional functions to a polynomial. In the present article by Taylor Series we obtain polynomial objective functions which are equivalent to fractional objective functions. Then on using the Kuhn-Tucker optimality condition of the lower level problem, we transform the linear bilevel programming problem into a corresponding single level programming. The complemen- tary and slackness condition of the lower level problem is appended to the upper level objective with a penalty, that can be reduce to a single objective function. In the other words, suitable transformations can be applied to formulate FBLP problems. Finally a numerical example is given to illustrate the complexity of the procedure to the solution.

Penalty function approaches for ship multidisciplinary design optimisation (MDO)

This paper focuses on the solution of difficult multidisciplinary optimisation formulations arising in ship design. The latter schemes are by nature the result of the interaction among several optimisation problems. Each optimisation problem summarises the issues related to a specific aspect (discipline) of the formulation, and it may be hardly solved by stand-alone methods which ignore the other disciplines. This usually yields very challenging numerical optimisation problems, due to the simultaneous solution of different schemes. In particular, in our ship design applications we stress the strong interaction between fluid-dynamics and optimisation, in order to get remarkable achievements. The ordinary stand-alone methods from mathematical programming prove to be often unsatisfactory on the latter multidisciplinary problems. This scenario requires a specific integration of both fluid-dynamics and optimisation, where constrained optimisation schemes frequently arise. We give evidence that the proper use of penalty methods, combined with global optimisation techniques, may both be a theoretically correct approach, and may yield a fruitful class of techniques for the solution of multidisciplinary problems. We provide numerical results with different penalty functions, over difficult multidisciplinary formulations from ship design. Here, the introduction of penalty methods proved to be a valuable tool since feasibility issues strongly affect the formulation.

A probabilistic constrained nonlinear optimization framework to optimize RED parameters

The random early detection (RED) technique has seen a lot of research over the years. However, the functional relationship between RED performance and its parameters viz., queue weight ( w q ), marking probability ( max p ), minimum threshold ( min t h ) and maximum threshold ( max t h ) is not analytically available. In this paper, we formulate a probabilistic constrained optimization problem by assuming a nonlinear relationship between the RED average queue length and its parameters. This problem involves all the RED parameters as the variables of the optimization problem. We use the barrier and the penalty function approaches for its solution. However (as above), the exact functional relationship between the barrier and penalty objective functions and the optimization variable is not known, but noisy samples of these are available for different parameter values. Thus, for obtaining the gradient and Hessian of the objective, we use certain recently developed simultaneous perturbation stochastic approximation (SPSA) based estimates of these. We propose two four-timescale stochastic approximation algorithms based on certain modified second-order SPSA updates for finding the optimum RED parameters. We present the results of detailed simulation experiments conducted over different network topologies and network/traffic conditions/settings, comparing the performance of our algorithms with variants of RED and a few other well known adaptive queue management (AQM) techniques discussed in the literature.

Multiple trial vectors in differential evolution for engineering design

This article presents a modified version of the differential evolution algorithm to solve engineering design problems. The aim is to allow each parent vector in the population to generate more than one trial (child) vector at each generation and therefore to increase its probability of generating a better one. To deal with constraints, some criteria based on feasibility and a diversity mechanism to maintain infeasible solutions in the population are used. The approach is tested on a set of well-known benchmark problems. After that, it is used to solve engineering design problems and its performance is compared with those provided by typical penalty function approaches and also against state-of-the-art techniques.

Integrating goal programming, Kuhn–Tucker conditions, and penalty function approaches to solve linear bi-level programming problems

Bi-level programming, a tool for modeling decentralized decisions, consists of the objective(s) of the leader at its first level and the follower(s) at the second level. Three level programming results when the second level itself is a bi-level programming which helps us develop the idea of bi-level programming to multi-level programs with any number of levels. There are many approaches for encountering with bi(multi)-level problems. The method introduced by Chenggen Shi et al. [Chenggen Shi, Jie Lu, Guangquan Zhang, An extended Kuhn–Tucker approach for linear bi-level programming, Applied Mathematics and Computation 162 (2005) 51–63] uses extended Kuhn–Tucker conditions to solve bi-level programming problems. In this paper, we integrate goal programming (GP), Kuhn–Tucker conditions (KKT) and Penalty Function approaches to solve linear bi-level programming problems. By using a numerical example we illustrate efficiency of our methodology.

A New Penalty Method for Nonlinear Programming

Penalty function methods, presented many years ago, play exceedingly important roles in the optimization community. According to numerical results, penalty function approaches work very efficiently for equality constrained problems. For inequality constrained problems, sequential quadratic programming (SQP) approaches do better than that of sequential penalty quadratical programming (S lQP) methods. Taking these into account, we propose another optimization approach, in which we aim to combine the advantages of penalty function techniques and SQP approaches. In the new technique, equality constraints are handled by penalty function technique, while inequality constraints are still treated as constraints. The corresponding theories are exploited in this work. The theories of the corresponding augmented Lagrangian function, especially quadratic augmented penalty methods, are also achieved. A new kind of penalty method, combining the advantages of SQP and S lQP, is therefore developed in this work.

The second generation of self-organizing adaptive penalty strategy for constrained genetic search

Penalty function approaches have been extensively applied to genetic algorithms for tackling constrained optimization problems. The effectiveness of the genetic searches to locate the global optimum on constrained optimization problems often relies on the proper selections of many parameters involved in the penalty function strategies. A successful genetic search is often completed after a number of genetic searches with varied combinations of penalty function related parameters. In order to provide a robust and effective penalty function strategy with which the design engineers use genetic algorithms to seek the optimum without the time-consuming tuning process, the self-organizing adaptive penalty strategy (SOAPS) for constrained genetic searches was proposed. This paper proposes the second generation of the self-organizing adaptive penalty strategy (SOAPS-II) to further improve the effectiveness and efficiency of the genetic searches on constrained optimization problems, especially when equality constraints are involved. The results of a number of illustrative testing problems show that the SOAPS-II consistently outperforms other penalty function approaches.

Hybrid-coded crossover for binary-coded genetic algorithms in constrained optimization

The success of genetic searches greatly depends on appropriate selections for many search parameters. This situation becomes more critical for optimization problems involving sharp constraint surfaces. When penalty function approaches are used to handle the constraints, the genetic algorithm conducts the search on a design space which has continually changing contours during the evolution. This often leads to incorrect gene extinctions at early stages of the search, and thus prevents the search from attaining the final global solution. This paper aims to devise a hybrid-coded crossover strategy for binary-coded genetic algorithms (BGA) which combines the major content of two-point crossover and the full domain representation characteristics embedded in many crossover schemes of real-coded genetic algorithms. It is found that the unique hybrid-coded crossover scheme can largely prevent the binary-coded genetic searches from falsely converging to sub-optimal solutions. Comparisons between BGA with the hybrid-coded crossover and the traditional two-point crossover are conducted in a number of illustrative problems with varied combinations of constraints.

A Lagrangian augmented Hopfield network for mixed integer non-linear programming problems

The augmented Lagrangian function has better convergence and solution properties for mixed integer non-linear programming problems than either standard Lagrangian function, or penalty function approaches. The Hopfield neural network based approaches that have been proposed to solve this function have used continuous neurons to represent discrete variables. The augmented Hopfield network has both discrete and continuous neurons. Here a Lagrangian augmented Hopfield network (LAHN) is constructed by including augmented Lagrangian multiplier neurons in the augmented Hopfield network. This new network is applied to the generator scheduling problem (a mixed integer non-linear programming problem) and results illustrate that improved solutions are obtained.

On using exterior penalty approaches for solving linear programming problems

In this paper, we investigate three exterior penalty function approaches for solving linear programming problems. These methods are an active set ℓ2 penalty approach (ASL2), an inequality–equality-based ℓ2 penalty approach (IEL2), and an augmented Lagrangian approach (ALAG). Particular effective variants are explored for each method, along with comments and experience on alternative algorithmic strategies that were empirically investigated. Our motivation is that heretofore, there has been no empirical evidence on how variants of these exterior point algorithms that have been suitably tuned might perform with respect to solving moderately large-scale linear programming problems. Our experiments using a set of randomly generated problems of varying sizes and density, as well as a set of NETLIB test problems, provide insights into the relative performance of these different approaches derived from the basic ℓ2 penalty function, and into their viability for solving linear programs. Average rank tests based on the computational results obtained are performed using two different statistics which assess the speed of convergence and the quality or accuracy of the solution. Overall, a particular variant (ALAG2) of ALAG yielded the best performance with respect to the joint criteria of speed of convergence and the quality of the final solution attained. On the other hand, IEL2 invariably achieved the best quality solutions using comparatively designed termination criteria. However, for highly dense linear programming problems, ASL2 appeared to be the best-alternative among the tested algorithms. Moreover, although our implementation was rudimentary in comparison with CPLEX 6.0, a commercial solver, the tested methods were competitive (sometimes favorable) in attaining a final (though only near optimal) solution for the set of large-scale low-density problems. Scope and purposeAlgorithms based on interior point methods for solving linear programming problems have gained popularity due to some recent techniques which eliminate the ill-conditioning that is inherent within barrier function approaches. Yet, there is comparatively less evidence on the performance of exterior penalty function approaches for solving linear programs. This article investigates three exterior penalty function approaches for solving such problems based on the quadratic penalty function, including the augmented Lagrangian function. Several related algorithmic strategies are explored and tested on a set of randomly generated, degenerate problems, as well as on a standard collection of NETLIB problems. We provide insights into the algorithmic performance and suggest avenues for further research in this relatively less explored domain of approaches for solving linear programming problems.