Penalty Problem Research Articles

A novel penalty function method for semivectorial bilevel programming problem

The bilevel problem is usually much more difficult to solve than the single level problem. If the bilevel problem can be transformed into a single level problem, it will be much easier for one to design effective algorithms. In this paper, a novel penalty function method for semivectorial bilevel programming problem is proposed. First, based on Benson’s method and dual theory of linear programming, the semivectorial bilevel programming problem is transformed into a single level optimization problem by setting the dual gap of the lower level problem to zero; then the transformed problem is further relaxed in order to make the transformed problem solving easier; furthermore, a twice differentiable function called dual gap indicator is designed and a penalty function is constructed by using the dual gap indicator as the penalty term for the relaxed transformed problem. Based on all these, a new penalty function method is proposed, and the solution of the penalty problem can be proved to be the solution of the original problem under mild conditions. Finally, numerical examples are provided to demonstrate the good performance of the proposed algorithm.

<italic>In situ</italic> catalytic hydrogenation of carbon dioxide

Carbon capture and sequestration (CCS) is being explored as potential methods for mitigating CO2 accumulation. However, the high energy input in desorption/compression processes is a barrier. As a nontoxic, abundant, renewable C1 building block, CO2 can be used as an alternative carbonylating reagent to replace phosgene and carbon monoxide in organic synthesis and the chemical industry. The strategy of turning waste gas CO2 into useful compounds and the use of CO2 in the production of high-value-added compounds are important topics, especially the reduction of CO2 to energy products. We describe a strategy involving a combination of CO2 capture and utilization, in which the captured CO2 is considered as the substrate for further in situ catalytic transformations. This process circumvents the desorption and compression steps, and solves the energy penalty problem in CCS. It also avoids the use of high-pressure CO2, which poses safety concerns and is not economically viable. CO2 molecules can also be activated upon adsorption, which is favorable for subsequent in situ conversion under mild conditions. Of the many possible pathways for CO2 use, in situ catalytic hydrogenation is important, and has a wide range of potential application. The CO2 captured as carbamate or carbonic salts can be in situ hydrogenated to formic acid, methanol, and other important energy products. In this review, CO2 adsorption and chemical fixation methods are briefly introduced, and the catalytic systems for CO2 hydrogenation are systematically summarized. In particular, the latest progress on in situ catalytic hydrogenation and mechanistic understanding at the molecular level is discussed in detail.

Incorporating Implicit Link Preference Into Overlapping Community Detection

Community detection is an important technique to understand structures and patterns in complex networks. Recently, overlapping community detection becomes a trend due to the ubiquity of overlapping and nested communities in real world. However, existing approaches have ignored the use of implicit link preference information, i.e., links can reflect a node's preference on the targets of connections it wants to build. This information has strong impact on community detection since a node prefers to build links with nodes inside its community than those outside its community. In this paper, we propose a preference-based nonnegative matrix factorization (PNMF) model to incorporate implicit link preference information. Unlike conventional matrix factorization approaches, which simply approximate the original adjacency matrix in value, our model maximizes the likelihood of the preference order for each node by following the intuition that a node prefers its neighbors than other nodes. Our model overcomes the indiscriminate penalty problem in which non-linked pairs inside one community are equally penalized in objective functions as those across two communities. We propose a learning algorithm which can learn a node-community membership matrix via stochastic gradient descent with bootstrap sampling. We evaluate our PNMF model on several real-world networks. Experimental results show that our model outperforms state-of-the-art approaches and can be applied to large datasets.

On Properties of the Bilinear Penalty Function Method for Mathematical Programs with Semidefinite Cone Complementarity Constraints

This paper is devoted to the study of properties of the bilinear penalty function method for the mathematical programs with semidefinite cone complementarity constraints(SDCMPCC), in which the complementarity constraint appears as a bilinear penalty term in the objective. We discuss the relationships of solutions between the SDCMPCC and its bilinear penalty problem as well as the second order sufficient conditions and the constraint qualifications between these two problems. Moreover, under certain conditions, we demonstrate that any accumulation point of the sequence generated by the bilinear penalty function method is an S-stationary point of the SDCMPCC.

Study of a primal-dual algorithm for equality constrained minimization

The paper proposes a primal-dual algorithm for solving an equality constrained minimization problem. The algorithm is a Newton-like method applied to a sequence of perturbed optimality systems that follow naturally from the quadratic penalty approach. This work is first motivated by the fact that a primal-dual formulation of the quadratic penalty provides a better framework than the standard primal form. This is highlighted by strong convergence properties proved under standard assumptions. In particular, it is shown that the usual requirement of solving the penalty problem with a precision of the same size as the perturbation parameter, can be replaced by a much less stringent criterion, while guaranteeing the superlinear convergence property. A second motivation is that the method provides an appropriate regularization for degenerate problems with a rank deficient Jacobian of constraints. The numerical experiments clearly bear this out. Another important feature of our algorithm is that the penalty parameter is allowed to vary during the inner iterations, while it is usually kept constant. This alleviates the numerical problem due to ill-conditioning of the quadratic penalty, leading to an improvement of the numerical performances.

An improved robust and sparse twin support vector regression via linear programming

Twin support vector regression (TSVR) was proposed recently as a novel regressor that tries to find a pair of nonparallel planes, i.e. $$\epsilon $$ ∈ -insensitive up- and down-bounds, by solving two related SVM-type problems. Though TSVR exhibits good performance compared with conventional methods like SVR, it suffers from the following issues: (1) it lacks model complexity control and thus may incur overfitting and suboptimal solution; (2) it needs to solve a pair of quadratic programming problems which are relatively complex to implement; (3) it is sensitive to outliers; and (4) its solution is not sparse. To address these problems, we propose in this paper a novel regression algorithm termed as robust and sparse twin support vector regression. The central idea is to reformulate TSVR as a convex problem by introducing regularization technique first and then derive a linear programming (LP) formulation which is not only simple but also allows robustness and sparseness. Instead of solving the resulting LP problem in the primal, we present a Newton algorithm with Armijo step-size to resolve the corresponding exact exterior penalty problem. The experimental results on several publicly available benchmark data sets show the feasibility and effectiveness of the proposed method.

Robust and Sparse Linear Programming Twin Support Vector Machines

In this paper, we propose a new linear programming formulation of exact 1-norm twin support vector machine (TWSVM) for classification whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton–Armijo algorithm. The idea of our formulation is to reformulate TWSVM as a strongly convex problem by incorporated regularization techniques and then derive an exact 1-norm linear programming formulation for TWSVM to improve robustness and sparsity. The solution of two modified unconstrained minimization problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems in TWSVM and TBSVM, which leads to extremely simple and fast algorithm. One significant advantage of our proposed method is the implementation of structural risk minimization principle. However, only empirical risk is considered in the primal problems of TWSVM due to its complex structure and thus may incur overfitting and suboptimal in some cases. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point that can be easily implemented in MATLAB without using any optimization packages. Computational comparisons of our proposed method against original TWSVM, GEPSVM and SVM have been made on both synthetic and benchmark datasets. Experimental results show that our method is better or comparable in both computation time and classification accuracy.

An integro-differential parabolic variational inequality arising from the valuation of double barrier American option

This paper studies the nonlinear variational inequality with integro-differential term arising from valuation of American style double barrier option. First, the authors use the penalty method to transform the variational inequality into a nonlinear parabolic initial boundary problem (i.e., penalty problem). Second, the existence and uniqueness of solution to the penalty problem are proved by using the Scheafer fixed point theory. Third, the authors prove the existence of variational inequality’ solution by showing the fact that the penalized PDE converges to the variational inequality. The uniqueness of solution to the variational inequality is also proved by contradiction.

Second-order integro-differential parabolic variational inequalities arising from the valuation of American option

This article deals with the existence and uniqueness of solution to the Rd1 -valued parabolic variational inequalities with integro-differential terms which arise from the valuation of American option. The authors use the penalty method to construct a sequence of approximation parabolic problem and hence obtain the existence and uniqueness of solution to the approximation problem by using fixed point theory. Then the solution of parabolic variational inequalities is obtained by showing that the solution of this penalty problem converges to the variational inequalities. The uniqueness of the solution is also proven. MSC: 35B40; 35K35

Exact Penalty Decomposition Method for Zero-Norm Minimization Based on MPEC Formulation

We reformulate the zero-norm minimization problem as an equivalent mathematical program with equilibrium constraints and establish that its penalty problem, induced by adding the complementarity constraint to the objective, is exact. Then, by the special structure of the exact penalty problem, we propose a decomposition method that can seek a global optimal solution of the zero-norm minimization problem under the null space condition in [M. A. Khajehnejad et al. IEEE Trans. Signal. Process., 59 (2011), pp. 1985--2001] by solving a finite number of weighted $l_1$-norm minimization problems. To handle the weighted $l_1$-norm subproblems, we develop a partial proximal point algorithm where the subproblems may be solved approximately with the limited memory BFGS (L-BFGS) or the semismooth Newton-CG. Finally, we apply the exact penalty decomposition method with the weighted $l_1$-norm subproblems solved by combining the L-BFGS with the semismooth Newton-CG to several types of sparse optimization problems, and ...

New Exact Penalty Functions for Nonlinear Constrained Optimization Problems

For two kinds of nonlinear constrained optimization problems, we propose two simple penalty functions, respectively, by augmenting the dimension of the primal problem with a variable that controls the weight of the penalty terms. Both of the penalty functions enjoy improved smoothness. Under mild conditions, it can be proved that our penalty functions are both exact in the sense that local minimizers of the associated penalty problem are precisely the local minimizers of the original constrained problem.

Exact Penalization and Necessary Optimality Conditions for Multiobjective Optimization Problems with Equilibrium Constraints

A calmness condition for a general multiobjective optimization problem with equilibrium constraints is proposed. Some exact penalization properties for two classes of multiobjective penalty problems are established and shown to be equivalent to the calmness condition. Subsequently, a Mordukhovich stationary necessary optimality condition based on the exact penalization results is obtained. Moreover, some applications to a multiobjective optimization problem with complementarity constraints and a multiobjective optimization problem with weak vector variational inequality constraints are given.

Divergence-Conforming Discontinuous Galerkin Methods and $C^0$ Interior Penalty Methods

In this paper, we show that recently developed divergence-conforming methods for the Stokes problem have discrete stream functions. These stream functions in turn solve a continuous interior penalty problem for biharmonic equations. The equivalence is established for the most common methods in two dimensions based on interior penalty terms. Then, extensions of the concept to discontinuous Galerkin methods defined through lifting operators, for different weak formulations of the Stokes problem, and to three dimensions are discussed. Application of the equivalence result yields an optimal error estimate for the Stokes velocity without involving the pressure. Conversely, combined with a recent multigrid method for Stokes flow, we obtain a simple and uniform preconditioner for harmonic problems with simply supported and clamped boundary.

1-Norm extreme learning machine for regression and multiclass classification using Newton method

In this paper, a novel 1-norm extreme learning machine (ELM) for regression and multiclass classification is proposed as a linear programming problem whose solution is obtained by solving its dual exterior penalty problem as an unconstrained minimization problem using a fast Newton method. The algorithm converges from any starting point and can be easily implemented in MATLAB. The main advantage of the proposed approach is that it leads to a sparse model representation meaning that many components of the optimal solution vector will become zero and therefore the decision function can be determined using much less number of hidden nodes in comparison to ELM. Numerical experiments were performed on a number of interesting real-world benchmark datasets and their results are compared with ELM using additive and radial basis function (RBF) hidden nodes, optimally pruned ELM (OP-ELM) and support vector machine (SVM) methods. Similar or better generalization performance of the proposed method on the test data over ELM, OP-ELM and SVM clearly illustrates its applicability and usefulness.

A New Coordination Model for Ill-Posed Bilevel Programming Problem

This paper studied the leader how to motivate the follower to maximize its interests for ill-posed linear bilevel programming problem. We first presented a coordination model and gived its corresponding penalty problem. Under some conditions, we established the result on the existence of the solution. Then, an algorithm was developed to obtain a coordination solution to the original bilevel programming problem. Finally, numerical results show that the proposed method is feasible.

A New Link-Based Single Tree Building Algorithm for Shortest Path Searching in an Urban Road Transportation Network

The shortest-path searching algorithm must not only find a global solution to the destination, but also solve a turn penalty problem (TPP) in an urban road transportation network (URTN). Although the Dijkstra algorithm (DA) as a representative node-based algorithm secures a global solution to the shortest path search (SPS) in the URTN by visiting all the possible paths to the destination, the DA does not solve the TPP and the slow execution speed problem (SEP) because it must search for the temporary minimum cost node. Potts and Oliver solved the TPP by modifying the visiting unit from a node to the link type of a tree-building algorithm like the DA. The Multi Tree Building Algorithm (MTBA), classified as a representative Link Based Algorithm (LBA), does not extricate the SEP because the MTBA must search many of the origin and destination links as well as the candidate links in order to find the SPS. In this paper, we propose a new Link-Based Single Tree Building Algorithm in order to reduce the SEP of the MTBA by applying the breaking rule to the LBA and also prove its usefulness by comparing the proposed with other algorithms such as the node-based DA and the link-based MTBA for the error rates and execution speeds.

A Unified Approach for Constrained Extremum Problems: Image Space Analysis

In this paper, by exploiting the image space analysis we investigate a class of constrained extremum problems, the constraining function of which is set-valued. We show that a (regular) linear separation in the image space is equivalent to the existence of saddle points of Lagrangian and generalized Lagrangian functions and we also give Lagrangian type optimality conditions for the class of constrained extremum problems under suitable generalized convexity and compactness assumptions. Moreover, we consider an exact penalty problem for the class of constrained extremum problems and prove that it is equivalent to the existence of a regular linear separation under suitable generalized convexity and compactness assumptions.

Highly Efficient SO2Absorption and Its Subsequent Utilization by Weak Base/Polyethylene Glycol Binary System

A binary system consisting of polyethylene glycol (PEG, proton donor)/PEG-functionalized base with suitable basicity was developed for efficient gas desulfurization (GDS) and can be regarded as an alternative approach to circumvent the energy penalty problem in the GDS process. High capacity for SO(2) capture up to 4.88 mol of SO(2)/mol of base was achieved even under low partial pressure of SO(2). Furthermore, SO(2) desorption runs smoothly under mild conditions (N(2), 25 °C) and no significant drop in SO(2) absorption was observed after five-successive absorption-desorption cycles. On the other hand, the absorbed SO(2) by PEG(150)MeIm/PEG(150), being considered as the activated form of SO(2), can be directly transformed into value-added chemicals under mild conditions, thus eliminating the energy penalty for SO(2) desorption and simultaneously realizing recycle of the absorbents. Thus, this SO(2) capture and utilization (SCU) process offers an alternative way for GDS and potentially enables the SO(2) conversion from flue gas to useful chemicals as a value-added process.

Smoothing Technique on Linear Programming Twin Support Vector Machines

In this paper, a new smoothing technique on linear programming TWSVM formulation is proposed whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton method. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm and can be easily implemented in MATLAB without using any optimization toolbox. Computational comparisons of our proposed method against original TWSVM, GEPSVM and SVM indicate that our method is not only fast, but also shows good generalization performance. optimization problem (16) and Smooth TWSVM (9), we propose in this work smooth linear programming twin support vector machines (SLPTSVM) whose solution is obtained, by solving a pair of exterior penalty problems in dual as unconstrained optimization problems using Newton-Armijo algorithm. To demonstrate the effectiveness of the proposed method, we performed numerical experiments on a number of interesting real-world datasets and compared their results with other SVMs. In this work, all vectors are taken as column vectors. The inner product of two vectors y x , in the − n dimensional

Smoothing Approximation to the Square-Order Exact Penalty Functions for Constrained Optimization

A method is proposed to smooth the square-order exact penalty function for inequality constrained optimization. It is shown that, under some conditions, an approximately optimal solution of the original problem can be obtained by searching an approximately optimal solution of the smoothed penalty problem. An algorithm based on the smoothed penalty functions is given. The algorithm is shown to be convergent under mild conditions. Two numerical examples show that the algorithm seems efficient.