Penalty Problem Research Articles

An Algorithm for the Sequence Alignment with Gap Penalty Problem using Multiway Divide-and-Conquer and Matrix Transposition

We present a cache-efficient parallel algorithm for the sequence alignment with gap penalty problem for shared-memory machines using multiway divide-and-conquer and not-in-place matrix transposition. Our r−way divide-and-conquer algorithm, for a fixed natural number r≥2, performs Θ(n3) work, achieves Θ(nlogr⁡(2r−1)) span, and incurs O(n3/(BM)+(n2/B)log⁡M) serial cache misses for n>γM, and incurs O((n2/B)log⁡(n/M)) serial cache misses for αM<n≤γM, where, M is the cache size, B is the cache line size, and α and γ are constants.

Smoothing Approximation to the New Exact Penalty Function with Two Parameters

In this paper, we propose a new non-smooth penalty function with two parameters for nonlinear inequality constrained optimization problems. And we propose a twice continuously differentiable function which is smoothing approximation to the non-smooth penalty function and define the corresponding smoothed penalty problem. A global solution of the smoothed penalty problem is proved to be an approximation global solution of the non-smooth penalty problem. Based on the smoothed penalty function, we develop an algorithm and prove that the sequence generated by the algorithm can converge to the optimal solution of the original problem.

Generalized penalty method for history-dependent variational–hemivariational inequalities

We consider a history-dependent variational–hemivariational inequality with unilateral constraints in a reflexive Banach space. The unique solvability of the inequality follows from an existence and uniqueness result obtained in Sofonea and Migórski (2016, 2018). In this current paper we introduce and study a generalized penalty method associated to the inequality. To this end we consider a sequence of generalized penalty problems, governed by a parameter λn and an operator Pn. We prove the unique solvability of the penalty problems as well as the convergence of corresponding solutions sequence to the solution of original problem. These results extend the previous results in Sofonea et al. (2018) and Xiao and Sofonea (2019). Finally, we illustrate them in the study of a history-dependent problem with unilateral boundary conditions which describes the quasistatic evolution of a rod–spring system under the action of given applied force.

Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems

This paper proposes an inertial Bregman proximal gradient method for minimizing the sum of two possibly nonconvex functions. This method includes two different inertial steps and adopts the Bregman regularization in solving the subproblem. Under some general parameter constraints, we prove the subsequential convergence that each generated sequence converges to the stationary point of the considered problem. To overcome the parameter constraints, we further propose a nonmonotone line search strategy to make the parameter selections more flexible. The subsequential convergence of the proposed method with line search is established. When the line search is monotone, we prove the stronger global convergence and linear convergence rate under Kurdyka–Łojasiewicz framework. Moreover, numerical results on SCAD and MCP nonconvex penalty problems are reported to demonstrate the effectiveness and superiority of the proposed methods and line search strategy.

Positive semidefinite penalty method for quadratically constrained quadratic programming

Abstract Quadratically constrained quadratic programming (QCQP) appears widely in engineering applications such as wireless communications and networking and multiuser detection with examples like the MAXCUT problem and boolean optimization. A general QCQP problem is NP-hard. We propose a penalty formulation for the QCQP problem based on semidefinite relaxation. Under suitable assumptions we show that the optimal solutions of the penalty problem are the same as those of the original QCQP problem if the penalty parameter is sufficiently large. Then, to solve the penalty problem, we present a proximal point algorithm and an update rule for the penalty parameter. Numerically, we test our algorithm on two well-studied QCQP problems. The results show that our proposed algorithm is very effective in finding high-quality solutions.

A multi-stage convex relaxation approach to noisy structured low-rank matrix recovery

This paper concerns with a noisy structured low-rank matrix recovery problem which can be modeled as a structured rank minimization problem. We reformulate this problem as a mathematical program with a generalized complementarity constraint (MPGCC), and show that its penalty version, yielded by moving the generalized complementarity constraint to the objective, has the same global optimal solution set as the MPGCC does whenever the penalty parameter is over a threshold. Then, by solving the exact penalty problem in an alternating way, we obtain a multi-stage convex relaxation approach. We provide theoretical guarantees for our approach under a mild restricted eigenvalue condition, by quantifying the reduction of the error and approximate rank bounds of the first stage convex relaxation (which is exactly the nuclear norm relaxation) in the subsequent stages and establishing the geometric convergence of the error sequence in a statistical sense. Numerical experiments are conducted for some structured low-rank matrix recovery examples to confirm our theoretical findings.

Structured sparse logistic regression with application to lung cancer prediction using breath volatile biomarkers.

This article is motivated by a study of lung cancer prediction using breath volatile organic compound (VOC) biomarkers, where the challenge is that the predictors include not only high-dimensional time-dependent or functional VOC features but also the time-independent clinical variables. We consider a high-dimensional logistic regression and propose two different penalties: group spline-penalty or group smooth-penalty to handle the group structures of the time-dependent variables in the model. The new methods have the advantage for the situation where the model coefficients are sparse but change smoothly within the group, compared with other existing methods such as the group lasso and the group bridge approaches. Our methods are easy to implement since they can be turned into a group minimax concave penalty problem after certain transformations. We show that our fitting algorithm possesses the descent property and leads to attractive convergence properties. The simulation studies and the lung cancer application are performed to demonstrate the accuracy and stability of the proposed approaches.

Convergence properties of a class of exact penalty methods for semi-infinite optimization problems

In this paper, a new class of unified penalty functions are derived for the semi-infinite optimization problems, which include many penalty functions as special cases. They are proved to be exact in the sense that under Mangasarian–Fromovitz constraint qualification conditions, a local solution of penalty problem is a corresponding local solution of original problem when the penalty parameter is sufficiently large. Furthermore, global convergence properties are shown under some conditions. The paper is concluded with some numerical examples proving the applicability of our methods to PID controller design and linear-phase FIR digital filter design.

A 1-norm regularized linear programming nonparallel hyperplane support vector machine for binary classification problems

This research proposes a 1-norm regularized linear programming nonparallel hyperplane support vector machine (LNSVM) model to solve binary classification problems and enhance the robustness performance. Numerous nonparallel support vector machine (SVM) models have been studied with outstanding performance on classification tasks. However, most nonparallel SVM models require two independent models to determine hyperplanes. In addition, due to the involvement of the 2-norm terms, traditional SVM models may suffer from the lack of robustness to outliers and irrelevant features. Therefore, the LNSVM model is proposed by reformulating a typical nonparallel SVM model through the 1-norm regularization. By applying the exterior penalty theory, the proposed LNSVM model is converted to the dual exterior penalty problem, which is solved by the Newton–Armijo algorithm. The essential differences that distinguish the LNSVM model from other nonparallel SVM models are: (1) Different from typical nonparallel SVM models, which solve two quadratic programming (QP) problems, the proposed LNSVM model determines two nonparallel hyperplanes simultaneously by solving a single linear programming (LP) model; (2) The robustness performance of the proposed LNSVM model has been enhanced to tolerate noisy data through the involvement of 1-norm loss function, which can also eliminate redundant features by generating sparse solution during the training procedure. The performance of the proposed LNSVM model is tested through a comparison with state-of-art SVM-based classifiers using a synthetic dataset and 11 practical benchmark datasets. The experimental results show the superiority of the proposed LNSVM model, by achieving better classification performance regarding accuracy, sensitivity, specificity, and removing redundant features synchronously.

Modified fuzzy clustering with segregated cluster centroids

In this paper, we firstly propose a novel strategy to fuse k-means and fuzzy c-means objective functions from a multi-objective problem into a unified model, such that extreme difficulty of choosing the optimal level of the cluster fuzziness can be avoided (level of the cluster fuzziness vanishes). Accordingly, fused k-means and fuzzy c-means clustering (FKMFCM) problem is achieved with a brand new form. Based on the proposed FKMFCM, novel degrees of membership could be further derived with the closed forms. Additionally, a penalty problem is further introduced to the proposed FKMFCM, such that the fuzzy cluster centroids are modified to be more segregated from each other. Consequently, FKMFCM with the modified cluster centroid (FKMFCM-MCC) problem is represented as a bi-objective optimization for the efficient clustering. To address the proposed FKMFCM-MCC problem, an original characteristic function is introduced, such that the corresponding algorithm converges to the global optimum of the proposed FKMFCM-MCC problem. Eventually, theoretical analysis along with the empirical result is provided to validate the effectiveness of the proposed FKMFCM-MCC approach.

Convergence study and optimal weight functions of an explicit particle method for the incompressible Navier–Stokes equations

To increase the reliability of simulations by particle methods for incompressible viscous flow problems, convergence studies and improvements of accuracy are considered for a fully explicit particle method for incompressible Navier--Stokes equations. The explicit particle method is based on a penalty problem, which converges theoretically to the incompressible Navier--Stokes equations, and is discretized in space by generalized approximate operators defined as a wider class of approximate operators than those of the smoothed particle hydrodynamics (SPH) and moving particle semi-implicit (MPS) methods. By considering an analytical derivation of the explicit particle method and truncation error estimates of the generalized approximate operators, sufficient conditions of convergence are conjectured.Under these conditions, the convergence of the explicit particle method is confirmed by numerically comparing errors between exact and approximate solutions. Moreover, by focusing on the truncation errors of the generalized approximate operators, an optimal weight function is derived by reducing the truncation errors over general particle distributions. The effectiveness of the generalized approximate operators with the optimal weight functions is confirmed using numerical results of truncation errors and driven cavity flow. As an application for flow problems with free surface effects, the explicit particle method is applied to a dam break flow.

General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems

In this paper, we consider a general inertial proximal gradient method with constant and variable stepsizes for a class of nonconvex nonsmooth optimization problems. The proposed method incorporates two different extrapolations with respect to the previous iterates into the backward proximal step and the forward gradient step in classic proximal gradient method. Under more general parameter constraints, we prove that the proposed method generates a convergent subsequence and each limit point is a stationary point of the problem. Furthermore, the generated sequence is globally convergent to a stationary point if the objective function satisfies the Kurdyka–Łojasiewicz property. Local linear convergence also can be established for the proposed method with constant stepsizes by using a common error bound condition. In addition, we conduct some numerical experiments on nonconvex quadratic programming and SCAD penalty problems to demonstrate the advantage of the proposed method.

Variable Selection with Big Data based on Zero Norm and via Sequential Monte Carlo

Selecting a subset from many potential explanatory variables in linear regressions has long been the subject of research interest, and the matter is made more important in the era of big data when many more variables become available/accessible. Of late, the l_1-norm penalty based techniques such as Lasso of Tibshirani (1996) have become very popular. However, the variable selection problem in its natural setting is a zero-norm penalty problem, i.e., a penalty on the number of variables as opposed to the l_1-norm of the regression coefficients. The popularity of the l_1-norm penalty or its variants has more to do with computational considerations, because selection with the zero-norm penalty is a highly demanding combinatory optimization problem when the number of potential variables becomes large. We devise a sequential Monte Carlo (SMC) method as a practical and reliable tool for zero-norm variable selection problems, and selecting, say, best 20 out of 1,000 potential variables can, for example, be completed with a typical multi-core desktop computer in a couple of minutes. The methodological essence is to understand that the selection problem is equivalent to the task of sampling from a discrete probability function de fined over all possible combinations comprising, say, k regressors out of p>=k potential variables, where the peak of this function corresponds to the optimal combination. The solution technique sets out to sequentially generate samples, and after a while the final sample represents the target probability function. With the fi nal SMC sample in place, we deploy the extreme value theory to assess how likely and to what extent the maximum R2 has been achieved. We also demonstrate through a simulation study the method's reliability and superiority vis-a-vis the adaptive Lasso.

On Smoothing l1 Exact Penalty Function for Constrained Optimization Problems

This article introduces a smoothing technique to the l1 exact penalty function. An application of the technique yields a twice continuously differentiable penalty function and a smoothed penalty problem. Under some mild conditions, the optimal solution to the smoothed penalty problem becomes an approximate optimal solution to the original constrained optimization problem. Based on the smoothed penalty problem, we propose an algorithm to solve the constrained optimization problem. Every limit point of the sequence generated by the algorithm is an optimal solution. Several numerical examples are presented to illustrate the performance of the proposed algorithm.

An exact penalty method for semidefinite-box-constrained low-rank matrix optimization problems

AbstractThis paper considers a matrix optimization problem where the objective function is continuously differentiable and the constraints involve a semidefinite-box constraint and a rank constraint. We first replace the rank constraint by adding a non-Lipschitz penalty function in the objective and prove that this penalty problem is exact with respect to the original problem. Next, for the penalty problem we present a nonmonotone proximal gradient (NPG) algorithm whose subproblem can be solved by Newton’s method with globally quadratic convergence. We also prove the convergence of the NPG algorithm to a first-order stationary point of the penalty problem. Furthermore, based on the NPG algorithm, we propose an adaptive penalty method (APM) for solving the original problem. Finally, the efficiency of an APM is shown via numerical experiments for the sensor network localization problem and the nearest low-rank correlation matrix problem.

Penalty method for source reconstruction of multispectral bioluminescence tomography

Bioluminescence tomography (BLT) is a promising optical molecular imaging technique in preclinical research. One key problem for BLT is how to deal with the severe ill-posedness and obtain accurate and stable reconstruction. We propose a penalty method for recovering bioluminescence sources, in which we transform reconstruction of BLT into an L1/2-norm penalty problem and solve it via a nonmonotone proximal gradient method with a suitable penalty parameter update scheme. Simulations and phantom experiments based on multispectral measurements were designed to evaluate the proposed reconstruction method. The encouraging results show that the proposed method has better reconstruction accuracy and image quality than the comparative methods.

Solving Transfer Pricing Involving Collaborative and Non-cooperative Equilibria in Nash and Stackelberg Games: Centralized–Decentralized Decision Making

The classical transfer pricing model considers a cooperative approach taking into account that each division purchases goods from an upstream division in the supply chain. However, divisions of a firm can be decentralized and not necessarily act cooperatively. A conflict in decentralization arises. A lack of coordination between the divisions and central management occurs if the divisions have full autonomy: managers might engage in actions that would benefit their divisions to the detriment of maximizing the well-being of the entire firm. This paper presents a model for transfer pricing that is able to maintain divisional decentralization by computing the Nash equilibrium and at the same time encourage divisions to achieve central management optimal results by computing the strong Nash equilibrium. The resulting strong Nash/Nash equilibrium point involves the computation of a cooperative and non-cooperative behavior. The model is able to provide a measure of the divisional profit maximization of the firm. In addition, we consider a bi-level structure of the company’s supply chain and solve the problem employing a Stackelberg equilibrium approach. For solving the problem we represent the transfer pricing game in terms of a coupled nonlinear programming equations implementing the penalty approach. A regularization method is employed to ensure the convergence of the utility-functions to a unique equilibrium point. For computing the equilibrium point we employ a minimization of the Euclidean distance. Next, we transform the penalty problem into a new system of equations in the Euclidean distance format. For minimizing the Euclidean distance we use a gradient method approach. A numerical example validates the effectiveness and usefulness of the proposed model for transfer pricing.

Denoising of Magnetocardiography Based on Improved Variational Mode Decomposition and Interval Thresholding Method

Recently, magnetocardiography (MCG) has attracted increasing attention as a non-invasive and non-contact technique for detecting electrocardioelectric functions. However, the severe background noise makes it difficult to extract information. Variational Mode Decomposition (VMD), which is an entirely non-recursive model, is used to decompose the non-stationary signal into the intrinsic mode functions (IMFs). Traditional VMD algorithms cannot control the bandwidth of each IMF, whose quadratic penalty lacks adaptivity. As a result, baseline drift noise is still present or medical information is lost. In this paper, to overcome the unadaptable quadratic penalty problem, an improved VMD model via correlation coefficient and new update formulas are proposed to decompose MCG signals. To improve the denoising precision, this algorithm is combined with the interval threshold algorithm. First, the correlation coefficient is calculated, to determine quadratic penalty, in order to extract the first IMF made up of baseline drift. Then, the new update formulas derived from the variance that describes the noise level are used, to perform decomposition on the rest signal. Finally, the Interval thresholding algorithm is performed on each IMF. Theoretical analysis and experimental results show that this algorithm can effectively improve the output signal-to-noise ratio and has superior performance.

Overlapping community detection with preference and locality information: a non-negative matrix factorization approach

Community detection plays an important role in understanding structures and patterns in complex networks. In real-world networks, a node in most cases belongs to multiple communities, which makes communities overlap with each other. One popular technique to cope with overlapping community detection is matrix factorization (MF). However, existing MF approaches only make use of the existence of a link, but ignore the implicit preference information inside it. In this paper, we first propose a Preference-based Non-negative Matrix Factorization (PNMF) model to take link preference information into consideration. Distinguished from traditional value approximation-based matrix factorization approaches, our model maximizes the likelihood of the preference order for each node so that it overcomes the indiscriminate penalty problem in which non-linked pairs inside one community are equally penalized in objective functions as those across two communities. Moreover, we propose a Locality-based Non-negative Matrix Factorization (LNMF) model to further incorporate the concept of locality and generalize the preference system of PNMF. Particularly, we define a subgraph called “K-degree local network” to set a boundary between local non-neighbors and other non-neighbors, and explicitly treat these two classes of non-neighbors in objective function. Through experiments on various benchmark networks, we show that our PNMF model outperforms state-of-the-art baselines, and the generalized LNMF model further performs better than the PNMF model on datasets with high locality.

Smoothing approximation to the lower order exact penalty function for inequality constrained optimization

For inequality constrained optimization problem, we first propose a new smoothing method to the lower order exact penalty function, and then show that an approximate global solution of the original problem can be obtained by solving a global solution of a smooth lower order exact penalty problem. We propose an algorithm based on the smoothed lower order exact penalty function. The global convergence of the algorithm is proved under some mild conditions. Some numerical experiments show the efficiency of the proposed method.