Kuhn-Tucker Point Research Articles

The nearest graph Laplacian in Frobenius norm

AbstractWe address the problem of finding the nearest graph Laplacian to a given matrix, with the distance measured using the Frobenius norm. Specifically, for the directed graph Laplacian, we propose two novel algorithms by reformulating the problem as convex quadratic optimization problems with a special structure: one based on the active set method and the other on direct computation of Karush–Kuhn–Tucker points. The proposed algorithms can be applied to system identification and model reduction problems involving Laplacian dynamics. We demonstrate that these algorithms possess lower time complexities and the finite termination property, unlike the interior point method and V‐FISTA, the latter of which is an accelerated projected gradient method. Our numerical experiments confirm the effectiveness of the proposed algorithms.

Synchronous deautoconvolution algorithm for discrete-time positive signals via [formula omitted]-divergence approximation

We pose the problem of the optimal approximation of a given nonnegative signal yt with the scalar autoconvolution (x∗x)t of a nonnegative signal xt, where xt and yt are signals of equal length. The I-divergence has been adopted as optimality criterion, being well suited to incorporate nonnegativity constraints. To find a minimizer we derive an iterative descent algorithm of the alternating minimization type. The algorithm is based on the lifting of the original problem to a larger space, a relaxation technique developed by Csiszár and Tusnády in [Statistics & Decisions (S1) (1984), 205–237] which, in the present context, requires the solution of a hard partial minimization problem. We study the asymptotic behavior of the algorithm exploiting the optimality properties of the partial minimization problems and prove, among other results, that its limit points are Kuhn–Tucker points of the original minimization problem. Numerical experiments illustrate the results.

Sub-One Quasi-Norm-Based k-Means Clustering Algorithm and Analyses

Recognizing the pivotal role of choosing an appropriate distance metric in designing the clustering algorithm, our focus is on innovating the k-means method by redefining the distance metric in its distortion. In this study, we introduce a novel k-means clustering algorithm utilizing a distance metric derived from the ℓp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _p$$\\end{document} quasi-norm with p∈(0,1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p\\in (0,1)$$\\end{document}. Through an illustrative example, we showcase the advantageous properties of the proposed distance metric compared to commonly used alternatives for revealing natural groupings in data. Subsequently, we present a novel k-means type heuristic by integrating this sub-one quasi-norm-based distance, offer a step-by-step iterative relocation scheme, and prove the convergence to the Kuhn-Tucker point. Finally, we empirically validate the effectiveness of our clustering method through experiments on synthetic and real-life datasets, both in their original form and with additional noise introduced. We also investigate the performance of the proposed method as a subroutine in a deep learning clustering algorithm. Our results demonstrate the efficacy of the proposed k-means algorithm in capturing distinctive patterns exhibited by certain data types.

Extended convergence analysis of the Scholtes-type regularization for cardinality-constrained optimization problems

AbstractWe extend the convergence analysis of the Scholtes-type regularization method for cardinality-constrained optimization problems. Its behavior is clarified in the vicinity of saddle points, and not just of minimizers as it has been done in the literature before. This becomes possible by using as an intermediate step the recently introduced regularized continuous reformulation of a cardinality-constrained optimization problem. We show that the Scholtes-type regularization method is well-defined locally around a nondegenerate T-stationary point of this regularized continuous reformulation. Moreover, the nondegenerate Karush–Kuhn–Tucker points of the corresponding Scholtes-type regularization converge to a T-stationary point having the same index, i.e. its topological type persists. As consequence, we conclude that the global structure of the Scholtes-type regularization essentially coincides with that of CCOP.

On the weakest constraint qualification for sharp local minimizers

The sharp local minimality of feasible points of nonlinear optimization problems is known to possess a characterization by a strengthened version of the Karush–Kuhn–Tucker conditions, as long as the Mangasarian–Fromovitz constraint qualification holds. This strengthened condition is not easy to check algorithmically since it involves the topological interior of some set. In this article, we derive an algorithmically tractable version of this condition, called strong Karush–Kuhn–Tucker condition. We show that the Guignard constraint qualification is the weakest condition under which a feasible point is a strong Karush–Kuhn–Tucker point for every continuously differentiable objective function possessing the point as a sharp local minimizer. As an application, our results yield an algebraic characterization of strict local minimizers of linear programs with cardinality constraints.

Robust and Sparse Portfolio: Optimization Models and Algorithms

The robust and sparse portfolio selection problem is one of the most-popular and -frequently studied problems in the optimization and financial literature. By considering the uncertainty of the parameters, the goal is to construct a sparse portfolio with low volatility and decent returns, subject to other investment constraints. In this paper, we propose a new portfolio selection model, which considers the perturbation in the asset return matrix and the parameter uncertainty in the expected asset return. We define three types of stationary points of the penalty problem: the Karush–Kuhn–Tucker point, the strong Karush–Kuhn–Tucker point, and the partial minimizer. We analyze the relationship between these stationary points and the local/global minimizer of the penalty model under mild conditions. We design a penalty alternating-direction method to obtain the solutions. Compared with several existing portfolio models on seven real-world datasets, extensive numerical experiments demonstrate the robustness and effectiveness of our model in generating lower volatility.

An event-triggered collaborative neurodynamic approach to distributed global optimization

In this paper, we propose an event-triggered collaborative neurodynamic approach to distributed global optimization in the presence of nonconvexity. We design a projection neural network group consisting of multiple projection neural networks coupled via a communication network. We prove the convergence of the projection neural network group to Karush–Kuhn–Tucker points of a given global optimization problem. To reduce communication bandwidth consumption, we adopt an event-triggered mechanism to liaise with other neural networks in the group with the Zeno behavior being precluded. We employ multiple projection neural network groups for scattered searches and re-initialize their states using a meta-heuristic rule in the collaborative neurodynamic optimization framework. In addition, we apply the collaborative neurodynamic approach for distributed optimal chiller loading in a heating, ventilation, and air conditioning system.

Generalized Polarity and Weakest Constraint Qualifications in Multiobjective Optimization

In Haeser and Ramos (J Optim Theory Appl, 187:469–487, 2020), a generalization of the normal cone from single objective to multiobjective optimization is introduced, along with a weakest constraint qualification such that any local weak Pareto optimal point is a weak Kuhn–Tucker point. We extend this approach to other generalizations of the normal cone and corresponding weakest constraint qualifications, such that local Pareto optimal points are weak Kuhn–Tucker points, local proper Pareto optimal points are weak and proper Kuhn–Tucker points, respectively, and strict local Pareto optimal points of order one are weak, proper and strong Kuhn–Tucker points, respectively. The constructions are based on an appropriate generalization of polarity to pairs of matrices and vectors.

The Inverse Problem of Positive Autoconvolution

We pose the problem of approximating optimally a given nonnegative signal with the scalar autoconvolution of a nonnegative signal. The I-divergence is chosen as the optimality criterion being well suited to incorporate nonnegativity constraints. After proving the existence of an optimal approximation, we derive an iterative descent algorithm of the alternating minimization type to find a minimizer. The algorithm is based on the lifting technique developed by Csiszár and Tusnádi and exploits the optimality properties of the related minimization problems in the larger space. We study the asymptotic behavior of the iterative algorithm and prove, among other results, that its limit points are Kuhn-Tucker points of the original minimization problem. Numerical experiments confirm the asymptotic results and exhibit the fast convergence of the proposed algorithm.

Improved TV Image Denoising over Inverse Gradient

Noise in an image can affect one’s extraction of image information, therefore, image denoising is an important image pre-processing process. Many of the existing models have a large number of estimated parameters, which increases the time complexity of the model solution and the achieved denoising effect is less than ideal. As a result, in this paper, an improved image-denoising algorithm is proposed based on the TV model, which effectively solves the above problems. The L1 regularization term can make the solution generated by the model sparser, thus facilitating the recovery of high-quality images. Reducing the number of estimated parameters, while using the inverse gradient to estimate the regularization parameters, enables the parameters to achieve global adaption and improves the denoising effect of the model in combination with the TV regularization term. The split Bregman iteration method is used to decouple the model into several related subproblems, and the solutions of the coordinated subproblems are derived as optimal solutions. It is also shown that the solution of the model converges to a Karush–Kuhn–Tucker point. Experimental results show that the algorithm in this paper is more effective in both preserving image texture structure and suppressing image noise.

Properties of the Solution Set of Absolute Value Equations and the Related Matrix Classes

The absolute value equations (AVE) problem is an algebraic problem of solving . So far, most of the research has focused on methods for solving AVE, but we address the problem itself by analyzing properties of AVE and the corresponding solution set. In particular, we investigate topological properties of the solution set, such as convexity, boundedness, or connectedness, or whether it consists of finitely many solutions. Further, we address problems related to the nonnegativity of solutions such as solvability or unique solvability. AVE can be formulated by means of different optimization problems, and in this regard we are interested in how the solutions of AVE are related with optima, Karush–Kuhn–Tucker points, and feasible solutions of these optimization problems. We characterize the matrix classes associated with the above mentioned properties and inspect the computational complexity of the recognition problem; some of the classes are polynomially recognizable, but some others are proved to be NP-hard. For the intractable cases, we propose various sufficient conditions. We also post new challenging problems that were raised during the investigation of the problem.

The Boosted DC Algorithm for Linearly Constrained DC Programming

The Boosted Difference of Convex functions Algorithm (BDCA) has been recently introduced to accelerate the performance of the classical Difference of Convex functions Algorithm (DCA). This acceleration is achieved thanks to an extrapolation step from the point computed by DCA via a line search procedure. In this work, we propose an extension of BDCA that can be applied to difference of convex functions programs with linear constraints, and prove that every cluster point of the sequence generated by this algorithm is a Karush–Kuhn–Tucker point of the problem if the feasible set has a Slater point. When the objective function is quadratic, we prove that any sequence generated by the algorithm is bounded and R-linearly (geometrically) convergent. Finally, we present some numerical experiments where we compare the performance of DCA and BDCA on some challenging problems: to test the copositivity of a given matrix, to solve one-norm and infinity-norm trust-region subproblems, and to solve piecewise quadratic problems with box constraints. Our numerical results demonstrate that this new extension of BDCA outperforms DCA.

Feasible Newton methods for symmetric tensor Z-eigenvalue problems

Finding a Z-eigenpair of a symmetric tensor is equivalent to finding a Karush–Kuhn–Tucker point of a sphere constrained minimization problem. Based on this equivalency, in this paper, we first propose a class of iterative methods to get a Z-eigenpair of a symmetric tensor. Each method can generate a sequence of feasible points such that the sequence of function evaluations is decreasing. These methods can be regarded as extensions of the descent methods for unconstrained optimization problems. We pay particular attention to the Newton method. We show that under appropriate conditions, the Newton method is globally and quadratically convergent. Moreover, after finitely many iterations, the unit steplength will always be accepted. We also propose a nonlinear equations-based Newton method and establish its global and quadratic convergence. In the end, we do several numerical experiments to test the proposed Newton methods. The results show that both Newton methods are very efficient.

On a solution method in indefinite quadratic programming under linear constraints

We establish some properties of the Proximal Difference-of-Convex functions decomposition algorithm in indefinite quadratic programming under linear constraints. The first property states that any iterative sequence generated by the algorithm is root linearly convergent to a Karush–Kuhn–Tucker point, provided that the problem has a solution. The second property says that iterative sequences generated by the algorithm converge to a locally unique solution of the problem if the initial points are taken from a suitably chosen neighbourhood of it. Through a series of numerical tests, we analyse the influence of the decomposition parameter on the rate of convergence of the iterative sequences and compare the performance of the Proximal Difference-of-Convex functions decomposition algorithm with that of the Projection Difference-of-Convex functions decomposition algorithm. In addition, the performances of the above algorithms and the Gurobi software in solving some randomly generated nonconvex quadratic programs are compared.

LU-Optimality Conditions in Optimization Problems With Mechanical Work Objective Functionals.

In this article, we introduce interval-valued Kuhn-Tucker (KT)-pseudoinvex optimization problems governed by interval-valued path-independent curvilinear integral objective functionals. Concretely, it is proven that an interval-valued KT-pseudoinvex variational control problem is described such that every KT point is an LU-optimal solution. In addition, the main results are highlighted by two illustrative applications describing the controlled behavior of an artificial neural system.

Linear programming with a feasible direction interior point technique for smooth optimization

We propose an adaptation of the Feasible Direction Interior Points Algorithm (FDIPA) of J. Herskovits, for solving large-scale linear programs. At each step, the solution of two linear systems with the same coefficient matrix is determined. This step involves a significant computational effort. Reducing the solution time of linear systems is, therefore, a way to improve the performance of the method. The linear systems to be solved are associated with definite positive symmetric matrices. Therefore, we use Split Preconditioned Conjugate Gradient (SPCG) method to solve them, together with an Incomplete Cholesky preconditioner using Matlab’s ICHOL function. We also propose to use the first iteration of the conjugate gradient, and to presolve before applying the algorithm, in order to reduce the computational cost. Following, we then provide mathematica proof that show that the iterations approach Karush–Kuhn–Tucker points of the problem under reasonable assumptions. Finally, numerical evidence show that the method not only works in theory but is also competitive with more advanced methods.

Multispectrally Constrained MIMO Radar Beampattern Design via Sequential Convex Approximation

This article deals with the multiple-input–multiple-output (MIMO) radar beampattern design in an effort to the coexistence with multiple communication systems. A waveform optimization model accounting for the minimization of the beampattern integrated sidelobe level (ISL) along with the mainlobe width, peak-to-average power ratio, and energy constraints, as well as multispectral requirements where the interference energy injected by the MIMO radar in each shared frequency band in a particular direction, is precisely controlled to ensure the desired quality of service at each communication system. Through an equivalent reformulation of the original nonconvex problem, a polynomial-time sequential convex approximation (SCA) procedure that involves the tackling of a series of constrained convex problems is proposed to monotonically decrease the ISL with the convergence guaranteed to a Karush–Kuhn–Tucker point. Herein, to speed up the convergence, a fast iterative algorithm based on the alternating-direction-method-of-multipliers framework is introduced to globally solve the convex problems during each iteration of the SCA procedure. Numerical results are provided to assess the proposed algorithm in terms of the computational complexity, the achieved beampattern, and spectral compatibility with some competitive counterparts available in the open literatures.

Robust optimality in constrained optimization problems with application in mechanics

This paper deals with necessary and sufficient conditions of optimality for a new robust optimization problem involving curvilinear integral (mechanical work) functionals, which are independent of the path and are determined by uncertain multi-dimensional controlled Lagrangians of second order. More precisely, by using the notion of convex curvilinear integral, provided by uncertain complete integrable controlled Lagrange 1-form, and the concept of robust weak optimal solution to the problem under study, we build a new math context to state the optimality conditions which are necessary and sufficient for a point to be an extreme of the new studied class of robust optimization problems with uncertainty in the constraint and objective functionals. Also, we introduce the concept of a robust Kuhn-Tucker point and establish a characterization result. Moreover, we present a non-trivial illustrative example to validate the results established in the paper.

Distributed Nonconvex Optimization for Energy Efficiency in Mobile Ad Hoc Networks

This article proposes a distributed nonconvex optimization algorithm for energy efficiency (EE) in mobile ad hoc networks (MANETs). The proposed model for EE in MANETs can be classified into the category of nonconvex optimization problems. There are various distributed algorithms for solving nonconvex optimization problems. But, they are associated with some flaws such as complexity of implementation and low convergence rate. In this regard, in addition to the simplicity in implementation, the optimal point of the proposed model is a saddle point. Using augmented Lagrangian function with continuous-time optimization dynamics algorithm, a dynamic system will be established where its equilibrium points are Karush–Kuhn–Tucker points of the proposed model. Also, it will be shown that these points can be locally asymptotically stable under some conditions. Additionally, the proposed analytic method proves that the rate of convergence increases with an increasing penalty coefficient. To validate the performance of the presented algorithm, the problem of EE optimization for a sample mobile ad hoc network is simulated in MATLAB environment.

Existence of solution of constrained interval optimization problems with regularity concept

Objective of this article is to study the conditions for the existence of efficient solution of interval optimization problem with inequality constraints. Here the active constraints are considered in inclusion form. The regularity condition for the existence of the Karush–Kuhn–Tucker point is derived. This condition depends on the interval-valued gradient function of active constraints. These are new concepts in the literature of interval optimization. gH-differentiability is used for the theoretical developments. gH-pseudo convexity for interval valued constrained optimization problems is introduced to study the sufficient conditions. Theoretical developments are verified through numerical examples.