Cardinality-constrained Optimization Problem Research Articles

Sparse solution of least-squares twin multi-class support vector machine using [formula omitted] and [formula omitted]-norm for classification and feature selection

In the realm of multi-class classification, the twin K-class support vector classification (Twin-KSVC) generates ternary outputs {−1,0,+1} by evaluating all training data in a “1-versus-1-versus-rest” structure. Recently, inspired by the least-squares version of Twin-KSVC and Twin-KSVC, a new multi-class classifier called improvements on least-squares twin multi-class classification support vector machine (ILSTKSVC) has been proposed. In this method, the concept of structural risk minimization is achieved by incorporating a regularization term in addition to the minimization of empirical risk. Twin-KSVC and its improvements have an influence on classification accuracy. Another aspect influencing classification accuracy is feature selection, which is a critical stage in machine learning, especially when working with high-dimensional datasets. However, most prior studies have not addressed this crucial aspect. In this study, motivated by ILSTKSVC and the cardinality-constrained optimization problem, we propose ℓp-norm least-squares twin multi-class support vector machine (PLSTKSVC) with 0<p<1 to perform classification and feature selection at the same time. The technique employed to solve the optimization problems associated with PLSTKSVC is user-friendly, as it involves solving systems of linear equations to obtain an approximate solution for the proposed model. Under certain assumptions, we investigate the properties of the optimum solutions to the related optimization problems. Several real-world datasets were tested using the suggested method. According to the results of our experiments, the proposed method outperforms all current strategies in most datasets in terms of classification accuracy while also reducing the number of features.

Optimality conditions and constraint qualifications for cardinality constrained optimization problems

The cardinality constrained optimization problem (CCOP) is an optimization problem where the maximum number of nonzero components of any feasible point is bounded. In this paper, by rewriting the cardinality constraint as a constraint requiring that any feasible point must lie in the union of certain subspaces, we consider CCOP as a mathematical program with disjunctive subspaces constraints (MPDSC). Since a subspace is a special case of a convex polyhedral set, MPDSC is a special case of the mathematical program with disjunctive constraints (MPDC). Using the special structure of subspaces, we are able to obtain more precise formulas for the tangent and (directional) normal cones for the disjunctive set of subspaces. We then obtain first and second order optimality conditions by using the corresponding results from MPDC. Thanks to the special structure of the subspace, we are able to obtain some results for MPDSC that do not hold in general for MPDC. In particular we show that the relaxed constant positive linear dependence (RCPLD) is a sufficient condition for the metric subregularity/error bound property for MPDSC which is not true for MPDC in general. Finally we show that under all constraint qualifications presented in this paper, certain exact penalization holds for CCOP.

An augmented Lagrangian method for optimization problems with structured geometric constraints

This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints are nonconvex and possibly complicated, but allow for a fast computation of projections onto this nonconvex set. Typical problem classes which satisfy this requirement are optimization problems with disjunctive constraints (like complementarity or cardinality constraints) as well as optimization problems over sets of matrices which have to satisfy additional rank constraints. The key idea behind our method is to keep these complicated constraints explicitly in the constraints and to penalize only the remaining constraints by an augmented Lagrangian function. The resulting subproblems are then solved with the aid of a problem-tailored nonmonotone projected gradient method. The corresponding convergence theory allows for an inexact solution of these subproblems. Nevertheless, the overall algorithm computes so-called Mordukhovich-stationary points of the original problem under a mild asymptotic regularity condition, which is generally weaker than most of the respective available problem-tailored constraint qualifications. Extensive numerical experiments addressing complementarity- and cardinality-constrained optimization problems as well as a semidefinite reformulation of MAXCUT problems visualize the power of our approach.

An Alternating Method for Cardinality-Constrained Optimization: A Computational Study for the Best Subset Selection and Sparse Portfolio Problems

Cardinality-constrained optimization problems are notoriously hard to solve in both theory and practice. However, as famous examples, such as the sparse portfolio optimization and best subset selection problems, show, this class is extremely important in real-world applications. In this paper, we apply a penalty alternating direction method to these problems. The key idea is to split the problem along its discrete-continuous structure to obtain two subproblems that are much easier to solve than the original problem. In addition, the coupling between these subproblems is achieved via a classic penalty framework. The method can be seen as a primal heuristic for which convergence results are readily available from the literature. In our extensive computational study, we first show that the method is competitive to a commercial mixed-integer program solver for the portfolio optimization problem. On these instances, we also test a variant of our approach that uses a perspective reformulation of the problem. Regarding the best subset selection problem, it turns out that our method significantly outperforms commercial solvers and it is at least competitive to state-of-the-art methods from the literature. History: Accepted by Andrea Lodi, Area Editor for Design &amp; Analysis of Algorithms–Discrete. Funding: The first author thanks the Deutsche Forschungsgemeinschaft (DFG) for its support within “Algorithmic Optimization” [Grant RTG 2126]. The third author thanks the DFG for its support within project A05 and B08 in the “Sonderforschungsbereich/Transregio 154 Mathematical Modeling, Simulation and Optimization using the Example of Gas Networks.” Supplemental Material: The supplementary material is available at https://doi.org/10.1287/ijoc.2022.1211 .

High-dimensional index tracking based on the adaptive elastic net

When a portfolio consists of a large number of assets, it generally incorporates too many small and illiquid positions and needs a large amount of rebalancing, which can involve large transaction costs. For financial index tracking, it is desirable to avoid such atomized, unstable portfolios, which are difficult to realize and manage. A natural way of achieving this goal is to build a tracking portfolio that is sparse with only a small number of assets in practice. The cardinality constraint approach, by directly restricting the number of assets held in the tracking portfolio, is a natural idea. However, it requires the pre-specification of the maximum number of assets selected, which is rarely practicable. Moreover, the cardinality constrained optimization problem is shown to be NP-hard. Solving such a problem will be computationally expensive, especially in high-dimensional settings. Motivated by this, this paper employs a regularization approach based on the adaptive elastic-net (Aenet) model for high-dimensional index tracking. The proposed method represents a family of convex regularization methods, which nests the traditional Lasso, adaptive Lasso (Alasso), and elastic-net (Enet) as special cases. To make the formulation more practical and general, we also take the full investment condition and turnover restrictions (or transaction costs) into account. An efficient algorithm based on coordinate descent with closed-form updates is derived to tackle the resulting optimization problem. Empirical results show that the proposed method is computationally efficient and has competitive out-of-sample performance, especially in high-dimensional settings.

Solving multi-scenario cardinality constrained optimization problems via multi-objective evolutionary algorithms

Cardinality constrained optimization problems (CCOPs) are fixed-size subset selection problems with applications in several fields. CCOPs comprising multiple scenarios, such as cardinality values that form an interval, can be defined as multi-scenario CCOPs (MSCCOPs). An MSCCOP is expected to optimize the objective value of each cardinality to support decision-making processes. When the computation is conducted using traditional optimization algorithms, an MSCCOP often requires several passes (algorithmic runs) to obtain all the (near-)optima, where each pass handles a specific cardinality. Such separate passes abandon most of the knowledge (including the potential superior solution structures) learned in one pass that can also be used to improve the results of other passes. By considering this situation, we propose a generic transformation strategy that can be referred to as the Mucard strategy, which converts an MSCCOP into a low-dimensional multi-objective optimization problem (MOP) to simultaneously obtain all the (near-)optima of the MSCCOP in a single algorithmic run. In essence, the Mucard strategy combines separate passes that deal with distinct variable spaces into a single pass, enabling knowledge reuse and knowledge interchange of each cardinality among genetic individuals. The performance of the Mucard strategy was demonstrated using two typical MSCCOPs. For a given number of evolved individuals, the Mucard strategy improved the accuracy of the obtained solutions because of the in-process knowledge than that obtained by untransformed evolutionary algorithms, while reducing the average runtime. Furthermore, the equivalence between the optimal solutions of the transformed MOP and the untransformed MSCCOP can be theoretically proved.

Weighted Network Design With Cardinality Constraints via Alternating Direction Method of Multipliers

This paper examines simultaneous design of the network topology and the corresponding edge weights in the presence of a cardinality constraint on the edge set. Network properties of interest for this design problem lead to optimization formulations with convex objectives, convex constraint sets, and cardinality constraints. This class of problems is referred to as the cardinality-constrained optimization problem (CCOP); the cardinality constraint generally makes CCOPs NP-hard. In this paper, a customized alternating direction method of multipliers (ADMM) algorithm aiming to improve the scalability of the solution strategy for large-scale CCOPs is proposed. This algorithm utilizes the special structure of the problem formulation to obtain closed-form solutions during each iterative step of the corresponding ADMM updates. We also provide a convergence proof of the proposed customized ADMM to a stationary point under certain conditions. Simulation results illustrate that the customized ADMM algorithm has a significant computational advantage over existing methods, particularly for large-scale network design problems.

Efficient projected gradient methods for cardinality constrained optimization

Sparse optimization has attracted increasing attention in numerous areas such as compressed sens-ing, financial optimization and image processing. In this paper, we first consider a special class of cardinality constrained optimization problems, which involves box constraints and a singly linear constraint. An effcient approach is provided for calculating the projection over the feasibility set after a careful analysis on the projec- tion subproblem. Then we present several types of projected gradient methods for a general class of cardinality constrained optimization problems. Global convergence of the methods is established under suitable assump- tions. Finally, we illustrate some applications of the proposed methods for signal recovery and index tracking. Especially for index tracking, we propose a new model subject to an adaptive upper bound on the sparse portfo-lio weights. The computational results demonstrate that the proposed projected gradient methods are effcient in terms of solution quality.

Joint User Grouping and Linear Virtual Beamforming: Complexity, Algorithms and Approximation Bounds

In a wireless system with a large number of distributed nodes, the quality of communication can be greatly improved by pooling the nodes to perform joint transmission/reception. In this paper, we consider the problem of optimally selecting a subset of nodes from potentially a large number of candidates to form a virtual multi-antenna system, while at the same time designing their joint linear transmission strategies. We focus on two specific application scenarios: 1) multiple single antenna transmitters cooperatively transmit to a receiver; 2) a single transmitter transmits to a receiver with the help of a number of cooperative relays. We formulate the joint node selection and beamforming problems as cardinality constrained optimization problems with both discrete variables (used for selecting cooperative nodes) and continuous variables (used for designing beamformers). For each application scenario, we first characterize the computational complexity of the joint optimization problem, and then propose novel semi-definite relaxation (SDR) techniques to obtain approximate solutions. We show that the new SDR algorithms have a guaranteed approximation performance in terms of the gap to global optimality, regardless of channel realizations. The effectiveness of the proposed algorithms is demonstrated via numerical experiments.

Parameterized Complexity of Cardinality Constrained Optimization Problems

We study the parameterized complexity of cardinality constrained optimization problems, i.e. optimization problems that require their solutions to contain specified numbers of elements to optimize solution values. For this purpose, we consider around 20 such optimization problems, as well as their parametric duals, that deal with various fundamental relations among vertices and edges in graphs. We have almost completely settled their parameterized complexity by giving either FPT algorithms or W[1]-hardness proofs. Furthermore, we obtain faster exact algorithms for several cardinality constrained optimization problems by transforming them into problems of finding maximum (minimum) weight triangles in weighted graphs.