Rank-constrained Optimization Problem Research Articles

Exact penalization for cardinality and rank-constrained optimization problems via partial regularization

In this paper, we consider a class of constrained optimization problems whose constraints involve a cardinality or rank constraint. The penalty formulation based on a partial regularization has recently been promoted in the literature to approximate these problems, which usually outperforms the penalty formulation based on a full regularization in terms of solution quality. Nevertheless, the relation between the penalty formulation with a partial regularizer and the original problem was not much studied yet. Under some suitable assumptions, we show that the penalty formulation based on a partial regularization is an exact reformulation of the original problem in the sense that they both share the same global minimizers. We also show that a local minimizer of the original problem is that of the penalty reformulation. These results provide some theoretical justification for the often-observed superior performance of the penalty model based on a partial regularizer over a corresponding full regularizer.

A global exact penalty for rank-constrained optimization problem and applications

A global exact penalty for rank-constrained optimization problem and applications

Calmness of partial perturbation to composite rank constraint systems and its applications

This paper is concerned with the calmness of a partial perturbation to the composite rank constraint system, an intersection of the rank constraint set and a general closed set, which is shown to be equivalent to a local Lipschitz-type error bound and also a global Lipschitz-type error bound under a certain compactness. Based on its lifted formulation, we derive two criteria for identifying those closed sets such that the associated partial perturbation possesses the calmness, and provide a collection of examples to demonstrate that the criteria are satisfied by common nonnegative and positive semidefinite rank constraint sets. Then, we use the calmness of this perturbation to obtain several global exact penalties for rank constrained optimization problems, and a family of equivalent DC surrogates for rank regularized problems.

An Inexact Proximal DC Algorithm with Sieving Strategy for Rank Constrained Least Squares Semidefinite Programming

In this paper, the optimization problem of supervised distance preserving projection (SDPP) for data dimensionality reduction is considered, which is equivalent to a rank constrained least squares semidefinite programming (RCLSSDP). Due to the combinatorial nature of rank function, the rank constrained optimization problems are NP-hard in most cases. In order to overcome the difficulties caused by rank constraint, a difference-of-convex (DC) regularization strategy is employed, then RCLSSDP is transferred into a DC programming. For solving the corresponding DC problem, an inexact proximal DC algorithm with sieving strategy (s-iPDCA) is proposed, whose subproblems are solved by an accelerated block coordinate descent method. The global convergence of the sequence generated by s-iPDCA is proved. To illustrate the efficiency of the proposed algorithm for solving RCLSSDP, s-iPDCA is compared with classical proximal DC algorithm, proximal gradient method, proximal gradient-DC algorithm and proximal DC algorithm with extrapolation by performing dimensionality reduction experiment on COIL-20 database. From the computation time and the quality of solution, the numerical results demonstrate that s-iPDCA outperforms other methods. Moreover, dimensionality reduction experiments for face recognition on ORL and YaleB databases demonstrate that rank constrained kernel SDPP is efficient and competitive when comparing with kernel semidefinite SDPP and kernel principal component analysis in terms of recognition accuracy.

Quantum-Inspired Hierarchy for Rank-Constrained Optimization

Many problems in information theory can be reduced to optimizations over matrices, where the rank of the matrices is constrained. We establish a link between rank-constrained optimization and the theory of quantum entanglement. More precisely, we prove that a large class of rank-constrained semidefinite programs can be written as a convex optimization over separable quantum states and, consequently, we construct a complete hierarchy of semidefinite programs for solving the original problem. This hierarchy not only provides a sequence of certified bounds for the rank-constrained optimization problem, but also gives pretty good and often exact values in practice when the lowest level of the hierarchy is considered. We demonstrate that our approach can be used for relevant problems in quantum information processing, such as the optimization over pure states, the characterization of mixed unitary channels and faithful entanglement, and quantum contextuality, as well as in classical information theory including the maximum cut problem, pseudo-Boolean optimization, and the orthonormal representation of graphs. Finally, we show that our ideas can be extended to rank-constrained quadratic and higher-order programming.

Stabilization of two-dimensional mixed continuous-discrete-time systems via dynamic output feedback with application to iterative learning control design

This paper investigates the asymptotic stabilization of the 2-D continuous-discrete-time systems via dynamic output feedback. The problem is formulated in a general framework, where the system and controller are both described by the Roesser model. By rigorous mathematical works, a linear matrix inequality (LMI) condition is established to ensure the asymptotic stability of the closed-loop system. Using this LMI condition, the problem of determining the stabilizing controller parameters is converted to a rank minimization problem (RMP) with an LMI constraint, or equivalently to a rank-constrained optimization problem (RCOP). When this RMP (or equivalently this RCOP) has a solution, then the controller parameters are obtained uniquely by an explicit formula in terms of this solution. Moreover, it is shown that one can transform the problem of one-dimensional continuous-time iterative learning control (ILC) design to a 2-D stabilization problem, and consequently solve it using the presented two-dimensional scheme. Some numerical examples are given to illustrate the proposed method.

Frequency Response Function identification for multivariable motion control: Optimal experiment design with element-wise constraints

Frequency Response Functions (FRFs) are essential in mechatronic systems and its application ranges from system design and validation to controller design and diagnostics. The aim of this paper is to optimally design experiments for FRF identification of multivariable motion systems subject to element-wise power constraints. A multivariable excitation design framework is established that explicitly addresses the frequency-wise directionality of the system to be identified. The design problem involves solving a rank-constrained optimization problem, which is non-convex and NP-hard in most cases. Two algorithms to solving this problem approximately are presented that rely on a convex (semi-definite) relaxation of the original problem. Additionally, exact solutions for several special cases are presented. The two algorithms are shown to overcome the limitations of traditional excitation design. This is confirmed by experimental results from a 7 × 8 wafer stage setup, which show a significant improvement of the FRF quality using the proposed techniques over traditional design approaches.

Ranking from Crowdsourced Pairwise Comparisons via Smoothed Riemannian Optimization

Social Internet of Things has recently become a promising paradigm for augmenting the capability of humans and devices connected in the networks to provide services. In social Internet of Things network, crowdsourcing that collects the intelligence of the human crowd has served as a powerful tool for data acquisition and distributed computing. To support critical applications (e.g., a recommendation system and assessing the inequality of urban perception), in this article, we shall focus on the collaborative ranking problems for user preference prediction from crowdsourced pairwise comparisons. Based on the Bradley--Terry--Luce (BTL) model, a maximum likelihood estimation (MLE) is proposed via low-rank approach in order to estimate the underlying weight/score matrix, thereby predicting the ranking list for each user. A novel regularized formulation with the smoothed surrogate of elementwise infinity norm is proposed in order to address the unique challenge of the coupled the non-smooth elementwise infinity norm constraint and non-convex low-rank constraint in the MLE problem. We solve the resulting smoothed rank-constrained optimization problem via developing the Riemannian trust-region algorithm on quotient manifolds of fixed-rank matrices, which enjoys the superlinear convergence rate. The admirable performance and algorithmic advantages of the proposed method over the state-of-the-art algorithms are demonstrated via numerical results. Moreover, the proposed method outperforms state-of-the-art algorithms on large collaborative filtering datasets in both success rate of inferring preference and normalized discounted cumulative gain.

Joint Optimization of Dimension Assignment and Compression in Distributed Estimation Fusion

This paper studies linear distributed estimation of an unknown random parameter vector in a bandwidth-constrained multisensor network. To meet the bandwidth limitations, each sensor converts its observation into a low-dimensional datum via a suitable linear transformation. Then, the fusion center estimates the parameter vector by linearly combining all the received low-dimensional data, aiming at minimizing the estimation mean square error. The main purpose of this paper is to jointly determine the compression dimension of each sensor (referred to as dimension assignment) and design the corresponding compression matrix when the total compression dimensions is limited. Such a joint design problem can be formulated as a rank-constrained optimization problem and it is shown to be NP-hard for the first time. In addition, successive quadratic upper-bound minimization (SQUM), SQUM-block coordinate descent (SQUM-BCD) and nuclear norm regularization (NNR) methods are developed to solve it approximately. Furthermore, we show that any accumulation point of the sequence generated by the SQUM method satisfies the Karush-Kuhn-Tucker conditions of the rank-constrained optimization problem, and the Phase II algorithm of the SQUM-BCD and NNR methods (both are two-phase algorithms and have the same Phase II algorithm) guarantees convergence at least to a stationary point. Numerical experiments illustrate the advantages of the proposed methods compared with the existing method.

Identification of the dynamics of time-varying phase aberrations from time histories of the point-spread function.

To optimally compensate for time-varying phase aberrations with adaptive optics, a model of the dynamics of the aberrations is required to predict the phase aberration at the next time step. We model the time-varying behavior of a phase aberration, expressed in Zernike modes, by assuming that the temporal dynamics of the Zernike coefficients can be described by a vector-valued autoregressive (VAR) model. We propose an iterative method based on a convex heuristic for a rank-constrained optimization problem, to jointly estimate the parameters of the VAR model and the Zernike coefficients from a time series of measurements of the point-spread function (PSF) of the optical system. By assuming the phase aberration is small, the relation between aberration and PSF measurements can be approximated by a quadratic function. As such, our method is a blind identification method for linear dynamics in a stochastic Wiener system with a quadratic nonlinearity at the output and a phase retrieval method that uses a time-evolution-model constraint and a single image at every time step.

Joint Beamforming and Power Allocation in Downlink NOMA Multiuser MIMO Networks

In this paper, a novel joint design of beamforming and power allocation is proposed for a multi-cell multiuser multiple-input multiple-output (MIMO) non-orthogonal multiple access (NOMA) network. In this network, base stations (BSs) adopt coordinated multipoint (CoMP) for downlink transmission. We study a new scenario where the users are divided into two groups according to their quality-of-service (QoS) requirements, rather than their channel qualities as investigated in the literature. Our proposed joint design aims to maximize the sum-rate of the users in one group with the best-effort while guaranteeing the minimum required target rates of the users in the other group. The joint design is formulated as a non-convex NP-hard problem. To make the problem tractable, a series of transformations are adopted to simplify the design problem. Then, an iterative suboptimal resource allocation algorithm based on successive convex approximation is proposed. In each iteration, a rank-constrained optimization problem is solved optimally via semidefinite program relaxation. Numerical results reveal that the proposed scheme offers significant sum-rate gains compared to the existing schemes and converges fast to a suboptimal solution.

Flexible Low-Rank Statistical Modeling with Missing Data and Side Information

We explore a general statistical framework for low-rank modeling of matrix-valued data, based on convex optimization with a generalized nuclear norm penalty. We study several related problems: the usual low-rank matrix completion problem with flexible loss functions arising from generalized linear models; reduced-rank regression and multi-task learning; and generalizations of both problems where side information about rows and columns is available, in the form of features or smoothing kernels. We show that our approach encompasses maximum a posteriori estimation arising from Bayesian hierarchical modeling with latent factors, and discuss ramifications of the missing-data mechanism in the context of matrix completion. While the above problems can be naturally posed as rank-constrained optimization problems, which are nonconvex and computationally difficult, we show how to relax them via generalized nuclear norm regularization to obtain convex optimization problems. We discuss algorithms drawing inspiration from modern convex optimization methods to address these large scale convex optimization computational tasks. Finally, we illustrate our flexible approach in problems arising in functional data reconstruction and ecological species distribution modeling.

Identification of low-order models using Laguerre basis function expansions

In this paper, a method for identification of low-order output-error models expressed in terms of Laguerre basis functions is presented. The identification problem is formulated as a rank-constrained optimization problem in the Laguerre domain, which can be solved efficiently using nuclear norm regularization. Since the identified models are of low order, a minimal state-space realization of the system can be obtained without the use of additional approximative model-order reduction steps.

Low-Rank Inducing Norms with Optimality Interpretations

Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations. Nuclear norm regularization is the prevailing convexifying technique for dealing with these types of problem. This paper introduces a family of low-rank inducing norms and regularizers which includes the nuclear norm as a special case. A posteriori guarantees on solving an underlying rank constrained optimization problem with these convex relaxations are provided. We evaluate the performance of the low-rank inducing norms on three matrix completion problems. In all examples, the nuclear norm heuristic is outperformed by convex relaxations based on other low-rank inducing norms. For two of the problems there exist low-rank inducing norms that succeed in recovering the partially unknown matrix, while the nuclear norm fails. These low-rank inducing norms are shown to be representable as semi-definite programs. Moreover, these norms have cheaply computable proximal mappings, which makes it possible to also solve problems of large size using first-order methods.

Sparse Memoryless LQR Design for Uncertain Linear Time-Delay Systems

The sparse memoryless LQR design problem is formulated for uncertain linear time-delay systems. In such a problem, the goal is to minimize a quadratic cost supplemented by sparsity-promoting term (weighted-ℓ1 in our case) subject to stability of closed-loop system under norm-bounded uncertainty. It is shown that such an optimization problem can be reformulated as a rank-constrained optimization problem which consists of convex constraints except one rank constraint. Utilizing the bi-linear rank penalty technique, the sparse memoryless LQR is designed. Numerous numerical results depict that there exists a trade-off between time-delay and sparsification quality. In addition, the larger time-delay, the poorer performance-sparsity trade-off is observed.

Rank-constrained optimization and its applications

This paper investigates an iterative approach to solve the general rank-constrained optimization problems (RCOPs) defined to optimize a convex objective function subject to a set of convex constraints and rank constraints on unknown rectangular matrices. In addition, rank minimization problems (RMPs) are introduced and equivalently transformed into RCOPs by introducing a quadratic matrix equality constraint. The rank function is discontinuous and nonconvex, thus the general RCOPs are classified as NP-hard in most of the cases. An iterative rank minimization (IRM) method, with convex formulation at each iteration, is proposed to gradually approach the constrained rank. The proposed IRM method aims at solving RCOPs with rank inequalities constrained by upper or lower bounds, as well as rank equality constraints. Proof of the convergence to a local minimizer with at least a sublinear convergence rate is provided. Four representative applications of RCOPs and RMPs, including system identification, output feedback stabilization, and structured H2 controller design problems, are presented with comparative simulation results to verify the feasibility and improved performance of the proposed IRM method.

Resource Allocation for Transmit Hybrid Beamforming in Decoupled Millimeter Wave Multiuser-MIMO Downlink

This paper presents a study on joint radio resource allocation and hybrid precoding in multi-carrier massive multiple-input multiple-output communications for 5G cellular networks. In this paper, we present the resource allocation algorithm to maximize the proportional fairness (PF) spectral efficiency under the per subchannel power and the beamforming rank constraints. Two heuristic algorithms are designed. The proportional fairness hybrid beamforming algorithm provides the transmit precoder with a proportional fair spectral efficiency among users for the desired number of radio-frequency (RF) chains. Then, we transform the number of RF chains or rank constrained optimization problem into convex semidefinite programming (SDP) problem, which can be solved by standard techniques. Inspired by the formulated convex SDP problem, a low-complexity, two-step, PF-relaxed optimization algorithm has been provided for the formulated convex optimization problem. Simulation results show that the proposed suboptimal solution to the relaxed optimization problem is near-optimal for the signal-to-noise ratio $SNR\leq 10$ dB and has a performance gap not greater than 2.33 b/s/Hz within the SNR range 0–25 dB. It also outperforms the maximum throughput and PF-based hybrid beamforming schemes for sum spectral efficiency, individual spectral efficiency, and fairness index.

Error bounds for rank constrained optimization problems and applications

For the rank constrained optimization problem whose feasible set is the intersection of the rank constraint set R={X∈X∣rank(X)≤κ} and a closed convex set Ω, we establish the local (global) Lipschitzian type error bounds for estimating the distance from any X∈Ω (X∈X) to the feasible set and the solution set, under the calmness of a multifunction associated to the feasible set at the origin, which is satisfied by three classes of common rank constrained optimization problems.

Robust Adaptive Array Beamforming With Subspace Steering Vector Uncertainties

In this letter, the steering vector of the signal-of-interest (SOI) is assumed to belong to a given linear subspace but the associated coordinates are otherwise unknown. In the case where the signal subspace is poorly estimated, a rank-constrained optimization problem is formulated in which the signal subspace is forced to intersect with the known linear subspace (where the SOI steering vector is located). Such problem can be reformulated as two subproblems via the variable alternation method. Also, the analytical solutions of these two subproblems can be found. Numerical results demonstrate that the proposed method can be regarded as an improved subspace-based method, which lowers the requirements in terms of snapshot number or signal-to-noise ratio (SNR) to outperform the diagonal-loading-type methods as compared with the traditional signal-subspace projection method.

Rank-Constrained Schur-Convex Optimization With Multiple Trace/Log-Det Constraints

Rank-constrained optimization problems have received an increasing intensity of interest recently, because many optimization problems in communications and signal processing applications can be cast into a rank-constrained optimization problem. However, due to the nonconvex nature of rank constraints, a systematic solution to general rank-constrained problems has remained open for a long time. In this paper, we focus on a rank-constrained optimization problem with a Schur-convex/concave objective function and multiple trace/log-determinant constraints. We first derive a structural result on the optimal solution of the rank-constrained problem using majorization theory. Based on the solution structure, we transform the rank-constrained problem into an equivalent problem with a unitary constraint. After that, we derive an iterative projected steepest descent algorithm which converges to a local optimal solution. Furthermore, we shall show that under some special cases, we can derive a closed-form global optimal solution. The numerical results show the superior performance of our proposed technique over the baseline schemes.