Rank Minimization Problem Research Articles

Functional identification of complex cells from spike times and the decoding of visual stimuli

Neural circuits built with complex cells play a key role in modeling the primary visual cortex. The encoding capability of an ensemble of complex cells has not been systematically studied, however. Can visual scenes be reconstructed using the spike times generated by an ensemble of complex cells? Can the processing taking place in complex cells be identified with high accuracy? Processing by complex cells has the complexity of Volterra models [1]. General Volterra based models call, among others, for efficient functional identification and decoding algorithms. We demonstrate that complex cells exhibit Volterra dendritic stimulus processors (Volterra DSPs) that are analytically and computationally tractable. Decoding and identification problems arising in neural circuits built with complex cells can be efficiently solved as rank minimization problems [2]. We provide (i) an algorithm that reconstructs the visual stimuli based on the spike times generated by circuits with widely employed complex cells models (Complex Cell Time Decoding Machines), and (ii) propose a mechanistic algorithm for functionally identifying the processing in complex cells using the spike times they generate (Complex Cell Channel Identification Machines). These algorithms are based on the key observation that the functional identification of processing in a single complex cell is dual to the problem of decoding stimuli encoded by an ensemble of complex cells. In addition, we show that the number of spikes needed for perfect reconstruction of a band-limited stimulus is proportional to the dimension of the stimulus space rather than the square of its dimension, thereby reducing the required number of neurons/measurements to a physiologically plausible range. This result demonstrates that visual stimuli can be efficiently reconstructed from the amplitude information carried in the complex cells alone. Similar results obtained for identification establish the computational tractability of higher order Volterra DSPs. We provide examples of perfect reconstruction of space-time stimuli (Figure ​(Figure1A)1A) and examples of identification of complex cell DSPs (Figure ​(Figure1B).1B). We demonstrate that our identification algorithms substantially outperform algorithms based on spike-triggered covariance (STC) (Figure ​(Figure1C).1C). Finally, we evaluate our identification algorithms by reconstructing novel stimuli in the input space using identified Volterra DSPs (Figure ​(Figure1D)1D) [3]. Figure 1 A. SNR of reconstruction when varying the number of measurements/spikes (dimension of the input space: 117). B. Mean SNR of identified complex cell DSP when varying the number of input trials used in identification. C. Comparison of identification by ...

Low-Rank Matrix Decomposition and Spatio-Temporal Sparse Recovery for STAP Radar

We develop space-time adaptive processing (STAP) methods by leveraging the advantages of sparse signal processing techniques in order to detect a slowly-moving target. We observe that the inherent sparse characteristics of a STAP problem can be formulated as the low-rankness of clutter covariance matrix when compared to the total adaptive degrees-of-freedom, and also as the sparse interference spectrum on the spatio-temporal domain. By exploiting these sparse properties, we propose two approaches for estimating the interference covariance matrix. In the first approach, we consider a constrained matrix rank minimization problem (RMP) to decompose the sample covariance matrix into a low-rank positive semidefinite and a diagonal matrix. The solution of RMP is obtained by applying the trace minimization technique and the singular value decomposition with matrix shrinkage operator. Our second approach deals with the atomic norm minimization problem to recover the clutter response-vector that has a sparse support on the spatio-temporal plane. We use convex relaxation based standard sparse-recovery techniques to find the solutions. With extensive numerical examples, we demonstrate the performances of proposed STAP approaches with respect to both the ideal and practical scenarios, involving Doppler-ambiguous clutter ridges, spatial and temporal decorrelation effects. The low-rank matrix decomposition based solution requires secondary measurements as many as twice the clutter rank to attain a near-ideal STAP performance; whereas the spatio-temporal sparsity based approach needs a considerably small number of secondary data.

NuMax: A Convex Approach for Learning Near-Isometric Linear Embeddings

We propose a novel framework for the deterministic construction of linear, near-isometric embeddings of a finite set of data points. Given a set of training points X ⊂ \BBR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sup> , we consider the secant set S(X) that consists of all pairwise difference vectors of X, normalized to lie on the unit sphere. We formulate an affine rank minimization problem to construct a matrix Ψ that preserves the norms of all the vectors in S(X) up to a distortion parameter δ. While affine rank minimization is NP-hard, we show that this problem can be relaxed to a convex formulation that can be solved using a tractable semidefinite program (SDP). In order to enable scalability of our proposed SDP to very large-scale problems, we adopt a two-stage approach. First, in order to reduce compute time, we develop a novel algorithm based on the Alternating Direction Method of Multipliers (ADMM) that we call Nuclear norm minimization with Max-norm constraints (NuMax) to solve the SDP. Second, we develop a greedy, approximate version of NuMax based on the column generation method commonly used to solve large-scale linear programs. We demonstrate that our framework is useful for a number of signal processing applications via a range of experiments on large-scale synthetic and real datasets.

Fast low rank representation based spatial pyramid matching for image classification

Spatial Pyramid Matching (SPM) and its variants have achieved a lot of success in image classification. The main difference among them is their encoding schemes. For example, ScSPM incorporates Sparse Code (SC) instead of Vector Quantization (VQ) into the framework of SPM. Although the methods achieve a higher recognition rate than the traditional SPM, they consume more time to encode the local descriptors extracted from the image. In this paper, we propose using Low Rank Representation (LRR) to encode the descriptors under the framework of SPM. Different from SC, LRR considers the group effect among data points instead of sparsity. Benefiting from this property, the proposed method (i.e., LrrSPM) can offer a better performance. To further improve the generalizability and robustness, we reformulate the rank-minimization problem as a truncated projection problem. Extensive experimental studies show that LrrSPM is more efficient than its counterparts (e.g., ScSPM) while achieving competitive recognition rates on nine image data sets.

Non-rigid structure estimation in trajectory space from monocular vision.

In this paper, the problem of non-rigid structure estimation in trajectory space from monocular vision is investigated. Similar to the Point Trajectory Approach (PTA), based on characteristic points’ trajectories described by a predefined Discrete Cosine Transform (DCT) basis, the structure matrix was also calculated by using a factorization method. To further optimize the non-rigid structure estimation from monocular vision, the rank minimization problem about structure matrix is proposed to implement the non-rigid structure estimation by introducing the basic low-rank condition. Moreover, the Accelerated Proximal Gradient (APG) algorithm is proposed to solve the rank minimization problem, and the initial structure matrix calculated by the PTA method is optimized. The APG algorithm can converge to efficient solutions quickly and lessen the reconstruction error obviously. The reconstruction results of real image sequences indicate that the proposed approach runs reliably, and effectively improves the accuracy of non-rigid structure estimation from monocular vision.

The Analysis of Multivariate Data Using Semi-Definite Programming

A model is presented for analyzing general multivariate data. The model puts as its prime objective the dimensionality reduction of the multivariate problem. The only requirement of the model is that the input data to the statistical analysis be a covariance matrix, a correlation matrix, or more generally a positive semi-definite matrix. The model is parameterized by a scale parameter and a shape parameter both of which take on non-negative values smaller than unity. We first prove a wellknown heuristic for minimizing rank and establish the conditions under which rank can be replaced with trace. This result allows us to solve our rank minimization problem as a Semi-Definite Programming (SDP) problem by a number of available solvers. We then apply the model to four case studies dealing with four well-known problems in multivariate analysis. The first problem is to determine the number of underlying factors in factor analysis (FA) or the number of retained components in principal component analysis (PCA). It is shown that our model determines the number of factors or components more efficiently than the commonly used methods. The second example deals with a problem that has received much attention in recent years due to its wide applications, and it concerns sparse principal components and variable selection in PCA. When applied to a data set known in the literature as the pitprop data, we see that our approach yields PCs with larger variances than PCs derived from other approaches. The third problem concerns sensitivity analysis of the multivariate models, a topic not widely researched in the sequel due to its difficulty. Finally, we apply the model to a difficult problem in PCA known as lack of scale invariance in the solutions of PCA. This is the problem that the solutions derived from analyzing the covariance matrix in PCA are generally different (and not linearly related to) the solutions derived from analyzing the correlation matrix. Using our model, we obtain the same solution whether we analyze the correlation matrix or the covariance matrix since the analysis utilizes only the signs of the correlations/covariances but not their values. This is where we introduce a new type of PCA, called Sign PCA, which we speculate on its applications in social sciences and other fields of science.

Learning Robust and Discriminative Subspace With Low-Rank Constraints.

In this paper, we aim at learning robust and discriminative subspaces from noisy data. Subspace learning is widely used in extracting discriminative features for classification. However, when data are contaminated with severe noise, the performance of most existing subspace learning methods would be limited. Recent advances in low-rank modeling provide effective solutions for removing noise or outliers contained in sample sets, which motivates us to take advantage of low-rank constraints in order to exploit robust and discriminative subspace for classification. In particular, we present a discriminative subspace learning method called the supervised regularization-based robust subspace (SRRS) approach, by incorporating the low-rank constraint. SRRS seeks low-rank representations from the noisy data, and learns a discriminative subspace from the recovered clean data jointly. A supervised regularization function is designed to make use of the class label information, and therefore to enhance the discriminability of subspace. Our approach is formulated as a constrained rank-minimization problem. We design an inexact augmented Lagrange multiplier optimization algorithm to solve it. Unlike the existing sparse representation and low-rank learning methods, our approach learns a low-dimensional subspace from recovered data, and explicitly incorporates the supervised information. Our approach and some baselines are evaluated on the COIL-100, ALOI, Extended YaleB, FERET, AR, and KinFace databases. The experimental results demonstrate the effectiveness of our approach, especially when the data contain considerable noise or variations.

A penalty decomposition method for rank minimization problem with affine constraints

The rank minimization problem with affine constraints is widely applied in the fields of control, system identification, and machine learning, and attracted much attention and well studied in the past few years. Unlike most of the existing methods where a nuclear norm is used to approximate the rank term, in this paper, we apply the penalty decomposition method to solve the rank minimization problem directly. One subproblem can be solved effectively by using linear conjugate gradient method, and the other one has closed-form solutions by taking full use of the problem’s favorable structure. Under some suitable assumptions, the convergence results for the proposed method are given. Finally, we do numerical experiments on randomly generated and real data, the results show that the proposed method is effective and promising.

On finding a generalized lowest rank solution to a linear semi-definite feasibility problem

In this note, we generalize the affine rank minimization problem and the vector cardinality minimization problem and show that the resulting generalized problem can be solved by solving a sequence of continuous concave minimization problems. In the case of the vector cardinality minimization problem, we show that it can be solved exactly by solving the continuous concave minimization problem.

A Nonconvex Relaxation Approach for Rank Minimization Problems

Recently, solving rank minimization problems by leveraging nonconvex relaxations has received significant attention. Some theoretical analyses demonstrate that it can provide a better approximation of original problems than convex relaxations. However, designing an effective algorithm to solve nonconvex optimization problems remains a big challenge. In this paper, we propose an Iterative Shrinkage-Thresholding and Reweighted Algorithm (ISTRA) to solve rank minimization problems using the nonconvex weighted nuclear norm as a low rank regularizer. We prove theoretically that under certain assumptions our method achieves a high-quality local optimal solution efficiently. Experimental results on synthetic and real data show that the proposed ISTRA algorithm outperforms state-of-the-art methods in both accuracy and efficiency.

Generalized Singular Value Thresholding

This work studies the Generalized Singular Value Thresholding (GSVT) operator associated with a nonconvex function g defined on the singular values of X. We prove that GSVT can be obtained by performing the proximal operator of g on the singular values since Proxg(.) is monotone when g is lower bounded. If the nonconvex g satisfies some conditions (many popular nonconvex surrogate functions, e.g., lp-norm, 0 &lt; p &lt; 1, of l0-norm are special cases), a general solver to find Proxg(b) is proposed for any b ≥ 0. GSVT greatly generalizes the known Singular Value Thresholding (SVT) which is a basic subroutine in many convex low rank minimization methods. We are able to solve the nonconvex low rank minimization problem by using GSVT in place of SVT.

Linear Models Based on Noisy Data and the Frisch Scheme.

We address the problem of identifying linear relations among variables based on noisy measurements. This is a central question in the search for structure in large data sets. Often a key assumption is that measurement errors in each variable are independent. This basic formulation has its roots in the work of Charles Spearman in 1904 and of Ragnar Frisch in the 1930s. Various topics such as errors-in-variables, factor analysis, and instrumental variables all refer to alternative viewpoints on this problem and on ways to account for the anticipated way that noise enters the data. In the present paper we begin by describing certain fundamental contributions by the founders of the field and provide alternative modern proofs to certain key results. We then go on to consider a modern viewpoint and novel numerical techniques to the problem. The central theme is expressed by the Frisch-Kalman dictum, which calls for identifying a noise contribution that allows a maximal number of simultaneous linear relations among the noise-free variables-a rank minimization problem. In the years since Frisch's original formulation, there have been several insights, including trace minimization as a convenient heuristic to replace rank minimization. We discuss convex relaxations and theoretical bounds on the rank that, when met, provide guarantees for global optimality. A complementary point of view to this minimum-rank dictum is presented in which models are sought leading to a uniformly optimal quadratic estimation error for the error-free variables. Points of contact between these formalisms are discussed, and alternative regularization schemes are presented.

Convergence analysis of projected gradient descent for Schatten-p nonconvex matrix recovery

The matrix rank minimization problem arises in many engineering applications. As this problem is NP-hard, a nonconvex relaxation of matrix rank minimization, called the Schatten- p quasi-norm minimization (0 p p quasi-norm minimization (0 p <1) problem. Based on the matrix restricted isometry property (M-RIP), we give the convergence guarantee and error bound for this algorithm and show that the algorithm is robust to noise with an exponential convergence rate.

Iterative Concave Rank Approximation for Recovering Low-Rank Matrices

In this paper, we propose a new algorithm for recovery of low-rank matrices from compressed linear measurements. The underlying idea of this algorithm is to closely approximate the rank function with a smooth function of singular values, and then minimize the resulting approximation subject to the linear constraints. The accuracy of the approximation is controlled via a scaling parameter $\delta$, where a smaller $\delta$ corresponds to a more accurate fitting. The consequent optimization problem for any finite $\delta$ is nonconvex. Therefore, in order to decrease the risk of ending up in local minima, a series of optimizations is performed, starting with optimizing a rough approximation (a large $\delta$) and followed by successively optimizing finer approximations of the rank with smaller $\delta$'s. To solve the optimization problem for any $\delta > 0$, it is converted to a new program in which the cost is a function of two auxiliary positive semidefinete variables. The paper shows that this new program is concave and applies a majorize-minimize technique to solve it which, in turn, leads to a few convex optimization iterations. This optimization scheme is also equivalent to a reweighted Nuclear Norm Minimization (NNM), where weighting update depends on the used approximating function. For any $\delta > 0$, we derive a necessary and sufficient condition for the exact recovery which are weaker than those corresponding to NNM. On the numerical side, the proposed algorithm is compared to NNM and a reweighted NNM in solving affine rank minimization and matrix completion problems showing its considerable and consistent superiority in terms of success rate, especially, when the number of measurements decreases toward the lower-bound for the unique representation.

Ghost-Free High Dynamic Range Imaging via Rank Minimization

We propose a ghost-free high dynamic range (HDR) image synthesis algorithm using a low-rank matrix completion framework, which we call RM-HDR. Based on the assumption that irradiance maps are linearly related to low dynamic range (LDR) image exposures, we formulate ghost region detection as a rank minimization problem. We incorporate constraints on moving objects, i.e., sparsity, connectivity, and priors on under- and over-exposed regions into the framework. Experiments on real image collections show that the RM-HDR can often provide significant gains in synthesized HDR image quality over state-of-the-art approaches. Additionally, a complexity analysis is performed which reveals computational merits of RM-HDR over recent advances in deghosting for HDR.

Penalty decomposition methods for rank minimization

In this paper we consider general rank minimization problems with rank appearing either in the objective function or as a constraint. We first establish that a class of special rank minimization problems has closed-form solutions. Using this result, we then propose penalty decomposition (PD) methods for general rank minimization problems in which each subproblem is solved by a block coordinate descent method. Under some suitable assumptions, we show that any accumulation point of the sequence generated by the PD methods satisfies the first-order optimality conditions of a nonlinear reformulation of the problems. Finally, we test the performance of our methods by applying them to the matrix completion and nearest low-rank correlation matrix problems. The computational results demonstrate that our methods are generally comparable or superior to the existing methods in terms of solution quality.

On the Achievable Degrees of Freedom of Two-Cell Multiuser MIMO Interference Networks

In this paper, we study the sum degrees of freedom (DoF) of an uplink two-cell multiuser MIMO interference network with asymmetric number of users in the cells. The achievable DoF is devised based on a two-dimensional space-time spreading code framework with linear precoding/decoding design and finite channel extension. The derivation of the achievable DoF is shown related to a rank minimization problem, which corresponds to the minimization of the dimension of the interference subspace. The problem is solved by the proposed grouping algorithm (GA) based on aligning interfering signals into a low-dimensional subspace as a group and attaining the minimum number of groups. The achievable sum DoF derived based on the proposed GA is shown to be greater than prior arts and achieves the theoretic upper bound in several cases. We also give a closed-form expression of the maximum achievable sum DoF when there is the maximum number of admissible users in the considered finite diversity environment.

Exact minimum rank approximation via Schatten [formula omitted]-norm minimization

Minimizing the rank of a matrix with a given system of affine constraints is to recover the lowest-rank matrix with many important applications in engineering and science. A convex relaxation of the rank minimization problem by minimizing the nuclear norm instead of the rank of the matrix has recently been proposed. Recht and Fazel concluded that nuclear norm minimization subject to affine constraints is equivalent to rank minimization under a certain condition in terms of the rank-restricted isometry property. In this paper, we extend some recent results from nuclear norm minimization to Schatten p-norm minimization and present a theoretical guarantee for the Schatten p-norm minimization if a certain restricted isometry property holds for the linear affine transform. Our results improve on the previous works where recovery is used for nuclear norm minimization. An algorithm based on the Majorization Minimization algorithm has been proposed to solve Schatten p-norm minimization. By using an approximate singular value decomposition procedure, we obtain a fast and robust algorithm. The numerical results on the matrix completion problem with noisy measurements indicate that our algorithm gives a more accurate reconstruction and takes less time compared with some other existing algorithms.

Iterative partial matrix shrinkage algorithm for matrix rank minimization

This paper proposes a new matrix shrinkage algorithm for matrix rank minimization problems. The proposed algorithm provides a low rank solution by estimating a matrix rank and shrinking non-dominant singular values iteratively. We study the convergence properties of the algorithm, which indicate that the algorithm gives approximate low-rank solutions. Numerical results show that the proposed algorithm works efficiently for hard problems with low computing time.

Blind Identification via Lifting

Blind system identification is known to be an ill-posed problem and without further assumptions, no unique solution is at hand. In this contribution, we are concerned with the task of identifying an ARX model from only output measurements. We phrase this as a constrained rank minimization problem and present a relaxed convex formulation to approximate its solution. To make the problem well posed we assume that the sought input lies in some known linear subspace.