Sparse Rank Research Articles

Sparse Convoluted Rank Regression in High Dimensions

Wang et al. studied the high-dimensional sparse penalized rank regression and established its nice theoretical properties. Compared with the least squares, rank regression can have a substantial gain in estimation efficiency while maintaining a minimal relative efficiency of 86.4 % . However, the computation of penalized rank regression can be very challenging for high-dimensional data, due to the highly nonsmooth rank regression loss. In this work we view the rank regression loss as a nonsmooth empirical counterpart of a population level quantity, and a smooth empirical counterpart is derived by substituting a kernel density estimator for the true distribution in the expectation calculation. This view leads to the convoluted rank regression loss and consequently the sparse penalized convoluted rank regression (CRR) for high-dimensional data. We prove some interesting asymptotic properties of CRR. Under the same key assumptions for sparse rank regression, we establish the rate of convergence of the l 1 -penalized CRR for a tuning free penalization parameter and prove the strong oracle property of the folded concave penalized CRR. We further propose a high-dimensional Bayesian information criterion for selecting the penalization parameter in folded concave penalized CRR and prove its selection consistency. We derive an efficient algorithm for solving sparse convoluted rank regression that scales well with high dimensions. Numerical examples demonstrate the promising performance of the sparse convoluted rank regression over the sparse rank regression. Our theoretical and numerical results suggest that sparse convoluted rank regression enjoys the best of both sparse least squares regression and sparse rank regression. Supplementary materials for this article are available online.

Implementation of a real‐time stereo vision algorithm on a cost‐effective heterogeneous multicore platform

SummaryStereo vision is a major computer vision technique commonly used for robotics applications. Existing software implementations of this technique on general‐purpose processors offer low time‐to‐market compared to other platforms. However, such implementations can hardly achieve real‐time and their cost is usually relatively high. These issues can be solved by embedded multicore platforms. In this article, we present a low‐cost, improved software implementation of a stereo matching algorithm in the correlation stage that combines a sparse rank transform with a combination of sum of absolute differences 1‐D and 2‐D box filtering algorithms. A circular buffer scheme is used to optimize memory usage during the rank computation stage. The system runs on a heterogeneous multicore platform (ODROID XU4). Through the extensive use of single instruction multiple data Neon intrinsics, the system can process images with a size of pixels and a disparity range of 20 pixels at a rate of 111 frames per second. The proposed system can be used in mobile robot platforms that require low power consumption while delivering real‐time performance.

A Robust Tensor-Based Submodule Clustering for Imaging Data Using l12 Regularization and Simultaneous Noise Recovery via Sparse and Low Rank Decomposition Approach.

The massive generation of data, which includes images and videos, has made data management, analysis, information extraction difficult in recent years. To gather relevant information, this large amount of data needs to be grouped. Real-life data may be noise corrupted during data collection or transmission, and the majority of them are unlabeled, allowing for the use of robust unsupervised clustering techniques. Traditional clustering techniques, which vectorize the images, are unable to keep the geometrical structure of the images. Hence, a robust tensor-based submodule clustering method based on regularization with improved clustering capability is formulated. The induced tensor nuclear norm (TNN), integrated into the proposed method, offers better low rankness while retaining the self-expressiveness property of submodules. Unlike existing methods, the proposed method employs a simultaneous noise removal technique by twisting the lateral image slices of the input data tensor into frontal slices and eliminates the noise content in each image, using the principles of the sparse and low rank decomposition technique. Experiments are carried out over three datasets with varying amounts of sparse, Gaussian and salt and pepper noise. The experimental results demonstrate the superior performance of the proposed method over the existing state-of-the-art methods.

Nonrecurrent traffic congestion detection with a coupled scalable Bayesian robust tensor factorization model

Nonrecurrent traffic congestion (NRTC) usually brings unexpected delays to commuters. Hence, it is critical to accurately detect and recognize the NRTC in a real-time manner. The advancement of road traffic detectors provides researchers with a large-scale multivariable temporal-spatial traffic data, which allows the deep research on NRTC to be conducted. However, it remains a challenging task to construct an analytical framework through which the natural temporal-spatial structural properties of multivariable traffic information can be effectively represented and exploited to better understand and detect NRTC. In this paper, we present a novel analytical training-free framework based on the coupled scalable Bayesian robust tensor factorization (Coupled SBRTF). The framework can couple multivariable traffic variables including traffic flow, road speed, and occupancy through sharing the same sparse structure. Moreover, it naturally captures the high-dimensional temporal-spatial patterns of the traffic data by tensor factorization. With its entries revealing the distribution and magnitude of NRTC, the shared sparse structure of the framework compasses sufficiently abundant information about NRTC. While the low-rank part of the framework, expresses the distribution of general expected traffic conditions as an auxiliary product. Experimental results on real-world traffic data show that the proposed method outperforms the NRTC detection models based on the coupled Bayesian robust principal component analysis (coupled BRPCA), the rank sparsity tensor decomposition (RSTD), and standard normal deviates (SND). The proposed method performs even better when only traffic data in weekdays are utilized, and hence can provide more precise estimations of NRTC for daily commuters.

Block rank sparsity-aware DOA estimation with large-scale arrays in the presence of unknown mutual coupling

With the development of massive multiple-input mutiple-output (MIMO) technique, high-resolution direction-of-arrival (DOA) estimation has attracted great attention. A novel sparse signal reconstruction method based on the inherent block rank sparsity of the sub-matrix is proposed for high resolution DOA estimation with large-scale arrays under the condition of unknown mutual coupling. In the proposed method, by taking advantage of the banded symmetric Toeplitz structure of the mutual coupling matrix (MCM), a novel block representation model is firstly formulated by parameterizing the steering vector. Then, exploiting the inherent block sparsity characteristics of the sub-matrix, a reweighted nuclear norm minimization algorithm is proposed to reconstruct the sparse matrix, in which the weighted matrix is designed by using the spectrum of MUSIC-Like algorithm. Finally, the DOAs are achieved by searching the non-zeros blocks of the recovered matrix. The proposed method not only makes full use of the block rank sparsity characteristics of the sub-matrix and weighted matrix for enhancing the sparse solution, but also avoids the array aperture loss. Thus, the proposed method has superior estimation performance than the state-of-the-art algorithms under the condition of unknown mutual coupling. Especially, in the case of large-scale antennas, the advantage of the proposed method is more obvious. Some computer simulation results are performed to verify the advantage of our proposed method.

Reweighted nuclear norm minimisation for DOA estimation with unknown mutual coupling

A reweighted nuclear norm minimisation algorithm by considering the inherent rank sparsity of the submatrix block for direction-of-arrival (DOA) estimation with unknown mutual coupling in a uniform linear array (ULA) is proposed. A novel block overcomplete dictionary is first derived by parameterising the steering vector to avoid the unknown mutual coupling effect. Then, in order to take advantage of the inherent rank sparsity of the submatrix block, a reweighted nuclear norm minimisation algorithm is introduced, and the weighted matrix can be designed by the Capon spectrum for enhancing the sparsity of scheme. Finally, the DOA estimates can be obtained by the spatial spectrum of the reconstructed matrix. Simulation results verify that the proposed method has distinct advantages over state-of-the-art methods.

Nonconvex Low Tubal Rank Tensor Minimization

In the sparse vector recovery problem, the L 0 -norm can be approximated by a convex function or a nonconvex function to achieve sparse solutions. In the low-rank matrix recovery problem, the nonconvex matrix rank can be replaced by a convex function or a nonconvex function on the singular value of matrix to achieve low-rank solutions. Although the convex relaxation can easily lead to the optimal solution, the nonconvex approximation tends to yield more sparse or lower rank local solutions. As a natural extension of vector and matrix to high order structure, tensor can better represent the essential structure of data for modeling the high-dimensional data. In this paper, we study the low tubal rank tensor recovery problem by nonconvex optimization. Instead of using convex tensor nuclear norm, we use nonconvex surrogate functions to approximate the tensor tubal rank, and propose a tensor based iteratively reweighted nuclear norm solver. We further provide the convergence analysis of our new solver. Sufficient experiments on synthetic data and real images verify the effectiveness of our new method.

3D shape co-segmentation via sparse and low rank representations

In this paper, we propose a 3D shape co-segmentation method, which divides 3D shapes in the same category into consistent feature representations. We involve sparse and low-rank constraints to obtain compact feature representations among the 3D shapes. After pre-segmentation and feature extraction processes, we convert the co-segmentation problem into feature clustering issues. With the sparse and low-rank constraints, the initial geometry features are mapped into a compact coefficient space. Then, we gather the coefficients and weight them by a confidence weighting procedure. Finally, we apply fuzzy cuts method for optimization and achieve the final shape co-segmentation results. Experimental results on two public benchmarks demonstrate that our approach is robust for various 3D meshes, and outperforms other state-of-the-art approaches.

Rank–sparsity balanced representation for subspace clustering

Subspace learning has many applications such as motion segmentation and image recognition. The existing algorithms based on self-expressiveness of samples for subspace learning may suffer from the unsuitable balance between the rank and sparsity of the expressive matrix. In this paper, a new model is proposed that can balance the rank and sparsity well. This model adopts the log-determinant function to control the rank of solution. Meanwhile, the diagonals are penalized, rather than the strict zero-restriction on diagonals. This strategy makes the rank–sparsity balance more tunable. We furthermore give a new graph construction from the low-rank and sparse solution, which absorbs the advantages of the graph constructions in the sparse subspace clustering and the low-rank representation for further clustering. Numerical experiments show that the new method, named as RSBR, can significantly increase the accuracy of subspace clustering on the real-world data sets that we tested.

Semiring Rank Matrix Factorization

Rank data, in which each row is a complete or partial ranking of available items (columns), is ubiquitous. Among others, it can be used to represent preferences of users, levels of gene expression, and outcomes of sports events. It can have many types of patterns, among which consistent rankings of a subset of the items in multiple rows, and multiple rows that rank the same subset of the items highly. In this article, we show that the problems of finding such patterns can be formulated within a single generic framework that is based on the concept of semiring matrix factorization. In this framework, we employ the max-product semiring rather than the plus-product semiring common in traditional linear algebra. We apply this semiring matrix factorization framework on two tasks: sparse rank matrix factorization and rank matrix tiling. Experiments on both synthetic and real world datasets show that the framework is capable of discovering different types of structure as well as obtaining high quality solutions.

Extreme point inequalities and geometry of the rank sparsity ball

We investigate geometric features of the unit ball corresponding to the sum of the nuclear norm of a matrix and the $$l_1$$l1 norm of its entries--a common penalty function encouraging joint low rank and high sparsity. As a byproduct of this effort, we develop a calculus (or algebra) of faces for general convex functions, yielding a simple and unified approach for deriving inequalities balancing the various features of the optimization problem at hand, at the extreme points of the solution set.

Putting nonnegative matrix factorization to the test: a tutorial derivation of pertinent cramer—rao bounds and performance benchmarking

Nonnegative matrix factorization (NMF) is a useful tool in a broad range of applications, from signal separation to computer vision and machine learning. NMF is a hard (NP-hard) computational problem for which various approximate solutions have been developed over the years. Given the widespread interest in NMF and its applications, it is perhaps surprising that the pertinent Cram?r?Rao lower bound (CRLB) on the accuracy of the nonnegative latent factor estimates has not been worked out in the literature. In hindsight, one reason may be that the required computations are more subtle than usual: the problem involves constraints and ambiguities that must be dealt with, and the Fisher information matrix is always singular. We provide a concise tutorial derivation of the CRLB for both symmetric NMF and asymmetric NMF, using the latest CRLB tools, which should be of broad interest for analogous derivations in related factor analysis problems. We illustrate the behavior of these bounds with respect to model parameters and put some of the best NMF algorithms to the test against one another and the CRLB. The results help illuminate what can be expected from the current state of art in NMF algorithms, and they are reassuring in that the gap to optimality is small in relatively sparse and low rank scenarios.

A fast recursive orthogonalization scheme for the Macaulay matrix

In this article we present a fast recursive orthogonalization scheme for two important subspaces of the Macaulay matrix: its row space and null space. It requires a graded monomial ordering and exploits the resulting structure of the Macaulay matrix induced by this graded ordering. The resulting orthogonal basis for the row space will retain a similar structure as the Macaulay matrix and is as a consequence sparse. The computed orthogonal basis for the null space is dense but typically has smaller dimensions. Two alternative implementations for the recursive orthogonalization scheme are presented: one using the singular value decomposition and another using a sparse rank revealing multifrontal QR decomposition. Numerical experiments show the effectiveness of the proposed recursive orthogonalization scheme in both running time and required memory compared to a standard orthogonalization. The sparse multifrontal QR implementation is superior in both total run time and required memory at the cost of being slightly less reliable for determining the numerical rank.

Randomized Sparse Direct Solvers

We propose randomized direct solvers for large sparse linear systems, which integrate randomization into rank structured multifrontal methods. The use of randomization highly simplifies various essential steps in structured solutions, where fast operations on skinny matrix-vector products replace traditional complex ones on dense or structured matrices. The new methods thus significantly enhance the flexibility and efficiency of using structured methods in sparse solutions. We also consider a variety of techniques, such as some graph methods, the inclusion of additional structures, the concept of reduced matrices, information reuse, and adaptive schemes. The methods are applicable to various sparse matrices with certain rank structures. Particularly, for discretized matrices whose factorizations yield dense fill-in with some off-diagonal rank patterns, the factorizations cost about $O(n)$ flops in two dimensions (2D), and about $O(n)$ to $O(n^{4/3})$ flops in three dimensions (3D). The solution costs and memory sizes are nearly $O(n)$ in both 2D and 3D. These counts are obtained based on two optimization strategies and a sparse rank relaxation idea. The methods are especially useful for approximate solutions and preconditioning. Numerical tests on both discretized PDEs and more general problems are used to demonstrate the efficiency and accuracy. The ideas here also have the potential to be generalized to matrix-free sparse direct solvers based on matrix-vector multiplications in future developments.

Hole-Filling by Rank Sparsity Tensor Decomposition for Medical Imaging

Surface integrity of 3D medical data is crucial for surgery simulation or virtual diagnoses. However, undesirable holes often exist due to external damage on bodies or accessibility limitation on scanners. To bridge the gap, hole-filling for medical imaging is a popular research topic in recent years[1]-[3]. Considering that a medical image, e.g. CT or MRI, has the natural form of a tensor, we recognize the problem of medical hole-filling as the extension of Principal Component Pursuit (PCP) problem from matrix case to tensor case. Since the new problem in the tensor case is much more difficult than the matrix case, an efficient algorithm for the extension is presented by relaxation technique. The most significant feature of our algorithm is that unlike traditional methods which follow a strictly local approach, our method fixes the hole by the global structure in the specific medical data. Another important difference from the previous algorithm [4] is that our algorithm is able to automatically separate the completed data from the hole in an implicit manner. Our experiments demonstrate that the proposed method can lead to satisfactory results.

Conjugate Gradient Method for Rank Deficient Saddle Point Problems

We propose an alternative iterative method to solve rank deficient problems arising in many real applications such as the finite element approximation to the Stokes equation and computational genetics. Our main contribution is to transform the rank deficient problem into a smaller full rank problem, with structure as sparse as possible. The new system improves the condition number greatly. Numerical experiments suggest that the new iterative method works very well for large sparse rank deficient saddle point problems.