Low Rank Approximation Of Matrices Research Articles

Reliable Low-Rank Approximation of Matrices Detection Aided Multicarrier DCSK Receiver Design

Multicarrier differential chaos shift keying (MC-DCSK) systems transmit the reference chaotic signal for information recovery to remove the complex synchronization circuit. However, transmission errors in the reference chaotic signal would induce the error propagations when directly used for demodulations. In order to address this issue, we propose a low-rank approximation of matrices (LRAM) detection method to improve the reliability performances. In our design, we exploit the low-rank characteristic of MC-DCSK signals sharing the reference chaotic signal. Instead of directly using the received reference chaotic signal for correlated demodulation, we propose to apply the LRAM method to jointly estimate the reference signal and the information-bearing signal. We use the singular value decomposition and generalized LRAM methods to minimize the distances between the estimates and transmitted data. Thus, the signal-to-noise ratio (SNR) can be improved to achieve lower bit error rate (BER). Moreover, theoretical BER expression is derived and LRAM-aided system is proved to be capable of proving the maximum-likelihood detection for received signals. Simulation results demonstrate the BER performances of our design outperform counterparts over additive white Gaussian noise and fading channels.

Generalized low-rank approximation of matrices based on multiple transformation pairs

Dimensionality reduction is a critical step in the learning process that plays an essential role in various applications. The most popular methods for dimensionality reduction, SVD and PCA, for instance, only work on one-dimensional data. This means that for higher-order data like matrices or more generally tensors, data should be fold to the vector format. Thus, this approach ignores the spatial relationships of features and increases the probability of overfitting as well. Due to the mentioned issues, several methods like Generalized Low-Rank Approximation of Matrices (GLRAM) and Multilinear PCA (MPCA) proposed to deal with multi-dimensional data in their original format. Consequently, the spatial relationships of features preserved and the probability of overfitting diminished. Besides, the time and space complexity in such methods are less than vector-based ones. However, since the multilinear approach needs fewer parameters, its search space is much smaller than that of the vector-based one. To solve the previous problems of multilinear methods like GLRAM, we proposed a novel extension of GLRAM in which instead one transformation pair use multiple left and right transformation pairs on the projected data. Consequently, this provides the problem with a larger feasible region and smaller reconstruction error. This article provides several analytical discussions and experimental results that confirm the quality of the proposed method.

Ensemble learning-based computational imaging method for electrical capacitance tomography

Electrical capacitance tomography (ECT) is a potential measurement technology for industrial process monitoring, but its applicability is generally limited by low-quality tomographic images. Boosting the performance of inverse computing imaging algorithms is the key to improving the reconstruction quality (RQ). Common regularization iteration imaging methods with analytical prior regularizers are less flexible in dealing with actual reconstruction tasks, leading to large reconstruction errors. To address the challenge, this study proposes a new imaging method, including reconstruction model and optimizer. The data-driven regularizer from a new ensemble learning model and the analytical prior regularizer with the focus on the sparsity of imaging objects are combined into a new optimization model for imaging. In the proposed ensemble learning model, the generalized low rank approximations of matrices (GLRAM) method is used to carry out the dimensionality reduction for decreasing the redundancy of the input data and improving the diversity, the extreme learning machine (ELM) serves as a base learner and the nuclear norm based matrix regression (NNMR) method is developed to aggregate the ensemble of solutions. The singular value thresholding method (SVTM) and the fast iterative shrinkage-thresholding algorithm (FISTA) are inserted into the split Bregman method (SBM) to generate a powerful optimizer for the built computational model. Its comparison to other competing methods through numerical experiments on typical imaging targets demonstrates that the developed algorithm reduces reconstruction error and achieves much more improvement in imaging quality and robustness.

Efficient Low-Rank Approximation of Matrices Based on Randomized Pivoted Decomposition

Given a matrix $\bf A$ with numerical rank $k$ , the two-sided orthogonal decomposition (TSOD) computes a factorization ${\bf A} = {\bf UDV}^T$ , where ${\bf U}$ and ${\bf V}$ are orthogonal, and ${\bf D}$ is (upper/lower) triangular. TSOD is rank-revealing as the middle factor ${\bf D}$ reveals the rank of $\bf A$ . The computation of TSOD, however, is demanding. In this paper, we present an algorithm called randomized pivoted TSOD (RP-TSOD), where the middle factor is lower triangular. Key in our work is the exploitation of randomization, and RP-TSOD is primarily devised to efficiently construct an approximation to a low-rank matrix. We provide three different types of bounds for RP-TSOD: (i) we furnish upper bounds on the error of the low-rank approximation, (ii) we bound the $k$ approximate principal singular values, and (iii) we derive bounds for the canonical angles between the approximate and the exact singular subspaces. Our bounds describe the characteristics and behavior of our proposed algorithm. Through numerical tests, we show the effectiveness of the devised bounds as well as our proposed algorithm.

Approximation Schemes for Low-rank Binary Matrix Approximation Problems

We provide a randomized linear time approximation scheme for a generic problem about clustering of binary vectors subject to additional constraints. The new constrained clustering problem generalizes a number of problems and by solving it, we obtain the first linear time-approximation schemes for a number of well-studied fundamental problems concerning clustering of binary vectors and low-rank approximation of binary matrices. Among the problems solvable by our approach are L ow GF(2)-R ank A pproximation , L ow B oolean -R ank A pproximation , and various versions of B inary C lustering . For example, for L ow GF(2)-R ank A pproximation problem, where for an m × n binary matrix A and integer r > 0, we seek for a binary matrix B of GF(2) rank at most r such that the ℓ 0 -norm of matrix A−B is minimum, our algorithm, for any ϵ > 0 in time f ( r ,ϵ)⋅ n ⋅ m , where f is some computable function, outputs a (1+ϵ)-approximate solution with probability at least (1−1\ e ). This is the first linear time approximation scheme for these problems. We also give (deterministic) PTASes for these problems running in time n f ( r )1\ϵ 2 log 1\ϵ , where f is some function depending on the problem. Our algorithm for the constrained clustering problem is based on a novel sampling lemma, which is interesting on its own.

A cross-product approach for low-rank approximations of large matrices

In this paper we present a new type of restarted Krylov method for calculating low-rank approximations of large matrices. In contrast to former Krylov methods, our approach does not apply the Lanczos algorithm, or Lanczos bidiagonalization. This simplifies the basic iteration and allows the introduction of several innovations.An advantage of the proposed method is that it requires a minimal amount of computer storage. The Krylov matrix which is built at each iteration is using an improved starting vector and a new three-term recurrence relation. These modifications lead to fast rate of convergence. Numerical experiments illustrate the usefulness of the proposed approach.

Low Rank Methods of Approximation in an Electromagnetic Problem

In this article authors present a new method to construct low-rank approximations of dense huge-size matrices. The method develops mosaic-skeleton method and belongs to kernel-independent methods. In distinction from a mosaic-skeleton method, the new one utilizes the hierarchical structure of matrix not only to define matrix block structure but also to calculate factors of low-rank matrix representation. The new method was applied to numerical calculation of boundary integral equations that appear from 3D problem of scattering monochromatic electromagnetic wave by ideal-conducting bodies. The solution of model problem is presented as an example of method evaluation.

Hypercomplex Tensor Completion via Convex Optimization

Expressing multidimensional information as a value in hypercomplex number systems (e.g., quaternion, octonion, etc.) has great potential, in data sciences, e.g., signal processing, to enjoy their nontrivial algebraic benefits which are not available in standard real or complex vector systems. Strategic utilizations of such benefits would include, e.g., hypercomplex singular value decomposition (SVD) and low-rank approximation of matrices. In this paper, as powerful mathematical tools for wider signal processing applications of hypercomplex number systems, we first propose novel definitions of SVD and best low-rank approximation of matrices based on algebraic translations of Cayley–Dickson (C–D) number systems. We then derive an algorithmic solution to hypercomplex tensor completion problem based on a convex optimization technique. Numerical experiments in a scenario of color tensor completion problem show that the proposed algorithm recovers much more faithfully the original color information, masked randomly by noise, than part-wise tensor completion algorithms.

Segmenting Diabetic Retinopathy Lesions in Multispectral Images Using Low-Dimensional Spatial-Spectral Matrix Representation.

Multispectral imaging (MSI) provides a sequence of en-face fundus spectral slices and allows for the examination of structures and signatures throughout the thickness of retina to characterize diabetic retinopathy (DR) lesions comprehensively. Manual interpretation of MSI images is commonly conducted by qualitatively analyzing both the spatial and spectral properties of multiple spectral slices. Meanwhile, there exist few computer-based algorithms that can effectively exploit the spatial and spectral information of MSI images for the diagnosis of DR. We propose a new approach that can quantify the spatial-spectral features of MSI retinal images for automatic DR lesion segmentation. It combines a generalized low-rank approximation of matrices with a supervised regularization term to generate low-dimensional spatial-spectral representations using the feature vectors in all spectral slices. Experimental results showed that the proposed approach is very effective for the segmentation of DR lesions in MSI images, which suggests it as an interesting tool for assisting ophthalmologists in diagnosing, analyzing, and managing DR lesions in MSI.

Face Identification With Top-Push Constrained Generalized Low-Rank Approximation of Matrices

We present a novel top-push constrained feature learning (TFL) method for face identification. The idea is to learn low-rank approximations from raw intensity face images such that the squared distance between all faces of the same identity should be smaller than that of different identities by a margin. To this end, we formulate the learning process under the framework of generalized low-rank approximation of matrices (GLRAM) supervised with a top-push constraint. GLRAM operates on the matrix representation of images to seek low-dimensional and compact features, which improves the discriminative power of the features while reducing the time and space costs. The top-push constraint aims to optimize the face identification accuracy of the top-rank matching list, which makes the learned features more discriminative and compact, thus reducing the variability and ambiguity. Once the optimization is finished, the features are vectorized and used for similarity measurement. The effectiveness of TFL is verified on four publicly available face datasets.

An exploratory data analysis method for identifying brain regions and frequencies of interest from large-scale neural recordings.

High-resolution whole brain recordings have the potential to uncover unknown functionality but also present the challenge of how to find such associations between brain and behavior when presented with a large number of regions and spectral frequencies. In this paper, we propose an exploratory data analysis method that sorts through a massive quantity of multivariate neural recordings to quickly extract a subset of brain regions and frequencies that encode behavior. This approach combines existing tools and exploits low-rank approximation of matrices without a priori selection of regions and frequency bands for analysis. In detail, the spectral content of neural activity across all frequencies of each recording contact is computed and represented as a matrix. Then, the rank-1 approximation of the matrix is computed using singular value decomposition and the associated singular vectors are extracted. The temporal singular vector, which captures the salient features of the spectrogram, is then correlated to the trial-varying behavioral signal. The distribution of correlations for each brain region is efficiently computed and used to find a subset of regions and frequency bands of interest for further examination. As an illustration, we apply this approach to a data set of local field potentials collected using stereoelectroencephalography from a human subject performing a reaching task. Using the proposed procedure, we produced a comprehensive set of brain regions and frequencies related to our specific behavior. We demonstrate how this tool can produce preliminary results that capture neural patterns related to behavior and aid in formulating data-driven hypotheses, hence reducing the time it takes for any scientist to transition from the exploratory to the confirmatory phase.

Supervised non-negative matrix factorization methods for MALDI imaging applications.

MotivationNon-negative matrix factorization (NMF) is a common tool for obtaining low-rank approximations of non-negative data matrices and has been widely used in machine learning, e.g. for supporting feature extraction in high-dimensional classification tasks. In its classical form, NMF is an unsupervised method, i.e. the class labels of the training data are not used when computing the NMF. However, incorporating the classification labels into the NMF algorithms allows to specifically guide them toward the extraction of data patterns relevant for discriminating the respective classes. This approach is particularly suited for the analysis of mass spectrometry imaging (MSI) data in clinical applications, such as tumor typing and classification, which are among the most challenging tasks in pathology. Thus, we investigate algorithms for extracting tumor-specific spectral patterns from MSI data by NMF methods.ResultsIn this article, we incorporate a priori class labels into the NMF cost functional by adding appropriate supervised penalty terms. Numerical experiments on a MALDI imaging dataset confirm that the novel supervised NMF methods lead to significantly better classification accuracy and stability as compared with other standard approaches.Availability and implementaton https://gitlab.informatik.uni-bremen.de/digipath/Supervised_NMF_Methods_for_MALDI.git Supplementary information Supplementary data are available at Bioinformatics online.

Numerical Tensor Techniques for Multidimensional Convolution Products

In order to treat high-dimensional problems, one has to find data-sparse representations. Starting with a six-dimensional problem, we first introduce the low-rank approximation of matrices. One purpose is the reduction of memory requirements, another advantage is that now vector operations instead of matrix operations can be applied. In the considered problem, the vectors correspond to grid functions defined on a three-dimensional grid. This leads to the next separation: these grid functions are tensors in mathbb {R}^{n}otimes mathbb {R}^{n}otimes mathbb {R}^{n} and can be represented by the hierarchical tensor format. Typical operations as the Hadamard product and the convolution are now reduced to operations between mathbb {R}^{n} vectors. Standard algorithms for operations with vectors from mathbb {R}^{n} are of order mathcal {O}(n) or larger. The tensorisation method is a representation method introducing additional data-sparsity. In many cases, the data size can be reduced from mathcal {O}(n) to mathcal {O}(log n). Even more important, operations as the convolution can be performed with a cost corresponding to these data sizes.

Covariance Modeling in Applications of Data Assimilation to High-dimensional Earth and Geospace Systems

Data assimilation and inverse methods play a key role in integrating remote-sensing and in-situ Earth and Geospace observations into models of the Earth and Geospace system and subsystems, enabling weather prediction and climate projection of high societal relevance. The problem presents considerable methodological and computational challenges because of the high-dimensionality and non-linearity of the underling dynamics of the Earth and Geospace system and the large volumes of heterogeneous observations from various observing platforms, including Low-Earth-Orbit and geosynchronous-Earth-orbit satellites as well as ground-based and airborne platforms. To make the problem tractable, most data assimilation methods resort to scalable methods derived from well-established Gaussian process-based formulation of the sequential Bayesian estimation problem. One such approach is to use approximate Kalman filters that rely on low-rank approximations of the covariance matrices. In ensemble data assimilation, the covariance is expressed in the subspace spanned by the model ensemble that evolves dynamically. Though the low-rank ensemble-based approximation allows the covariance to track non-linear dynamics with a moderate computational cost, the impact of the observations that lie outside of the subspace represented by the ensemble cannot be effectively incorporated into the assimilation analysis. One heuristic remedy widely used in numerical weather prediction is to adopt a hybrid method that uses the full-rank stationary covariance and the ensemble-based covariance. As a scalable approach to modeling the full-rank nonstationary covariance, the use of a multiresolution-based covariance is discussed.

Randomized Clustered Nystrom for Large-Scale Kernel Machines

The Nystrom method is a popular technique for generating low-rank approximations of kernel matrices that arise in many machine learning problems. The approximation quality of the Nystrom method depends crucially on the number of selected landmark points and the selection procedure. In this paper, we introduce a randomized algorithm for generating landmark points that is scalable to large high-dimensional data sets. The proposed method performs K-means clustering on low-dimensional random projections of a data set and thus leads to significant savings for high-dimensional data sets. Our theoretical results characterize the tradeoffs between accuracy and efficiency of the proposed method. Moreover, numerical experiments on classification and regression tasks demonstrate the superior performance and efficiency of our proposed method compared with existing approaches.

Computing Low-Rank Approximations of Large-Scale Matrices with the Tensor Network Randomized SVD

We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the matrix product operator (MPO) format, also called the tensor train matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensor-based alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be up to 13 times faster while achieving better accuracy. In addition, our TNrSVD algorithm also produces accurate approximations in particular cases where both ALS-SVD and MALS-SVD fail to converge. We also propose a new algorithm for the fast conversion of a sparse matrix into its corresponding MPO form, which is up to 509 times faster than the standard tensor train SVD method while achieving machine precision accuracy. The efficiency and accuracy of both algorithms are demonstrated in numerical experiments.

Drusen Segmentation From Retinal Images via Supervised Feature Learning

This paper presents a supervised feature learning method to learn discriminative and compact descriptors for drusen segmentation from retinal images. This method combines generalized low rank approximation of matrices with supervised manifold regularization to learn new features from image patches sampled from retinal images. The learned features are closely related to drusen and potentially free from information that is redundant in distinguishing drusen from background. The learned feature representations are then vectorized and used to train a support vector machine (SVM) classifier. Finally, the obtained SVM classifier is employed to classify the pixels in the test images as drusen or non-drusen. The performance of the proposed method is validated on the STARE and DRIVE databases, where it achieves an average sensitivity/specificity/accuracy of 90.03%/97.06%/96.92% and of 87.41%/94.93%/94.81%, respectively. We also experimentally compare the proposed method with the several representative state-of-the-art drusen segmentation techniques and find that it generates superior accuracy.

RTensor: An R Package for Multidimensional Array (Tensor) Unfolding, Multiplication, and Decomposition

rTensor is an R package designed to provide a common set of operations and decompositions for multidimensional arrays (tensors). We provide an S4 class that wraps around the base 'array' class and overloads familiar operations to users of 'array', and we provide additional functionality for tensor operations that are becoming more relevant in recent literature. We also provide a general unfolding operation, for which the k-mode unfolding and the matrix vectorization are special cases of. Finally, package rTensor implements common tensor decompositions such as canonical polyadic decomposition, Tucker decomposition, multilinear principal component analysis, t-singular value decomposition, as well as related matrix-based algorithms such as generalized low rank approximation of matrices and popular value decomposition.

A Probabilistic Subspace Bound with Application to Active Subspaces

Given a real symmetric positive semidefinite matrix $E$, and an approximation $S$ that is a sum of $n$ independent matrix-valued random variables, we present bounds on the relative error in $S$ due to randomization. The bounds do not depend on the matrix dimensions but only on the numerical rank (intrinsic dimension) of $E$. Our approach resembles the low-rank approximation of kernel matrices from random features, but our accuracy measures are more stringent. In the context of parameter selection based on active subspaces, where $S$ is computed via Monte Carlo sampling, we present a bound on the number of samples so that with high probability the angle between the dominant subspaces of $E$ and $S$ is less than a user-specified tolerance. This is a substantial improvement over existing work, as it is a nonasymptotic and fully explicit bound on the sampling amount $n$, and it allows the user to tune the success probability. It also suggests that Monte Carlo sampling can be efficient in the presence of many parameters, as long as the underlying function $f$ is sufficiently smooth.

Time Integration of Rank-Constrained Tucker Tensors

Dynamical low-rank approximation in the Tucker tensor format of given large time-dependent tensors and of tensor differential equations is the subject of this paper. In particular, a discrete time integration method for rank-constrained Tucker tensors is presented and analyzed. It extends the known projector-splitting integrator for dynamical low-rank approximation of matrices to Tucker tensors and is shown to inherit the same favorable properties. The integrator is based on iteratively applying the matrix projector-splitting integrator to tensor unfoldings but with inexact solution in a substep. It has the property that it reconstructs time-dependent Tucker tensors of the given rank exactly. The integrator is also shown to be robust to the presence of small singular values in the tensor unfoldings. Numerical examples with time-dependent problems from quantum physics and tensor optimization methods illustrate our theoretical results.