Generalized Low Rank Approximations Of Matrices Research Articles

Speckle attenuation for optical coherence tomography images using the generalized low rank approximations of matrices.

A frequently used technology in medical diagnosis is optical coherence tomography (OCT). However, coherent noise, also known as speckle noise, has the potential to severely reduce the quality of OCT images, which would be detrimental to the use of OCT images for disease diagnosis. In this paper, a despeckling method is proposed to effectively reduce the speckle noise in OCT images using the generalized low rank approximations of matrices (GLRAM). Specifically, the Manhattan distance (MD)-based block matching method is first used to find nonlocal similar blocks for the reference one. The left and right projection matrices shared by these image blocks are then found using the GLRAM approach, and an adaptive method based on asymptotic matrix reconstruction is proposed to determine how many eigenvectors are present in the left and right projection matrices. Finally, all the reconstructed image blocks are aggregated to create the despeckled OCT image. In addition, an edge-guided adaptive back-projection strategy is used to improve the despeckling performance of the proposed method. Experiments with synthetic and real OCT images show that the presented method performs well in both objective measurements and visual evaluation.

Ensemble learning-based computational imaging method for electrical capacitance tomography

Electrical capacitance tomography (ECT) is a potential measurement technology for industrial process monitoring, but its applicability is generally limited by low-quality tomographic images. Boosting the performance of inverse computing imaging algorithms is the key to improving the reconstruction quality (RQ). Common regularization iteration imaging methods with analytical prior regularizers are less flexible in dealing with actual reconstruction tasks, leading to large reconstruction errors. To address the challenge, this study proposes a new imaging method, including reconstruction model and optimizer. The data-driven regularizer from a new ensemble learning model and the analytical prior regularizer with the focus on the sparsity of imaging objects are combined into a new optimization model for imaging. In the proposed ensemble learning model, the generalized low rank approximations of matrices (GLRAM) method is used to carry out the dimensionality reduction for decreasing the redundancy of the input data and improving the diversity, the extreme learning machine (ELM) serves as a base learner and the nuclear norm based matrix regression (NNMR) method is developed to aggregate the ensemble of solutions. The singular value thresholding method (SVTM) and the fast iterative shrinkage-thresholding algorithm (FISTA) are inserted into the split Bregman method (SBM) to generate a powerful optimizer for the built computational model. Its comparison to other competing methods through numerical experiments on typical imaging targets demonstrates that the developed algorithm reduces reconstruction error and achieves much more improvement in imaging quality and robustness.

Robust Generalized Low Rank Approximations of Matrices.

In recent years, the intrinsic low rank structure of some datasets has been extensively exploited to reduce dimensionality, remove noise and complete the missing entries. As a well-known technique for dimensionality reduction and data compression, Generalized Low Rank Approximations of Matrices (GLRAM) claims its superiority on computation time and compression ratio over the SVD. However, GLRAM is very sensitive to sparse large noise or outliers and its robust version does not have been explored or solved yet. To address this problem, this paper proposes a robust method for GLRAM, named Robust GLRAM (RGLRAM). We first formulate RGLRAM as an l 1-norm optimization problem which minimizes the l 1-norm of the approximation errors. Secondly, we apply the technique of Augmented Lagrange Multipliers (ALM) to solve this l 1-norm minimization problem and derive a corresponding iterative scheme. Then the weak convergence of the proposed algorithm is discussed under mild conditions. Next, we investigate a special case of RGLRAM and extend RGLRAM to a general tensor case. Finally, the extensive experiments on synthetic data show that it is possible for RGLRAM to exactly recover both the low rank and the sparse components while it may be difficult for previous state-of-the-art algorithms. We also discuss three issues on RGLRAM: the sensitivity to initialization, the generalization ability and the relationship between the running time and the size/number of matrices. Moreover, the experimental results on images of faces with large corruptions illustrate that RGLRAM obtains the best denoising and compression performance than other methods.

Inexact and incremental bilinear Lanczos components algorithms for high dimensionality reduction and image reconstruction

Recently, matrix-based methods have gained wide attentions in pattern recognition and machine learning communities. The generalized low rank approximations of matrices (GLRAM) and the bilinear Lanczos components (BLC) algorithm are two popular algorithms that treat data as the native two-dimensional matrix patterns. However, these two algorithms often require heavy computation time and memory space in practice, especially for large scale problems. In this paper, we propose inexact and incremental bilinear Lanczos components algorithms for high dimensionality reduction and image reconstruction. We first introduce the thick-restarting strategy to the BLC algorithm, and present a thick-restarted Lanczos components (TRBLC) algorithm. In this algorithm, we use the Ritz vectors as approximations to dominant eigenvectors instead of the Lanczos vectors. In our implementation, the iterative matrices are neither formed nor stored explicitly, thanks to the characteristic of the Lanczos procedure. Then, we explore the relationship between the reconstruction error and the accuracy of the Ritz vectors, so that the computational complexities of eigenpairs can be reduced significantly. As a result, we propose an inexact thick-restarted Lanczos components (Inex-TRBLC) algorithm. Moreover, we investigate the problem of incremental generalized low rank approximations of matrices, and propose an incremental and inexact TRBLC (Incr-TRBLC) algorithm. Numerical experiments illustrate the superiority of the new algorithms over the GLRAM algorithm and its variations, as well as the BLC algorithm for some real-world image reconstruction and face recognition problems.

A QUINTET SINGULAR VALUE DECOMPOSITION THROUGH EMPIRICAL MODE DECOMPOSITIONS

A particular quintet singular valued decomposition (Quintet-SVD) is introduced in this paper via empirical mode decompositions (EMDs). The Quintet-SVD results in four specific orthogonal matrices with a diagonal matrix of singular values. Furthermore, this paper shows relationships between the Quintet-SVD and traditional SVD, generalized low rank approximations of matrices (GLRAM) of one single matrix, and EMDs. One application of the Quintet-SVD for speech enhancement is shown and compared with an application of traditional SVD.

Adaptation of Hidden Markov Models Using Model-as-Matrix Representation

In this paper, we describe basis-based speaker adaptation techniques using the matrix representation of training models. Bases are obtained from training models by decomposition techniques for matrix-variate objects: two-dimensional principal component analysis (2DPCA) and generalized low rank approximations of matrices (GLRAM). The motivation for using matrix representation is that the sample covariance matrix of training models can be more accurately computed and the speaker weight becomes a matrix. Speaker adaptation equations are derived in the maximum-likelihood (ML) framework, and the adaptation equations can be solved using the maximum-likelihood linear regression technique. Additionally, novel applications of probabilistic 2DPCA and GLRAM to speaker adaptation are presented. From the probabilistic 2DPCA/GLRAM of training models, speaker adaptation equations are formulated in the maximum a posteriori (MAP) framework. The adaptation equations can be solved using the MAP linear regression technique. In the isolated-word experiments, the matrix representation-based methods in the ML and MAP frameworks outperformed maximum-likelihood linear regression adaptation, MAP adaptation, eigenvoice, and probabilistic PCA-based model for adaptation data longer than 20 s. Furthermore, the adaptation methods using probabilistic 2DPCA/GLRAM showed additional performance improvement over the adaptation methods using 2DPCA/GLRAM for small amounts of adaptation data.

Bilinear Lanczos components for fast dimensionality reduction and feature extraction

Matrix-based methods such as generalized low rank approximations of matrices (GLRAM) have gained wide attention from researchers in pattern recognition and machine learning communities. In this paper, a novel concept of bilinear Lanczos components (BLC) is introduced to approximate the projection vectors obtained from eigen-based methods without explicit computing eigenvectors of the matrix. This new method sequentially reduces the reconstruction error for a Frobenius-norm based optimization criterion, and the resulting approximation performance is thus improved during successive iterations. In addition, a theoretical clue for selecting suitable dimensionality parameters without losing classification information is presented in this paper. The BLC approach realizes dimensionality reduction and feature extraction by using a small number of Lanczos components. Extensive experiments on face recognition and image classification are conducted to evaluate the efficiency and effectiveness of the proposed algorithm. Results show that the new approach is competitive with the state-of-the-art methods, while it has a much lower training cost.

Generalized Low-Rank Approximations of Matrices Revisited

Compared to singular value decomposition (SVD), generalized low-rank approximations of matrices (GLRAM) can consume less computation time, obtain higher compression ratio, and yield competitive classification performance. GLRAM has been successfully applied to applications such as image compression and retrieval, and quite a few extensions have been successively proposed. However, in literature, some basic properties and crucial problems with regard to GLRAM have not been explored or solved yet. For this sake, we revisit GLRAM in this paper. First, we reveal such a close relationship between GLRAM and SVD that GLRAM's objective function is identical to SVD's objective function except the imposed constraints. Second, we derive a lower bound of GLRAM's objective function, and discuss when the lower bound can be touched. Moreover, from the viewpoint of minimizing the lower bound, we answer one open problem raised by Ye (Machine Learning, 2005), i.e., a theoretical justification of the experimental phenomenon that, under given number of reduced dimension, the lowest reconstruction error is obtained when the left and right transformations have equal number of columns. Third, we explore when and why GLRAM can perform well in terms of compression, which is a fundamental problem concerning the usability of GLRAM.

A simplified GLRAM algorithm for face recognition

In this paper we propose a new face recognition method based on the generalized low rank approximations of matrices (GLRAM). First, we investigate the GLRAM and its associated coupled subspace analysis and then propose a new simplified algorithm, which is named as SGLRAM aiming at deriving the projection matrices for GLRAM. We implement all these algorithms (GLRAM SGLRAM) for face recognition on the ORL and YaleB databases and the experiments show that the SGLRAM can produce comparable high performance compared to the approached of two-dimensional principal component analysis (2DPCA) and GLRAM. However, it will cost much less time than the GLRAM in training and save more space than the 2DPCA in testing.

The theoretical analysis of GLRAM and its applications

Matrix-based methods such as two-dimensional principal component analysis (2DPCA) and generalized low rank approximations of matrices (GLRAM) have gained wide attention from researchers due to their computational efficiency. In this paper, we propose a non-iterative algorithm for GLRAM. Firstly, the optimal property of GLRAM is revealed, which is closely related to PCA. Moreover, it also shows that the reconstruction error of GLRAM is not smaller than that of PCA when considering the same dimensionality. Secondly, a non-iterative algorithm for GLRAM is derived. And the proposed method obtains smaller reconstruction error than 2DPCA or GLRAM. Finally, experimental results on face images and handwritten numeral characters show that the proposed method can achieve competitive results with some existing methods such as 2DPCA and PCA in terms of the classification performance or the reconstruction error.

An analytical algorithm for generalized low-rank approximations of matrices

The algorithm for generalized low-rank approximations of matrices (GLRAM) has been developed recently. In this paper, the optimality property of GLRAM is revealed. Accordingly, an analytical method for GLRAM is proposed. The proposed method is non-iterative. Moreover, the relationship between 2DPCA and GLRAM is shown.