ULV Decomposition Research Articles

A novel model based collaborative filtering recommender system via truncated ULV decomposition

Collaborative filtering is a technique that takes into account the common characteristics of users and items in recommender systems. Matrix decompositions are one of the most used techniques in collaborative filtering based recommendation systems. Singular Value Decomposition (SVD) and Non-negative Matrix Factorization (NMF) based approaches are widely used. Although they are quite good at dealing with the scalability problem, their complexities are high. In this study, the Truncated-ULV decomposition (T-ULVD) technique was used as an alternative technique to improve the accuracy and quality of recommendations. The proposed method has been tested with Movielens 100 k, Movielens 1 M, Filmtrust, and Netflix datasets, which are widely used in recommender system researches. In order to assess the performance of the proposed model, standart metrics (MAE, RMSE, precision, recall, and F1 score) were used. It is seen that while progress was achieved in all experiments with the T-ULVD compared to the NMF, very close or better results were obtained compared to the SVD. Moreover, this study may guide T-ULVD based future studies on solving the cold-start problem and reducing the sparsity in collaborative filtering based recommender systems.

A novel image watermarking scheme using ULV decomposition

Matrix decompositions play an important role in most of watermarking techniques. Especially Singular Value Decomposition (SVD) is one of the most preferred techniques. In addition, matrix decompositions such as Non-negative Matrix Factorization (NMF), QR, LU are some of the methods used in previous studies. In this study, a new scheme using ULV Decomposition (ULVD) technique is proposed as an alternative matrix decomposition. In this scheme, which is a frequency-based technique using R-level Discrete Wavelet Transform (DWT), the scaling factor is determined adaptively according to the cover image and watermark used. Another issue to be solved in watermarking studies is the False Positive Problem (FFP). For this purpose, a control mechanism is used against the FFP in watermark embedding and watermark extracting processes. In addition, the watermarking process is carried out in any size, depending on the size of the cover image. As a result, the performed experiments show that the proposed scheme provides high imperceptibility and robustness.

Alternate Low-Rank Matrix Approximation in Latent Semantic Analysis

The latent semantic analysis (LSA) is a mathematical/statistical way of discovering hidden concepts between terms and documents or within a document collection (i.e., a large corpus of text). Each document of the corpus and terms are expressed as a vector with elements corresponding to these concepts to form a term-document matrix. Then, the LSA uses a low-rank approximation to the term-document matrix in order to remove irrelevant information, to extract more important relations, and to reduce the computational time. The irrelevant information is called as “noise” and does not have a noteworthy effect on the meaning of the document collection. This is an essential step in the LSA. The singular value decomposition (SVD) has been the main tool obtaining the low-rank approximation in the LSA. Since the document collection is dynamic (i.e., the term-document matrix is subject to repeated updates), we need to renew the approximation. This can be done via recomputing the SVD or updating the SVD. However, the computational time of recomputing or updating the SVD of the term-document matrix is very high when adding new terms and/or documents to preexisting document collection. Therefore, this issue opened the door of using other matrix decompositions for the LSA as ULV- and URV-based decompositions. This study shows that the truncated ULV decomposition (TULVD) is a good alternative to the SVD in the LSA modeling.

Block classical Gram–Schmidt-based block updating in low-rank matrix approximation

Low-rank matrix approximations have recently gained broad popularity in scientific computing areas. They are used to extract correlations and remove noise from matrix-structured data with limited loss of information. Truncated singular value decomposition (SVD) is the main tool for computing low-rank approximation. However, in applications such as latent semantic indexing where document collections are dynamic over time, i.e. the term document matrix is subject to repeated updates, SVD becomes prohibitive due to the high computational expense. Alternative decompositions have been proposed for these applications such as low-rank ULV/URV decompositions and truncated ULV decomposition. Herein, we propose a BLAS-3 compatible block updating truncated ULV decomposition algorithm based on the block classical Gram-Schmidt process. The simulation results presented show that the block update algorithm is promising.

Modifiable low‐rank approximation to a matrix

AbstractA truncated ULV decomposition (TULVD) of an m×n matrix X of rank k is a decomposition of the form X = ULVT+E, where U and V are left orthogonal matrices, L is a k×k non‐singular lower triangular matrix, and E is an error matrix. Only U,V, L, and ∥E∥F are stored, but E is not stored. We propose algorithms for updating and downdating the TULVD. To construct these modification algorithms, we also use a refinement algorithm based upon that in (SIAM J. Matrix Anal. Appl. 2005; 27(1):198–211) that reduces ∥E∥F, detects rank degeneracy, corrects it, and sharpens the approximation. Copyright © 2009 John Wiley & Sons, Ltd.

Source separation and tracking for time-varying systems

Many situations arise when it is desired to recover from a noisy mixture of data both an estimate of a propagating source signal as well as its angular position. Accordingly, the problem of source separation and tracking for time-varying systems of moving sources is studied. Using techniques borrowed from independent component analysis (ICA) and numerical linear algebra, a processing solution is formulated. The ULV decomposition (ULVD), a member of the complete orthogonal decompositions (CODs) family is used to normalize and decorrelate the data. Since the ULVD is known to stably update and downdate with time-varying data, this step is named ULVD adaptive whitening. The ICA-based EASI (equivariant adaptive separation based on independence) algorithm is adapted to provide not only separated sources but direction-of-arrival (DOA) estimates. Developed improvements to this technique are an adaptive filter method for estimating the mixing matrix from the newly estimated signals as well as a simple cross-over detection scheme. Finally, the developed solutions are tested in simulation examples using meaningful performance measures.

An efficient algorithm for rank and subspace tracking

Traditionally, the singular value decomposition (SVD) has been used in rank and subspace tracking methods. However, the SVD is computationally costly, especially when the problem is recursive in nature and the size of the matrix is large. The truncated ULV decomposition (TULV) is an alternative to the SVD. It provides a good approximation to subspaces for the data matrix and can be modified quickly to reflect changes in the data. It also reveals the rank of the matrix. This paper presents a TULV updating algorithm. The algorithm is most efficient when the matrix is of low rank. Numerical results are presented that illustrate the accuracy of the algorithm.

Prewhitening for rank-deficient noise in subspace methods for noise reduction

A fundamental issue in connection with subspace methods for noise reduction is that the covariance matrix for the noise is required to have full rank in order for the prewhitening step to be defined. However, there are important cases where this requirement is not fulfilled, e.g., when the noise has narrowband characteristics or in the case of tonal noise. We extend the concept of prewhitening to include the case when the noise covariance matrix is rank deficient, using a weighted pseudoinverse and the quotient singular value decomposition, and we show how to formulate a general rank-reduction algorithm that works also for rank-deficient noise. We also demonstrate how to formulate this algorithm by means of a quotient ULV decomposition, which allows for faster computation and updating. Finally, we apply our algorithm to a problem involving a speech signal contaminated by narrowband noise.

An Alternative Algorithm for a Sliding Window ULV Decomposition

The ULV decomposition (ULVD) is an important member of a class of rank-revealing two-sided orthogonal decompositions used to approximate the singular value decomposition (SVD). The ULVD can be modified much faster than the SVD. When modifiying the ULVD, the accurate computation of the subspaces is required in certain time varying applications in signal processing. In this paper, we present an updating algorithm which is suitable for large scaled matrices of low rank and as effective as alternatives. The algorithm runs in O(n2) time.

Subspace tracking in low-rank real-time systems

The truncated ULV decomposition (TULVD) provides good approximation to subspaces for the data matrix and can be modified quickly to reflect changes in the data. It also reveals the rank of the matrix. We develop an updating routine that is suitable for large scaled matrices of low rank. Numerical results presented that illustrate the accuracy of the algorithm.

Computing ULV Decomposition by Divide Conquer Method

The ULV decomposition (ULVD) is an important member of a class of rank-revealing two-sided orthogonal decompositions used to approximate the Singular Value Decomposition (SVD). The ULVD can be modified much faster than the SVD. The accurate computation of the subspaces is required in applications in signal processing. In this study we introduce a divide conquer ULVD algorithm.

An Alternative Algorithm for the Refinement of ULV Decompositions

The ULV decomposition (ULVD) is an important member of a class of rank-revealing two-sided orthogonal decompositions used to approximate the singular value decomposition (SVD). It is useful in applications of the SVD such as principal components where we are interested in approximating a matrix by one of lower rank. It can be updated and downdated much more quickly than an SVD. In many instances, the ULVD must be refined to improve the approximation it gives for the important right singular subspaces or to improve the matrix approximation. Present algorithms to perform this refinement require O(mn) operations if the rank of the matrix is k, where k is very close to 0 or n, but these algorithms require O(m n2) operations otherwise. Presented here is an alternative refinement algorithm that requires O(mn) operations no matter what the rank is. Our tests show that this new refinement algorithm produces similar improvement in matrix approximation and in the subspaces. We also propose slight improvements on the error bounds on subspaces and singular values computed by the ULVD.

A modified Gram–Schmidt-based downdating technique for ULV decompositions with applications to recursive TLS problems

The ULV decomposition (ULVD) is an important member of a class of rank-revealing two-sided orthogonal decompositions used to approximate the singular value decomposition (SVD). The ULVD can be updated and downdated much faster than the SVD, hence its utility in the solution of recursive total least squares (TLS) problems. However, the robust implementation of ULVD after the addition and deletion of rows (called updating and downdating, respectively) is not altogether straightforward. When updating or downdating the ULVD, the accurate computation of the subspaces necessary to solve the TLS problem is of great importance. In this paper, algorithms are given to compute simple parameters that can often show when good subspaces have been computed.

Computing Symmetric Rank-Revealing Decompositions via Triangular Factorization

We present a family of algorithms for computing symmetric rank-revealing VSV decompositions based on triangular factorization of the matrix. The VSV decomposition consists of a middle symmetric matrix that reveals the numerical rank in having three blocks with small norm, plus an orthogonal matrix whose columns span approximations to the numerical range and null space. We show that for semidefinite matrices the VSV decomposition should be computed via the ULV decomposition, while for indefinite matrices it must be computed via a URV-like decomposition that involves hypernormal rotations.

QR factorization based blind channel identification with second-order statistics

Most eigenstructure-based blind channel identification and equalization algorithms with second-order statistics need SVD or EVD of the correlation matrix of the received signal. In this paper, we address new algorithms based on QR factorization of the received signal directly without calculating the correlation matrix. This renders the QR factorization-based algorithms more robust against ill-conditioned channels, i.e., those channels with almost common zeros among the subchannels. First, we present a block algorithm that performs the QR factorization of the received data matrix as a whole. Then, a recursive algorithm is developed based on the QR factorization by updating a rank-revealing ULV decomposition. Compared with existing algorithms in the same category, our algorithms are computationally more efficient. The computation in each recursion of the recursive algorithm is on the order of O(m/sup 2/) if only equalization is required, where m is the dimension of the received signal vector. Our recursive algorithm preserves the fast convergence property of the subspace algorithms, thus converging faster than other adaptive algorithms such as the super-exponential algorithm with comparable computational complexities. Moreover, our proposed algorithms do not require noise variance estimation. Numerical simulations demonstrate the good performance of the proposed algorithms.

UTV Tools: Matlab Templates for Rank-Revealing UTV Decompositions

We describe a Matlab 5.2 package for computing and modifying certain rank-revealing decompositions that have found widespread use in signal processing and other applications. The package focuses on algorithms for URV and ULV decompositions, collectively known as UTV decompositions. We include algorithms for the ULLV decomposition, which generalizes the ULV decomposition to a pair of matrices. For completeness a few algorithms for computation of the RRQR decomposition are also included. The software in this package can be used as is, or can be considered as templates for specialized implementations on signal processors and similar dedicated hardware platforms.

An efficient rank detection procedure for modifying the ULV decomposition

The ULV decomposition (ULVD) is an important member of a class of rank-revealing two-sided orthogonal decompositions used to approximate the singular value decomposition (SVD). The problem of adding and deleting rows from the ULVD (called updating and downdating, respectively) is considered. The ULVD can be updated and downdated much faster than the SVD, hence its utility.

ULV and generalized ULV subspace tracking adaptive algorithms

Traditional adaptive filters assume that the effective rank of the input signal is the same as the input covariance matrix or the filter length N. Therefore, if the input signal lives in a subspace of dimension less than N, these filters fail to perform satisfactorily. In this paper, we present two new algorithms for adapting only in the dominant signal subspace. The first of these is a low-rank recursive-least-squares (RLS) algorithm that uses a ULV decomposition (Stewart 1992) to track and adapt in the signal subspace. The second adaptive algorithm is a subspace tracking least-mean-squares (LMS) algorithm that uses a generalized ULV (GULV) decomposition, developed in this paper, to track and adapt in subspaces corresponding to several well-conditioned singular value clusters. The algorithm also has an improved convergence speed compared with that of the LMS algorithm. Bounds on the quality of subspaces isolated using the GULV decomposition are derived, and the performance of the adaptive algorithms are analyzed.

An algorithm and a stability theory for downdating the ULV decomposition

An alternative to performing the singular value decomposition is to factor a matrixA into $$A = U\left( {\begin{array}{*{20}c} C \\ 0 \\ \end{array} } \right)V^T $$ , whereU andV are orthogonal matrices andC is a lower triangular matrix which indicates a separation between two subspaces by the size of its columns. These subspaces are denoted byV = (V 1,V 2), where the columns ofC are partitioned conformally intoC = (C 1,C 2) with ‖C 2 ‖ F ≤ e. Here e is some tolerance. In recent years, this has been called the ULV decomposition (ULVD). If the matrixA results from statistical observations, it is often desired to remove old observations, thus deleting a row fromA and its ULVD. In matrix terms, this is called a downdate. A downdating algorithm is proposed that preserves the structure in the downdated matrix $$\bar C$$ to the extent possible. Strong stability results are proven for these algorithms based upon a new perturbation theory.

Bounding the Subspaces from Rank Revealing Two-Sided Orthogonal Decompositions

The singular value decomposition (SVD) is a widely used computational tool in various applications. However, in some applications the SVD is viewed as computationally demanding or difficult to update. The rank revealing QR (RRQR) decomposition and the recently proposed URV and ULV decompositions are promising alternatives for determining the numerical rank k of an $m \times n$ matrix and approximating its fundamental numerical subspaces whenever $k \approx \min (m,n)$. In this paper we prove a posteriori bounds for assessing the quality of the subspaces obtained by two-sided orthogonal decompositions. In particular, we show that the quality of the subspaces obtained by the URV or ULV algorithm depends on the quality of the condition estimator and not on a gap condition. From our analysis we conclude that these decompositions may be more accurate alternatives to the SVD than the RRQR decomposition. Finally, we implement the algorithms in an adaptive manner, which is particularly useful for applications where the “noise” subspace must be computed, such as in signal processing or total least squares.