Low-Rank Matrix and Tensor Decomposition Using Randomized Two-Sided Subspace Iteration With Application to Video Reconstruction

Speech Denoising via Low-Rank and Sparse Matrix Decomposition

Randomized two-sided subspace iteration for low-rank matrix and tensor decomposition

High-dimensional seismic data are inevitably affected by missing traces and random noise, both of which negatively affect subsequent processing and interpretation. Methods based on the low-rank matrix or tensor decomposition are used to reconstruct seismic data under the assumption that noise-free and complete data are low rank in the frequency-space ( f- x) domain. However, the presence of missing traces and random noise increases the rank of the data matrix or tensor. We use the Vandermonde structure of exponential bases of linear events in the f- x domain, which is in accordance with the plane-wave hypothesis, and develop the Vandermonde constrained tensor CANDECOMP/PARAFAC decomposition (VCPD) method for high-dimensional seismic data reconstruction. The modified alternating direction method of the multipliers algorithm is developed, and the rank-1 matrix approximation is adopted to tackle this constrained optimization problem efficiently. Numerical examples of the synthetic data and field data indicate that the proposed VCPD method is effective and efficient for reconstructing the seismic data in the case of randomly missing with a poor signal-to-noise ratio. In addition, the VCPD method is insensitive to the selection of the tensor rank and more efficient than the related state-of-the-art methods, which should lead to wide applicability.

Low-Rank Matrix and Tensor–Based Reconstruction

Sparse and low-rank matrix decomposition (SLMD) tries to decompose a matrix into a low-rank matrix and a sparse matrix, it has recently attached much research interest and has good applications in many fields. An infrared image with small target usually has slowly transitional background, it can be seen as the sum of low-rank background component and sparse target component. So by SLMD, the sparse target component can be separated from the infrared image and then be used for small infrared target detection (SITD). The augmented Lagrange method, which is currently the most efficient algorithm used for solving SLMD, was applied in this paper for SITD, some parameters were analyzed and adjusted for SITD. Experimental results show our algorithm is fast and reliable.

Bayesian inference for adaptive low rank and sparse matrix estimation

In this paper, a randomized algorithm for high dimensional low rank plus sparse matrix decomposition is proposed. Existing decomposition methods are not scalable to big data since they rely on using the whole data to extract the low-rank/sparse components, and are based on an optimization problem whose dimensionality is equal to the dimension of the given data. We reformulate the low rank plus sparse matrix decomposition problem as a column-row subspace learning problem. It is shown that when the column/row subspace of the low rank matrix is incoherent with the standard basis, the column/row subspace can be obtained from a small random subset of the columns/rows of the given data matrix. Thus, the high dimensional matrix decomposition problem is converted to a subspace learning problem, which is a low-dimensional optimization problem, and the proposed method uses a small random subset of the data rather than the whole big data matrix. In the provided analysis, it is shown that the sufficient number of randomly sampled columns/rows scales linearly with the rank and the coherency parameter of the low rank component.

Temporal random noise (TRN) in uncooled infrared detectors significantly degrades image quality. Existing denoising techniques primarily address fixed-pattern noise (FPN) and do not effectively mitigate TRN. Therefore, a novel TRN denoising approach based on total variation regularization and low-rank tensor decomposition is proposed. This method effectively suppresses temporal noise by introducing twisted tensors in both horizontal and vertical directions while preserving spatial information in diverse orientations to protect image details and textures. Additionally, the Laplacian operator-based bidirectional twisted tensor truncated nuclear norm (bt-LPTNN), is proposed, which is a norm that automatically assigns weights to different singular values based on their importance. Furthermore, a weighted spatiotemporal total variation regularization method for nonconvex tensor approximation is employed to preserve scene details. To recover spatial domain information lost during tensor estimation, robust principal component analysis is employed, and spatial information is extracted from the noise tensor. The proposed model, bt-LPTVTD, is solved using an augmented Lagrange multiplier algorithm, which outperforms several state-of-the-art algorithms. Compared to some of the latest algorithms, bt-LPTVTD demonstrates improvements across all evaluation metrics. Extensive experiments conducted using complex scenes underscore the strong adaptability and robustness of our algorithm.

Lots of data is generated around us in today's big data age. Much of this data is time-varying or dynamic, such as the social network connections that change across time or dynamic functional brain connectivity networks constructed across multiple subjects. In these dynamic (time-varying) networks, it is important to reduce the large amount of data into a few meaningful descriptors. One way to achieve this goal is to detect change points or anomalies in the connectivity patterns across time. Recently, there has been an interest in describing the time-varying network activity as a tensor and detecting the anomalies in terms of the changes in the subspaces of the tensor along each mode [1], [2]. However, the current approaches to tensor decomposition are not robust to non-Gaussian noise, outliers, and corruption in the data. For this reason, a robust low-rank tensor recovery algorithm similar to robust principal components analysis (RPCA) has been recently proposed. In this paper, we employ higher order robust PCA (HoRPCA) for tracking dynamic networks in time and detecting anomalies using a subspace distance measure. The proposed approach assumes that most real life networks are low-rank in nature and considers a low-rank plus sparse tensor decomposition at each time point. The subspaces corresponding to each mode and each time point are described through a projection operator and the subspace distance is quantified through a Hausdorff distance measure. The proposed framework is evaluated on both simulated networks and dynamic functional connectivity brain networks.

As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association matrix. Although the collaborative cluster structure among basic clusterings can be well discovered by ensemble clustering, most advanced ensemble clustering utilizes the self-representation strategy with the constraint of low-rank to explore a shared consensus representation matrix in multiple views. However, they still encounter two challenges: (1) high computational cost caused by both the matrix inversion operation and singular value decomposition of large-scale square matrices; (2) less considerable attention on high-order correlation attributed to the pursue of the two-dimensional pair-wise relationship matrix. In this article, based on low-rank and sparse decomposition from both matrix and tensor perspectives, we propose two novel multi-view ensemble clustering methods, which tangibly decrease computational complexity. Specifically, our first method utilizes low-rank and sparse matrix decomposition to learn one common co-association matrix, while our last method constructs all co-association matrices into one third-order tensor to investigate the high-order correlation among multiple views by low-rank and sparse tensor decomposition. We adopt the alternating direction method of multipliers to solve two convex models by dividing them into several subproblems with closed-form solution. Experimental results on ten real-world datasets prove the effectiveness and efficiency of the proposed two multi-view ensemble clustering methods by comparing them with other advanced ensemble clustering methods.

Seismic weak responses from subsurface small-scale geologic discontinuities or inhomogeneities are encoded in 3D diffractions. Separating weak diffractions from a strong reflection background is a difficult problem for diffraction imaging, especially for the 3D case when they are tangent to or interfering with each other. Most conventional diffraction separation methods ignore the azimuth discrepancy between reflections and diffractions when suppressing reflections. In fact, the reflections associated with a specific pair of azimuth-dip angle possess sparse characteristics, and the diffractions adhering to Huygens’ principle behave as low-rank components. Therefore, we have developed a 3D low-rank diffraction imaging method that uses the Mahalanobis-based low-rank and sparse matrix decomposition method for separating and imaging 3D diffractions in the azimuth-dip angle image matrix. The advantages of our 3D diffraction imaging method not only includes the handling of interfering events but also includes ensuring a better protection of weak diffractions. The numerical experiment illustrates the good performance of our method in imaging small-scale discontinuities and inhomogeneities. The field data application of carbonate reservoirs further confirms its potential value in resolving the masked small-scale cavities that can provide storage spaces and a migration pathway for petroleum.

A multi-objective memetic algorithm for low rank and sparse matrix decomposition

Audio-visual recognition systems rely on efficient feature extraction. Many spatio-temporal interest point detectors for visual feature extraction are either too sparse, leading to loss of information, or too dense resulting in noisy and redundant information. Furthermore, interest point detectors designed for a controlled environment can be affected by camera motion. In this paper, a salient spatio-temporal interest point detector is proposed based on a low-rank and group-sparse matrix approximation. The detector handles the camera motion through a short-window video stabilization. The multimodal audio-visual features from multiple descriptors are represented by a super descriptor, from which a compact set of features is extracted through a tensor decomposition and feature selection. This tensor decomposition retains the spatiotemporal structure among features obtained from multiple descriptors. Experimental validation is conducted using two benchmark human interaction recognition datasets: TVHID and Parliament. Experimental results are presented which show that the proposed approach outperforms many state-ofthe-art methods, achieving classification rates of 74.7% and 88.5% on the TVHID and Parliament datasets, respectively.

Today, digital data is accumulated at a faster than ever speed in science, engineering, biomedicine, and real-world sensing. The ubiquitous phenomenon of massive data and sparse information imposes considerable challenges in data mining research. In this paper, we propose a theoretical framework, Exemplar-based low-rank sparse matrix decomposition (EMD), to cluster large-scale datasets. Capitalizing on recent advances in matrix approximation and decomposition, EMD can partition datasets with large dimensions and scalable sizes efficiently. Specifically, given a data matrix, EMD first computes a representative data subspace and a near-optimal low-rank approximation. Then, the cluster centroids and indicators are obtained through matrix decomposition, in which we require that the cluster centroids lie within the representative data subspace. By selecting the representative exemplars, we obtain a compact "sketch"of the data. This makes the clustering highly efficient and robust to noise. In addition, the clustering results are sparse and easy for interpretation. From a theoretical perspective, we prove the correctness and convergence of the EMD algorithm, and provide detailed analysis on its efficiency, including running time and spatial requirements. Through extensive experiments performed on both synthetic and real datasets, we demonstrate the performance of EMD for clustering large-scale data.

Hyperspectral image (HSI) is usually corrupted by various types of noise, including Gaussian
\nnoise, impulse noise, stripes, deadlines, and so on. Recently, sparse and low-rank matrix decomposition
\n(SLRMD) has demonstrated to be an effective tool in HSI denoising. However, the matrix-based SLRMD
\ntechnique cannot fully take the advantage of spatial and spectral information in a 3-D HSI data. In this paper,
\na novel group sparse and low-rank tensor decomposition (GSLRTD) method is proposed to remove different
\nkinds of noise in HSI, while still well preserving spectral and spatial characteristics. Since a clean 3-D HSI
\ndata can be regarded as a 3-D tensor, the proposed GSLRTD method formulates a HSI recovery problem
\ninto a sparse and low-rank tensor decomposition framework. Specifically, the HSI is first divided into a set
\nof overlapping 3-D tensor cubes, which are then clustered into groups by K-means algorithm. Then, each
\ngroup contains similar tensor cubes, which can be constructed as a new tensor by unfolding these similar
\ntensors into a set of matrices and stacking them. Finally, the SLRTD model is introduced to generate noisefree
\nestimation for each group tensor. By aggregating all reconstructed group tensors, we can reconstruct a
\ndenoised HSI. Experiments on both simulated and real HSI data sets demonstrate the effectiveness of the
\nproposed method.