Low-rank Approximation Research Articles

Efficient High-Quality Clustering for Large Bipartite Graphs

A bipartite graph contains inter-set edges between two disjoint vertex sets, and is widely used to model real-world data, such as user-item purchase records, author-article publications, and biological interactions between drugs and proteins. k-Bipartite Graph Clustering (k-BGC) is to partition the target vertex set in a bipartite graph into k disjoint clusters. The clustering quality is important to the utility of k-BGC in various applications like social network analysis, recommendation systems, text mining, and bioinformatics, to name a few. Existing approaches to k-BGC either output clustering results with compromised quality due to inadequate exploitation of high-order information between vertices, or fail to handle sizable bipartite graphs with billions of edges. Motivated by this, this paper presents two efficient k-BGC solutions, HOPE and HOPE+, which achieve state-of-the-art performance on large-scale bipartite graphs. HOPE obtains high scalability and effectiveness through a new k-BGC problem formulation based on the novel notion of high-order perspective (HOP) vectors and an efficient technique for low-rank approximation of HOP vectors. HOPE+ further elevates the k-BGC performance to another level with a judicious problem transformation and a highly efficient two-stage optimization framework. Two variants, HOPE+ (FNEM) and HOPE+ (SNEM) are designed when either the Frobenius norm or spectral norm is applied in the transformation. Extensive experiments, comparing HOPE and HOPE+ against 13 competitors on 10 real-world datasets, exhibit that our solutions, especially HOPE+, are superior to existing methods in terms of result quality, while being up to orders of magnitude faster. On the largest dataset MAG with 1.1 billion edges, HOPE+ is able to produce clusters with the highest clustering accuracy within 31 minutes, which is unmatched by any existing solution for k-BGC.

Fixed-precision randomized quaternion singular value decomposition algorithm for low-rank quaternion matrix approximations

The fixed-precision randomized quaternion singular value decomposition algorithm (FPRQSVD) is presented to compute the low-rank quaternion matrix approximation. The FPRQSVD algorithm estimates the appropriate rank according to the given tolerance without a predetermined rank parameter, and computes the corresponding low-rank quaternion matrix approximation automatically. Error analysis indicates that the FPRQSVD algorithm is mathematically equivalent to the randomized quaternion singular value decomposition algorithm. Numerical experiments are given to demonstrate that the FPRQSVD algorithm is feasible and effective. Moreover, we use it to deal with the problem of color image inpainting.

A DEIM Tucker tensor cross algorithm and its application to dynamical low-rank approximation

We introduce a Tucker tensor cross approximation method that constructs a low-rank representation of a d-dimensional tensor by sparsely sampling its fibers. These fibers are selected using the discrete empirical interpolation method (DEIM). Our proposed algorithm is referred to as DEIM fiber sampling (DEIM-FS). For a rank-r approximation of an O(Nd) tensor, DEIM-FS requires access to only dNrd−1 tensor entries, a requirement that scales linearly with the tensor size along each mode. We demonstrate that DEIM-FS achieves an approximation accuracy close to the Tucker-tensor approximation obtained via higher-order singular value decomposition at a significantly reduced cost. We also present DEIM-FS (iterative) that does not require access to singular vectors of the target tensor unfolding and can be viewed as a black-box Tucker tensor algorithm. We employ DEIM-FS to reduce the computational cost associated with solving nonlinear tensor differential equations (TDEs) using dynamical low-rank approximation (DLRA). The computational cost of solving DLRA equations can become prohibitive when the exact rank of the right-hand side tensor is large. This issue arises in many TDEs, especially in cases involving non-polynomial nonlinearities, where the right-hand side tensor has full rank. This necessitates the storage and computation of tensors of size O(Nd). We show that DEIM-FS results in significant computational savings for DLRA by constructing a low-rank Tucker approximation of the right-hand side tensor on the fly. Another advantage of using DEIM-FS is to significantly simplify the implementation of DLRA equations, irrespective of the type of TDEs. We demonstrate the efficiency of the algorithm through several examples including solving high-dimensional partial differential equations.

CUR and Generalized CUR Decompositions of Quaternion Matrices and their Applications

Low-rank approximations of quaternion matrices have garnered interest across various applications, such as color images and signal processing. In this paper, we propose the CUR and generalized CUR decompositions of quaternion matrices and utilize the CUR decomposition of the quaternion matrix to solve the robust quaternion principal component analysis (RQPCA) problem. Through the discrete empirical interpolation method (DEIM) for the subset selection, we present the error analysis of the approximation of the CUR and the generalized CUR decompositions of quaternion matrices. The error bound depends on the conditioning of sampling submatrices. Next, we employ the alternating projection and adopt the CUR decomposition of the quaternion matrix to tackle the RQPCA. The non-convex algorithm runs fast and significantly reduces computational complexity. The performance advantages of our algorithms have been experimentally verified on artificial datasets. The RQCUR significantly affects color video background subtraction in the experiment.

Hyperchaotic bilateral random low-rank approximation random sequence generation method and its application on compressive ghost imaging

Hyperchaotic bilateral random low-rank approximation random sequence generation method and its application on compressive ghost imaging

Active design of diffuse acoustic fields in enclosures.

This paper presents a numerical framework for designing diffuse fields in rooms of any shape and size, driven at arbitrary frequencies. That is, we aim at overcoming the Schroeder frequency limit for generating diffuse fields in an enclosed space. We formulate the problem as a Tikhonov regularized inverse problem and propose a low-rank approximation of the spatial correlation that results in significant computational gains. Our approximation is applicable to arbitrary sets of target points and allows us to produce an optimal design at a computational cost that grows only linearly with the (potentially large) number of target points. We demonstrate the feasibility of our approach through numerical examples where we approximate diffuse fields at frequencies well below the Schroeder limit.

A Similarity Matrix Low-Rank Approximation and Inconsistency Separation Fusion Approach for Multiview Clustering

In recent years, multi-view clustering algorithms have achieved promising performance by exploiting the com-plementarity and consistency of different views. However, many multi-view spectral clustering methods only focus on the con-sistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require post-processing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multi-view clustering algorithm. Firstly, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with non-negative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the n × n Laplacian matrix in spectral clustering with the singular value decomposition of a n × c low-rank matrix to reduce the computational burden, where c ≪ n. Furthermore, due to the non-negative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multi-view clustering methods on five benchmark multi-view datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

Pass-efficient truncated UTV for low-rank approximations

Pass-efficient truncated UTV for low-rank approximations

MvTuckER: Multi-view knowledge graphs representation learning based on tensor tucker model

Recently, more and more multi-view knowledge graphs with various important attributes are being constructed and applied, such as temporal information, geolocation, commonsense knowledge, multi-language and multi-modal. Knowledge graphs (KGs) have been widely applied in question answering systems, drug discovery, information retrieval, and so on. Knowledge graph representation learning (KGRL) is emerging as the most effective approach to tasks such as knowledge graph completion, entity alignment, knowledge reasoning and knowledge query. However, most previous work on KGs and KGRL are mainly designed for traditional KGs and only consider single-view, and the KGRL methods for special multi-view KGs faces the challenges of lacking a unified framework, insufficient interactive information and underutilizing important attributes. To bridge this gap, in this paper, we proposes a multi-view knowledge graphs (e.g., temporal KGs) representation learning method based on tensor tucker model (MvTuckER), which models the multi-view KGs from tuples to a nth-order binary tensor, and adopts the tensor n-mode product to capture complete interaction between and within different views. We also introduce low-rank and sparse approximations of core tensors to balance expressivity and complexity of MvTuckER. Moreover, we theoretically illustrate that our model is fully expressive for modeling multi-view KGs, discuss the working mechanism of MvTuckER from the perspective of logical operation, and explain how to migrate and extend our method to other types of views. We conducted extensive experiments on three multi-view knowledge graphs, and obtain 4.7%, 2.7%, and 5.3% effective improvement in Hit@1 respectively, which proves that the proposed MvTuckER can achieve state-of-the-art performance.

Activation-Based Pruning of Neural Networks

We present a novel technique for pruning called activation-based pruning to effectively prune fully connected feedforward neural networks for multi-object classification. Our technique is based on the number of times each neuron is activated during model training. We compare the performance of activation-based pruning with a popular pruning method: magnitude-based pruning. Further analysis demonstrated that activation-based pruning can be considered a dimensionality reduction technique, as it leads to a sparse low-rank matrix approximation for each hidden layer of the neural network. We also demonstrate that the rank-reduced neural network generated using activation-based pruning has better accuracy than a rank-reduced network using principal component analysis. We provide empirical results to show that, after each successive pruning, the amount of reduction in the magnitude of singular values of each matrix representing the hidden layers of the network is equivalent to introducing the sum of singular values of the hidden layers as a regularization parameter to the objective function.

Accelerating the simulation of kinetic shear Alfvén waves with a dynamical low-rank approximation

We propose a dynamical low-rank algorithm for a gyrokinetic model that is used to describe strongly magnetized plasmas. The low-rank approximation is based on a decomposition into variables parallel and perpendicular to the magnetic field, as suggested by the physics of the underlying problem. We show that the resulting scheme exactly recovers the dispersion relation even with rank 1. We then perform a simulation of kinetic shear Alfvén waves and show that using the proposed dynamical low-rank algorithm a drastic reduction (multiple orders of magnitude) in both computational time and memory consumption can be achieved. We also compare the performance of robust first and second-order projector splitting, BUG (also called unconventional), and augmented BUG integrators as well as a FFT-based spectral and Lax–Wendroff discretization.

Asymptotic-Preserving and Energy Stable Dynamical Low-Rank Approximation

Asymptotic-Preserving and Energy Stable Dynamical Low-Rank Approximation

Water removal in MR spectroscopic imaging with Casorati singular value decomposition.

Water removal is one of the computational bottlenecks in the processing of high-resolution MRSI data. The purpose of this work is to propose an approach to reduce the computing time required for water removal in large MRS data. In this work, we describe a singular value decomposition-based approach that uses the partial position-time separability and the time-domain linear predictability of MRSI data to reduce the computational time required for water removal. Our approach arranges MRS signals in a Casorati matrix form, applies low-rank approximations utilizing singular value decomposition, removes residual water from the most prominent left-singular vectors, and finally reconstructs the water-free matrix using the processed left-singular vectors. We have demonstrated the effectiveness of our proposed algorithm for water removal using both simulated and in vivo data. The proposed algorithm encompasses a pip-installable tool ( https://pypi.org/project/CSVD/), available on GitHub ( https://github.com/amirshamaei/CSVD), empowering researchers to use it in future studies. Additionally, to further promote transparency and reproducibility, we provide comprehensive code for result replication. The findings of this study suggest that the proposed method is a promising alternative to existing water removal methods due to its low processing time and good performance in removing water signals.

A highly accurate prediction for heart failure disease: a new deep attentive model with guided feature ranking

Heart failure (HF) is a life-threatening disease affecting at least 64 million people worldwide. Hence, it places great stresses on patients and healthcare systems. Accordingly, providing a computerized model for HF prediction will help in enhancing diagnosis, treatment, and long-term management of HF. In this paper, we introduce a new guided attentive HF prediction approach. In this method, a sparse-guided feature ranking method is proposed. Firstly, a Gauss–Seidel strategy is applied to the preprocessed feature pool for low-rank approximation procedure with a trace-norm regularization. The resultant sparse attributes, after a Spearman ranking elimination, are employed to guide the original feature pool through linear translation-variant model. Then, a fast Newton-based method is employed for a non-negative matrix factorization for the guided feature pool. The resultant bases of the factorization process are finally utilized in the adopted deep attentive predictive model. For the final prediction stage, instead of the commonly used machine learning approaches, we introduce an attentive-based classifier. It employs sequential attention to choose the most proper salient features for efficient interpretability and learning process. For the evaluation of the proposed HF prediction model, three different datasets are employed, i.e., UCI, Faisalabad, and Framingham datasets. Compared to state-of-the-art techniques, the proposed approach outperforms their performance on all datasets with even small feature sizes. With only four feature bases, the proposed method achieves an average accuracy of 98%, while, with full feature bases, full accuracy is gained.

Nonconvex Robust High-Order Tensor Completion Using Randomized Low-Rank Approximation.

Within the tensor singular value decomposition (T-SVD) framework, existing robust low-rank tensor completion approaches have made great achievements in various areas of science and engineering. Nevertheless, these methods involve the T-SVD based low-rank approximation, which suffers from high computational costs when dealing with large-scale tensor data. Moreover, most of them are only applicable to third-order tensors. Against these issues, in this article, two efficient low-rank tensor approximation approaches fusing random projection techniques are first devised under the order-d ( d ≥ 3 ) T-SVD framework. Theoretical results on error bounds for the proposed randomized algorithms are provided. On this basis, we then further investigate the robust high-order tensor completion problem, in which a double nonconvex model along with its corresponding fast optimization algorithms with convergence guarantees are developed. Experimental results on large-scale synthetic and real tensor data illustrate that the proposed method outperforms other state-of-the-art approaches in terms of both computational efficiency and estimated precision.

Tensorized Multi-View Low-Rank Approximation Based Robust Hand-Print Recognition.

Since hand-print recognition, i.e., palmprint, finger-knuckle-print (FKP), and hand-vein, have significant superiority in user convenience and hygiene, it has attracted greater enthusiasm from researchers. Seeking to handle the long-standing interference factors, i.e., noise, rotation, shadow, in hand-print images, multi-view hand-print representation has been proposed to enhance the feature expression by exploiting multiple characteristics from diverse views. However, the existing methods usually ignore the high-order correlations between different views or fuse very limited types of features. To tackle these issues, in this paper, we present a novel tensorized multi-view low-rank approximation based robust hand-print recognition method (TMLA_RHR), which can dexterously manipulate the multi-view hand-print features to produce a high-compact feature representation. To achieve this goal, we formulate TMLA_RHR by two key components, i.e., aligned structure regression loss and tensorized low-rank approximation, in a joint learning model. Specifically, we treat the low-rank representation matrices of different views as a tensor, which is regularized with a low-rank constraint. It models the across information between different views and reduces the redundancy of the learned sub-space representations. Experimental results on eight real-world hand-print databases prove the superiority of the proposed method in comparison with other state-of-the-art related works.

Ratio-Based Multitemporal SAR Image Despeckling With Low-Rank Approximation

Ratio-Based Multitemporal SAR Image Despeckling With Low-Rank Approximation

Quaternion Chromaticity Contrast Preserving Decolorization Method Based on Adaptive Singular Value Weighting

Color image decolorization can not only simplify the complexity of image processing and analysis, improving computational efficiency, but also help to preserve the key information of the image, enhance visual effects, and meet various practical application requirements. However, with existing decolorization methods it is difficult to simultaneously maintain the local detail features and global smooth features of the image. To address this shortcoming, this paper utilizes singular value decomposition to obtain the hierarchical local features of the image and utilizes quaternion theory to overcome the limitation of existing color image processing methods that ignore the correlation between the three channels of the color image. Based on this, we propose a singular value adaptive weighted fusion quaternion chromaticity contrast preserving decolorization method. This method utilizes the low-rank matrix approximation principle to design a singular value adaptive weighted fusion strategy for the three channels of the color image and implements image decolorization based on singular value adaptive weighting. To address the deficiency of the decolorization result obtained in this step, which cannot maintain global smoothness characteristics well, a contrast preserving decolorization algorithm based on quaternion chromaticity distance is further proposed, and the global weighting strategy obtained by this algorithm is integrated into the image decolorization based on singular value adaptive weighting. The experimental results show that the decolorization method proposed in this paper achieves excellent results in both subjective visual perception and objective evaluation metrics.

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning

On Exploring the Potential of Quantum Auto-Encoder for Learning Quantum Systems.

The frequent interactions between quantum computing and machine learning revolutionize both fields. One prototypical achievement is the quantum auto-encoder (QAE), as the leading strategy to relieve the curse of dimensionality ubiquitous in the quantum world. Despite its attractive capabilities, practical applications of QAE have yet largely unexplored. To narrow this knowledge gap, here, we devise three effective QAE-based learning protocols to address three classically computational hard learning problems when learning quantum systems, which are low-rank state fidelity estimation, quantum Fisher information (QFI) estimation, and Gibbs state preparation. Attributed to the versatility of QAE, our proposals can be readily executed on near-term quantum machines. Besides, we analyze the error bounds of the trained protocols and showcase the necessary conditions to provide practical utility from the perspective of complexity theory. We conduct numerical simulations to confirm the effectiveness of the proposed three protocols. This work sheds new light on developing advanced quantum learning algorithms to accomplish hard quantum physics and quantum information processing tasks.