Low-rank Approximation Algorithm Research Articles

A Multilinear Nyström Algorithm for Low-Rank Approximation of Tensors in Tucker Format

A Multilinear Nyström Algorithm for Low-Rank Approximation of Tensors in Tucker Format

Improved Low-Rank Matrix Approximation in Multivariate Case

The low-rank matrix approximation (LRMA) algorithm is an important method in signal processing. This algorithm is usually used in many tasks such as the matrix estimation and machine learning algorithms. In the past, many convex regularizers, such as the absolute-value norm, were presented for LRMA. Recently, many works show that LRMA on many nonconvex regularizers, such as the quadratic and logarithmic regularizers, can give better efficiency than the convex regularizer. Furthermore, many traditional works present LRMA in the univariate case, and do not consider in the multivariate case. Therefore, LRMA in the multivariate case on the nonconvex regularizer is presented in this work. Note that our proposed method is based on the singular value decomposition (SVD) algorithm. The relationship between singular values is considered by this multivariate case. We also present the novel nonconvex regularizer for LRMA. The simple solution for our method can be obtained from this regularizer. In many random signals, our proposed method is evaluated with the state-of-the-art algorithms. Experimental results show that the best results can be obtained from the proposed method.

An Improved Frequent Directions Algorithm for Low-Rank Approximation via Block Krylov Iteration.

Frequent directions (FDs), as a deterministic matrix sketching technique, have been proposed for tackling low-rank approximation problems. This method has a high degree of accuracy and practicality but experiences a lot of computational cost for large-scale data. Several recent works on the randomized version of FDs greatly improve the computational efficiency but unfortunately sacrifice some precision. To remedy such an issue, this article aims to find a more accurate projection subspace to further improve the efficiency and effectiveness of the existing FDs' techniques. Specifically, by utilizing the power of the block Krylov iteration and random projection technique, this article presents a fast and accurate FDs algorithm named r-BKIFD. The rigorous theoretical analysis shows that the proposed r-BKIFD has a comparable error bound with original FDs, and the approximation error can be arbitrarily small when the number of iterations is chosen appropriately. Extensive experimental results on both synthetic and real data further demonstrate the superiority of r-BKIFD over several popular FDs algorithms both in terms of computational efficiency and accuracy.

Approximation in the extended functional tensor train format

This work proposes the extended functional tensor train (EFTT) format for compressing and working with multivariate functions on tensor product domains. Our compression algorithm combines tensorized Chebyshev interpolation with a low-rank approximation algorithm that is entirely based on function evaluations. Compared to existing methods based on the functional tensor train format, the adaptivity of our approach often results in reducing the required storage, sometimes considerably, while achieving the same accuracy. In particular, we reduce the number of function evaluations required to achieve a prescribed accuracy by up to over 96%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$96\\%$$\\end{document} compared to the algorithm from Gorodetsky et al. (Comput. Methods Appl. Mech. Eng. 347, 59–84 2019).

Forecasting individual bids in real electricity markets through machine learning framework

With the increasing uncertainty caused by the complexity of the world’s energy environment and the increasing penetration rate of renewable energy, it is significant to estimate the future operation of power markets in advance. Forecasting individual bids in spot electricity markets is a promising new method for achieving so, but it has not been fully studied due to the difficulty of forecasting a bid function. The idealization of existing optimization-based models decreases their practical effects in real markets. Thus, we propose a scalable forecasting framework that incorporates several customized state-of-art machine learning methods according to the characteristics of the bidding data. First, several low-rank approximation algorithms are customized to encode the high-dimensional bidding curves into low-dimensional feature spaces and reconstruct them from the predicted feature space. Second, a transformer-based multidimensional time series prediction algorithm is proposed to predict the bidding feature based on both related factors and historical bidding records. To appropriately evaluate the performances of the forecasting methods, we introduce a dynamic criterion based on the economic implications of bids. The comprehensive framework is tested based on actual market data from the Australian national electricity market, and in the empirical example, the feasibility and effectiveness of the proposed framework are demonstrated.

Denoising for airborne transient electromagnetic data using noise-whitening-based weighted nuclear norm minimization

Abstract Recovering anomalous information covered under noise in late gates can enhance airborne transient electromagnetic (ATEM) detection. Conventional denoising mainly comprises filtering and gate correlation-based decomposition algorithms; the former fails to extract anomalies contaminated by noise and the latter relies on the correlation between gates, which may yield false late gate anomalies caused by early large-amplitude anomalies in early gates. In ATEM profiles, the correlation between anomalies in adjacent gates makes the anomalies to be measured to have low-rank characteristics relative to the noise-contaminated profiles; the noise is uniformly distributed in the profiles, which have nonlocal self-similarity. Therefore, the low-rank matrix approximation algorithm is applicable to ATEM data denoising. In this study, an algorithm—noise-whitening-based weighted nuclear norm minimization (NW-WNNM)—is designed to remove ATEM profile noise. First, we analyze the influence of patch size in block matching on anomalous and noisy patches and estimate the profile patch size adaptively. Then, we combine the estimation of noise variance in weighted nuclear norm minimization (WNNM) with the noise whitening of similar patch matrices to reduce the noise interference on the nuclear norm and add a whitening factor in the weight vector to make the soft-thresholding function applicable to the low-rank reconstruction of the whitened matrix. By analyzing the reconstructed low-rank matrix and its feature distribution, compared with WNNM, NW-WNNM can detect the feature information more accurately and eliminate the influence of noise on the nuclear norm. Simulation and field profile results indicate that NW-WNNM is superior to comparison denoising methods.

Randomized Algorithms for Rounding in the Tensor-Train Format

The tensor-train (TT) format is a highly compact low-rank representation for high-dimensional tensors. TT is particularly useful when representing approximations to the solutions of certain types of parametrized partial differential equations. For many of these problems, computing the solution explicitly would require an infeasible amount of memory and computational time. While the TT format makes these problems tractable, iterative techniques for solving the PDEs must be adapted to perform arithmetic while maintaining the implicit structure. The fundamental operation used to maintain feasible memory and computational time is called rounding, which truncates the internal ranks of a tensor already in TT format. We propose several randomized algorithms for this task that are generalizations of randomized low-rank matrix approximation algorithms and provide significant reduction in computation compared to deterministic TT-rounding algorithms. Randomization is particularly effective in the case of rounding a sum of TT-tensors (where we observe speedup), which is the bottleneck computation in the adaptation of GMRES to vectors in TT format. We present the randomized algorithms and compare their empirical accuracy and computational time with deterministic alternatives.

Infrared Small and Moving Target Detection on Account of the Minimization of Non-Convex Spatial-Temporal Tensor Low-Rank Approximation under the Complex Background

Infrared point-target detection is one of the key technologies in infrared guidance systems. Due to the long observation distance, the point target is often submerged in the background clutter and large noise in the process of atmospheric transmission and scattering, and the signal-to-noise ratio is low. On the other hand, the target in the image appears in the form of fuzzy points, so that the target has no obvious features and texture information. Therefore, scholars have proposed many object detection methods for dimming infrared images, which has become a hot research topic on account of the flow-rank model based on the image patch. However, the result has a high false alarm rate because the most low-rank models based on the image patch do not consider the spatial-temporal characteristics of the infrared sequences. Therefore, we introduce 3D total variation (3D-TV) to regularize the foreground on account of the non-convex rank approximation minimization method, so as to consider the spatial-temporal continuity of the target and effectively suppress the interference caused by dynamic background and target movement on the foreground extraction. Finally, this paper proposes the minimization of the non-convex spatial-temporal tensor low-rank approximation algorithm (MNSTLA) by studying the related algorithms of the point infrared target detection, and the experimental results show strong robustness and a low false alarm rate for the proposed method compared with other advanced algorithms, such as NARM, RIPT, and WSNMSTIPT.

Matrix Completion via Successive Low-rank Matrix Approximation

In this paper, a successive low-rank matrix approximation algorithm is presented for the matrix completion (MC) based on hard thresholding method, which approximate the optimal low-rank matrix from rank-one matrix step by step. The algorithm enables the distance between the matrix with the observed elements and the projection on low-rank manifold to be minimum. The optimal low-rank matrix with observed elements is obtained when the distance is zero. In theory, convergence and convergent error of the new algorithm are analyzed in detail. Furthermore, some numerical experiments show that the algorithm is more effective in CPU time and precision than the orthogonal rank-one matrix pursuit(OR1MP) algorithm and the augmented Lagrange multiplier (ALM) method when the sampling rate is low.

Static and incremental robust kernel factorization embedding graph regularization supporting ill-conditioned industrial data recovery

Low-rank approximation algorithms aim to utilize convex nuclear norm constraint of linear matrices to recover ill-conditioned entries caused by multi-sampling rates, sensor drop-out. However, these existing algorithms are often limited in solving high-dimensionality and rank minimization relaxation. In this paper, a robust kernel factorization embedding graph regularization method is developed to statically impute missing measurements. Specifically, the implicit high-dimensional feature space of ill-conditioned data is factorized by kernel sparse dictionary. Then, a robust sparse-norm and graph regularization constraints are performed in the objective function to ensure the consistency of the spatial information. For the optimization of the parameters involved in the model, a distributed adaptive proximal Newton gradient descent learning strategy is proposed to accelerate the convergence. Furthermore, considering the dynamic time-series and potentially non-stationary structure of industrial data, we propose extended incremental versions to alleviate the complexity of the overall model computation. Extensive data recovery experiments are conducted on two real industrial processes to evaluate the proposed method in comparison with existing state-of-the-art restorers. The results show that the proposed methods can impute better with different missing rates and have strong competitiveness in practical application.

A novel low-rank matrix approximation algorithm for face denoising and background/foreground separation

A novel low-rank matrix approximation algorithm for face denoising and background/foreground separation

Isogeometric analysis of diffusion problems on random surfaces

In this article, we discuss the numerical solution of diffusion equations on random surfaces within the isogeometric framework. We describe in detail, how diffusion problems on random surfaces can be modelled and how quantities of interest may be derived. In particular, we employ a low rank approximation algorithm for the high-dimensional space-time correlation of the random solution based on an online singular value decomposition, cp. [7]. Extensive numerical studies are performed to validate the approach. In particular, we consider complex computational geometries originating from surface triangulations. The latter can be recast into the isogeometric context by transforming them into quadrangulations using the procedure from [41] and a subsequent approximation by NURBS surfaces.

Cold-Start Link Prediction via Weighted Symmetric Nonnegative Matrix Factorization with Graph Regularization

Link prediction has attracted wide attention among interdisciplinary researchers as an important issue in complex network. It aims to predict the missing links in current networks and new links that will appear in future networks. Despite the presence of missing links in the target network of link prediction studies, the network it processes remains macroscopically as a large connected graph. However, the complexity of the real world makes the complex networks abstracted from real systems often contain many isolated nodes. This phenomenon leads to existing link prediction methods not to efficiently implement the prediction of missing edges on isolated nodes. Therefore, the cold-start link prediction is favored as one of the most valuable subproblems of traditional link prediction. However, due to the loss of many links in the observation network, the topological information available for completing the link prediction task is extremely scarce. This presents a severe challenge for the study of cold-start link prediction. Therefore, how to mine and fuse more available non-topological information from observed network becomes the key point to solve the problem of cold-start link prediction. In this paper, we propose a framework for solving the cold-start link prediction problem, a joint-weighted symmetric nonnegative matrix factorization model fusing graph regularization information, based on low-rank approximation algorithms in the field of machine learning. First, the nonlinear features in high-dimensional space of node attributes are captured by the designed graph regularization term. Second, using a weighted matrix, we associate the attribute similarity and first order structure information of nodes and constrain each other. Finally, a unified framework for implementing cold-start link prediction is constructed by using a symmetric nonnegative matrix factorization model to integrate the multiple information extracted together. Extensive experimental validation on five real networks with attributes shows that the proposed model has very good predictive performance when predicting missing edges of isolated nodes.

Overview of Adaptive and Low-Rank Approximation Algorithms for Modeling of The Influence of Electromagnetic Waves Generated by The Cell Phone Antenna on The Human Head

This paper presents an overview of formulations and algorithms dedicated to modeling the influence of electromagnetic waves on the human head. We start from the three-dimensional MRI scan of the human head. We approximate the MRI scan by the continuous approximation span over three-dimensional h adaptive mesh with quadratic polynomials. Next, we introduce time-harmonic Maxwell equations with a 1.8 GHz cell-phone antenna. We solve the problem of the propagation of electromagnetic waves on the human head. We compute the specific absorption rate used as the heat source for the Pennes bioheat equation. Finally, we introduce the Pennes bio-heat equation modeling the heat generated by the electromagnetic waves propagating through the skull, tissue, and air layers in the human head. We discuss the discretization and time-stepping algorithm for the Pennes equation’s solution over the human head. Namely, we focus on the Crank-Nicolson time integration scheme, to solve the bioheat transfer equations. We employ the hp finite elements with hierarchical shape functions and hp adaptive algorithm in three-dimensions. We propose an adaptive algorithm mixed with time-stepping iterations, where we simultaneously adapt the computational mesh, solve the Maxwell and Pennes equations, and we iterative with time steps. We employ the sparse Gaussian elimination algorithm with low-rank compression of the off-diagonal matrix blocks for the factorization of matrices. We conclude with the statement that 15 minutes of talk with a 1.8 GHz antenna of 1 Wat power results in increased brain tissue temperature up to 38.4 Celsius degree.

A new splitting algorithm for dynamical low-rank approximation motivated by the fibre bundle structure of matrix manifolds

In this paper, we propose a new splitting algorithm for dynamical low-rank approximation motivated by the fibre bundle structure of the set of fixed rank matrices. We first introduce a geometric description of the set of fixed rank matrices which relies on a natural parametrization of matrices. More precisely, it is endowed with the structure of analytic principal bundle, with an explicit description of local charts. For matrix differential equations, we introduce a first order numerical integrator working in local coordinates. The resulting algorithm can be interpreted as a particular splitting of the projection operator onto the tangent space of the low-rank matrix manifold. It is proven to be exact in some particular case. Numerical experiments confirm this result and illustrate the robustness of the proposed algorithm.

Low-Rank Approximation Algorithms for Matrix Completion with Random Sampling

Low-Rank Approximation Algorithms for Matrix Completion with Random Sampling

Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

The low multilinear rank approximation, also known as the truncated Tucker decomposition, has been extensively utilized in many applications that involve higher-order tensors. Popular methods for low multilinear rank approximation usually rely directly on matrix SVD, therefore often suffer from the notorious intermediate data explosion issue and are not easy to parallelize, especially when the input tensor is large. In this paper, we propose a new class of truncated HOSVD algorithms based on alternating least squares (ALS) for efficiently computing the low multilinear rank approximation of tensors. The proposed ALS-based approaches are able to eliminate the redundant computations of the singular vectors of intermediate matrices and are therefore free of data explosion. Also, the new methods are more flexible with adjustable convergence tolerance and are intrinsically parallelizable on high-performance computers. Theoretical analysis reveals that the ALS iteration in the proposed algorithms is q-linear convergent with a relatively wide convergence region. Numerical experiments with large-scale tensors from both synthetic and real-world applications demonstrate that ALS-based methods can substantially reduce the total cost of the original ones and are highly scalable for parallel computing.

Least upper bound of truncation error of low-rank matrix approximation algorithm using QR decomposition with pivoting

Low-rank approximation by QR decomposition with pivoting (pivoted QR) is known to be less accurate than singular value decomposition (SVD); however, the calculation amount is smaller than that of SVD. The least upper bound of the ratio of the truncation error, defined by Vert A-BCVert _2, using pivoted QR to that using SVD is proved to be sqrt{frac{4^k-1}{3}(n-k)+1} for Ain {mathbb {R}}^{mtimes n}(mge n), approximated as a product of Bin {mathbb {R}}^{mtimes k} and Cin {mathbb {R}}^{ktimes n} in this study.

A Parallel Structured Divide-and-Conquer Algorithm for Symmetric Tridiagonal Eigenvalue Problems

In this paper, a parallel structured divide-and-conquer (PSDC) eigensolver is proposed for symmetric tridiagonal matrices based on ScaLAPACK and a parallel structured matrix multiplication algorithm, called PSMMA. Computing the eigenvectors via matrix-matrix multiplications is the most computationally expensive part of the divide-and-conquer algorithm, and one of the matrices involved in such multiplications is a rank-structured Cauchy-like matrix. By exploiting this particular property, PSMMA constructs the local matrices by using generators of Cauchy-like matrices without any communication, and further reduces the computation costs by using a structured low-rank approximation algorithm. Thus, both the communication and computation costs are reduced. Experimental results show that both PSMMA and PSDC are highly scalable and scale to 4096 processes at least. PSDC has better scalability than PHDC that was proposed in [J. Comput. Appl. Math. 344 (2018) 512--520] and only scaled to 300 processes for the same matrices. Comparing with \texttt{PDSTEDC} in ScaLAPACK, PSDC is always faster and achieves $1.4$x--$1.6$x speedup for some matrices with few deflations. PSDC is also comparable with ELPA, with PSDC being faster than ELPA when using few processes and a little slower when using many processes.

A Modified Higher-Order Singular Value Decomposition Framework With Adaptive Multilinear Tensor Rank Approximation for Three-Dimensional Magnetic Resonance Rician Noise Removal.

The magnetic resonance (MR) images are acknowledged to be inevitably corrupted by Rician distributed noise, which adversely affected the image quality for diagnosis purpose. However, the traditional denoising methods may recover the images from corruptions with severe loss of detailed structure and edge information, which would affect the lesion detections and diagnostic decision making. In this study, we challenged improving the Rician noise removal from three-dimensional (3D) MR volumetric data through a modified higher-order singular value decomposition (MHOSVD) method. The proposed framework of MHOSVD involved a parameterized logarithmic nonconvex penalty function for low-rank tensor approximation (LRTA) algorithm optimization to suppress the image noise in MR dataset. Reference cubes were extracted from the noisy image volume, and block matching was performed according to nonlocal similarity for a fourth-order tensor construction. Then the LRTA problem was implemented by tensor factorization approaches, and the ranks of unfolding matrices along different modes of the tensor were estimated utilizing an adaptive nonconvex low-rank method. The denoised MR images were finally restored through aggregating all recovered cubes. We investigated the proposed algorithm MHOSVD on both the synthetic and real clinic 3D MR images for Rician noise removal, and relative results demonstrated that the MHOSVD can recover images with fine structures and detailed edge preservation with heavy noise even as high as 15% of the maximum intensity. The experimental results were also compared along with several classical denoising methods; the MHOSVD exhibited a sufficient improvement in noise-removal performance at various noise conditions in terms of different measurement indices such as peak signal-to-noise ratio and structural similarity index metrics. Based upon the comparison, the proposed MHOSVD has proved a relative state-of-the-art performance with excellent detailed structure reservation.