Approximation Of Matrices Research Articles

Refining uniform approximation algorithm for low-rank Chebyshev embeddings

Abstract Nowadays, low-rank approximations are a critical component of many numerical procedures. Traditionally the problem of low-rank approximation of matrices is solved in unitary invariant norms such as Frobenius or spectral norm due to the existence of efficient methods for constructing approximations. However, recent results discover the potential of low-rank approximations in the Chebyshev norm, which naturally arises in many applications. In this paper, we investigate the problem of uniform approximation of vectors, which is the main component in the low-rank approximations of matrices in the Chebyshev norm. The principal novelty of this paper is the accelerated algorithm for solving uniform approximation problems. We also analyze the iterative procedure of the proposed algorithm and demonstrate that it has a geometric convergence rate. Finally, we provide an extensive numerical evaluation, which demonstrates the effectiveness of the proposed procedures.

Finite-order method to calculate approximate density matrices in the Fock-space multireference coupled cluster theory

An efficient approach to calculate approximate pure-state and transition reduced density matrices in the framework of the multireference relativistic Fock-space coupled cluster (FS CC) theory is proposed. The method is based on the effective operator formalism and consists of the direct substitution of the FS CC Ansatz for a wave operator into the effective operator expression with the subsequent truncation of expansion at the terms quadratic in cluster amplitudes. The final density matrix is defined by active-space density matrices of different ranks ‘dressed’ with contributions from cluster operators. The method gives a connected expression for pure-state density matrices, provided that the intermediate normalisation condition is fulfilled. Moreover, under some additional assumptions, the connectivity can also be ensured for calculated transition property matrix elements and natural transition spinors. The developed technique allows for fast and accurate calculations of one-particle reduced density matrices for a wide range of electronic states. A pilot application of the new technique to construct averaged atomic natural orbital (ANO) basis sets for fully relativistic electronic structure calculations is presented.

Low rank approximation method for perturbed linear systems with applications to elliptic type stochastic PDEs

In this paper, we propose a low rank approximation method for efficiently solving stochastic partial differential equations. Specifically, our method utilizes a novel low rank approximation of the stiffness matrices, which can significantly reduce the computational load and storage requirements associated with matrix inversion without losing accuracy. To demonstrate the versatility and applicability of our method, we apply it to address two crucial uncertainty quantification problems: stochastic elliptic equations and optimal control problems governed by stochastic elliptic PDE constraints. Based on varying dimension reduction ratios, our algorithm exhibits the capability to yield a high-precision numerical solution for stochastic partial differential equations, or provides a rough representation of the exact solutions as a pre-processing phase. Meanwhile, our algorithm for solving stochastic optimal control problems allows a diverse range of gradient-based unconstrained optimization methods, rendering it particularly appealing for computationally intensive large-scale problems. Numerical experiments are conducted and the results provide strong validation of the feasibility and effectiveness of our algorithm.

Reduced-order model to approximate response matrices for filter stack spectrometers.

We present a reduced-order model to calculate response matrices rapidly for filter stack spectrometers (FSSs). The reduced-order model allows response matrices to be built modularly from a set of pre-computed photon and electron transport and scattering calculations through various filter and detector materials. While these modular response matrices are not appropriate for high-fidelity analysis of experimental data, they encode sufficient physics to be used as a forward model in design optimization studies of FSSs, particularly for machine learning approaches that require sampling and testing a large number of FSS designs.

Approximate GCD of several multivariate sparse polynomials based on SLRA interpolation

To compute the greatest common divisor (GCD) of a set of multivariate polynomials, modular algorithms are typically employed to prevent any growth in the coefficient polynomials in the intermediate expressions. However, when dealing with multivariate polynomials with a priori errors on their coefficients, using such modular algorithms becomes challenging. This is because any resulting approximate GCD computed in one variable may have perturbations depending on the evaluation point and may not be an image of the same desired multivariate approximate GCD. This necessitates computing it as given multivariate polynomials, and operating with large matrices whose size is exponential in the number of variables. In this paper, we present a new modular algorithm, suitable for dense cases and effective for sparse ones, called “SLRA interpolation”. This algorithm uses the multidimensional fast Fourier transform (FFT) and the structured low-rank approximation (SLRA) of non-square block diagonal matrices. The SLRA interpolation technique may reduce the time-complexity for one iteration in the computation of approximate GCD of several multivariate polynomials, especially for the sparse case.

Reconstruction and denoising of high-dimensional seismic data via Frobenius-nuclear mixed norm constraints

Abstract The seismic data acquisition design with ‘two-wide and one-high’ geometry effectively improves the imaging quality of seismic records. However, when data are acquired in the field, complex near surface conditions and environmental factors can introduce a variety of noises and gaps in seismic data, impacting the accuracy of seismic imaging. Currently, the method of low-rank matrix/tensor completion is commonly employed for data reconstruction after normal moveout (NMO). In a complex subsurface medium, common midpoint data processed with NMO may not satisfy the linear or quasi-linear assumptions within local data windows. Therefore, this paper exploits the inherent low-rank structure of high-dimensional data to propose a high-dimensional tensor completion method under the Frobenius-nuclear mixed norm constraint (FN-TC). This method unfolds the 4D data tensor into the frequency-space domain along its modes (m, n) and subsequently imposes a non-convex Frobenius-nuclear mixed norm constraint on the unfolded approximate matrices. This approach closely approximates the rank function of the factor matrices, thereby enhancing the accuracy of data modeling. Theoretical and practical studies demonstrate that the novel FN-TC approach can effectively reconstruct high-dimensional seismic data and suppress noise, thereby providing data support for subsequent high-precision seismic imaging.

Data-distribution-informed Nyström approximation for structured data using vector quantization-based landmark determination

We present an effective method for supervised landmark selection in sparse Nyström approximations of kernel matrices for structured data. Our approach transforms structured non-vectorial input data, like graphs or text, into a dissimilarity representation, facilitating the identification of data-distribution-informed landmarks through prototype-based learning. Experimental results indicate competitive approximation quality when compared to existing strategies, showcasing the advantageous impact of incorporating more information into the Nyström landmark selection process. This positions our method as an efficient and versatile solution for large-scale kernel learning.

A note on approximate Hadamard matrices

A note on approximate Hadamard matrices

Sparse Approximate Pseudoinverse Preconditioning for Sparse Supervised Learning Problems with More Features than Samples

In this paper, we propose a new technique for handling a class of sparse supervised learning problems, specifically focusing on regression, binary, and multi-class classification problems. Sparse regression problems are associated with datasets including both arithmetic and categorical features and by encoding the dataset leads to an underdetermined sparse linear system. Furthermore, logistic regression can be used for both sparse binary and multi-class classification problems, solving an underdetermined sparse linear system. A new technique called Sparse Approximate Pseudoinverse Preconditioning (SAPP), namely the Explicit Preconditioned Conjugate Gradient for Normal Equations (EPCGNE) method based on Generic Approximate Sparse Pseudoinverse matrices is introduced for solving underdetermined sparse least square problems. Numerical experiments were carried out demonstrating a significant improvement of the performance metrics for the proposed SAPP scheme compared to other learners.

Randomized two-sided subspace iteration for low-rank matrix and tensor decomposition

The low-rank approximation of big data matrices and tensors plays a pivotal role in many modern applications. Although, a truncated version of the singular value decomposition (SVD) furnishes the best approximation, its computation is challenging on modern, multicore architectures. Recently, the randomized subspace iteration has shown to be a powerful tool in approximating large-scale matrices. In this paper we present a two-sided variant of the randomized subspace iteration. Novel in our work is the exploitation of the unpivoted QR factorization, rather than the SVD, for factorizing the compressed matrix. Hence our algorithm is a randomized rank-revealing URV decomposition. We prove the rank-revealingness of our algorithm by establishing bounds for the singular values as well as the other blocks of the compressed matrix. We further provide bounds on the error of the low-rank approximations of the proposed algorithm, in both 2- and Frobenius norm. In addition, we employ the proposed algorithm to efficiently compute low rank tensor decompositions: we present two randomized algorithms, one for the truncated higher-order SVD, and the other for the tensor SVD. We conduct numerical tests on (i) various classes of matrices, and (ii) synthetic tensors and real datasets to demonstrate the efficacy of the proposed algorithms.

CUR and Generalized CUR Decompositions of Quaternion Matrices and their Applications

Low-rank approximations of quaternion matrices have garnered interest across various applications, such as color images and signal processing. In this paper, we propose the CUR and generalized CUR decompositions of quaternion matrices and utilize the CUR decomposition of the quaternion matrix to solve the robust quaternion principal component analysis (RQPCA) problem. Through the discrete empirical interpolation method (DEIM) for the subset selection, we present the error analysis of the approximation of the CUR and the generalized CUR decompositions of quaternion matrices. The error bound depends on the conditioning of sampling submatrices. Next, we employ the alternating projection and adopt the CUR decomposition of the quaternion matrix to tackle the RQPCA. The non-convex algorithm runs fast and significantly reduces computational complexity. The performance advantages of our algorithms have been experimentally verified on artificial datasets. The RQCUR significantly affects color video background subtraction in the experiment.

Randomized low rank approximation for nonnegative pure quaternion matrices

Pure quaternion matrix has been widely employed to represent data in real-world applications, such as colour images. Additionally, the imaginary part of quaternion matrix is usually nonnegative due to the natural nonnegativity of real-world data. In this paper, we propose an alternating projection-based algorithm for low rank nonnegative pure quaternion matrix approximation, which can exactly calculate the optimal fixed rank approximation while preserve the pure and nonnegative properties from the given data. More concretely, the proposed algorithm alternatively projects the given quaternion matrix onto the fixed rank quaternion matrix set and nonnegative pure quaternion matrix set in an iterative fashion. Moreover, we establish the theoretical convergence guarantee of the proposed algorithm. To extend the proposed algorithm to large-scaled data, we further propose a randomized algorithm with significant lower computational complexity and comparable accuracy. Numerical experiments on colour images show that our algorithms outperform the other state-of-the-art algorithms.

Feature selection based on self-information combining double-quantitative class weights and three-order approximation accuracies in neighborhood rough sets

Feature selection is related to information processing, and its measurement and algorithm use various intelligent methodologies, such as neighborhood rough sets (NRSs). At present by NRSs, the relative neighborhood self-information (Relative-NSI) introduces an information function for algebraic characterization, and its feature selection algorithm (NSI-FS) has been successfully applied. However, Relative-NSI and NSI-FS ignore the underlying recognition information of decision classes; thus, they have the advancement space. In this study, absolute rates and correlative coefficients of decision classes are double-quantitatively introduced, and three-order approximation accuracies are systematically established to induce a robust self-information measure; thus, corresponding feature selection optimally improves NSI-FS to have generalization abilities. Firstly, three-order approximation accuracies are constructed by using two-time class weights of absolute rates and Spearman's correlation coefficients, and the new measure called ClaWNSI promotes Relative-NSI in terms of class recognition. Then, the three-order approximation accuracies and subsequent ClaWNSI are evolved in matrix forms, and relevant class weights of Spearman's correlation coefficients are realized by two vectors from approximation matrices. Furthermore, ClaWNSI and its feature significance motivate a heuristic selection algorithm (called ClaWNSI-FS). Finally, data experiments on 14 datasets validate ClaWNSI and ClaWNSI-FS; ClaWNSI-FS outperforms NSI-FS and five other contrast algorithms to acquire better classification performance.

Approximating sparse Hessian matrices using large-scale linear least squares

Large-scale optimization algorithms frequently require sparse Hessian matrices that are not readily available. Existing methods for approximating large sparse Hessian matrices have limitations. To try and overcome these, we propose a novel approach that reformulates the problem as the solution of a large linear least squares problem. The least squares problem is sparse but can include a number of rows that contain significantly more entries than other rows and are regarded as dense. We exploit recent work on solving such problems using either the normal equations or an augmented system to derive a robust approach for computing approximate sparse Hessian matrices. Example sparse Hessians from the CUTEst test problem collection for optimization illustrate the effectiveness and robustness of the new method.

Tangent Space Based Alternating Projections for Nonnegative Low Rank Matrix Approximation

In this paper, we develop a new alternating projection method to compute nonnegative low rank matrix approximation for nonnegative matrices. In the nonnegative low rank matrix approximation method, the projection onto the manifold of fixed rank matrices can be expensive as the singular value decomposition is required. We propose to use the tangent space of the point in the manifold to approximate the projection onto the manifold in order to reduce the computational cost. We show that the sequence generated by the alternating projections onto the tangent spaces of the fixed rank matrices manifold and the nonnegative matrix manifold, converge linearly to a point in the intersection of the two manifolds where the convergent point is sufficiently close to optimal solutions. This convergence result based inexact projection onto the manifold is new and is not studied in the literature. Numerical examples in data clustering, pattern recognition and hyperspectral data analysis are given to demonstrate that the performance of the proposed method is better than that of nonnegative matrix factorization methods in terms of computational time and accuracy.

An Improved Approach for Implementing Dynamic Mode Decomposition with Control

Dynamic Mode Decomposition with Control is a powerful technique for analyzing and modeling complex dynamical systems under the influence of external control inputs. In this paper, we propose a novel approach to implement this technique that offers computational advantages over the existing method. The proposed scheme uses singular value decomposition of a lower order matrix and requires fewer matrix multiplications when determining corresponding approximation matrices. Moreover, the matrix of dynamic modes also has a simpler structure than the corresponding matrix in the standard approach. To demonstrate the efficacy of the proposed implementation, we applied it to a diverse set of numerical examples. The algorithm’s flexibility is demonstrated in tests: accurate modeling of ecological systems like Lotka-Volterra, successful control of chaotic behavior in the Lorenz system and efficient handling of large-scale stable linear systems. This showcased its versatility and efficacy across different dynamical systems.

On the optimal rank-1 approximation of matrices in the Chebyshev norm

The problem of low rank approximation is ubiquitous in science. Traditionally this problem is solved in unitary invariant norms such as Frobenius or spectral norm due to existence of efficient methods for building approximations. However, recent results reveal the potential of low rank approximations in Chebyshev norm, which naturally arises in many applications. In this paper we tackle the problem of building optimal rank-1 approximations in the Chebyshev norm. We investigate the properties of alternating minimization algorithm for building the low rank approximations and demonstrate how to use it to construct optimal rank-1 approximation. As a result we propose an algorithm that is capable of building optimal rank-1 approximations in Chebyshev norm for moderate matrices.

An efficient modified residual-based algorithm for large scale symmetric nonlinear equations by approximating successive iterated gradients

In this paper, a new descent approximate modified residual algorithm is developed to solve a large scale system of nonlinear symmetric equations, where the basic strategy to improve its numerical performance is to approximately compute the gradients and the difference of gradients. The error bounds of this approximation are presented, and in virtue of this approximation, a conjugate gradient algorithm for solving large-scale optimization problems in the literature is extended to solve the system of nonlinear symmetric equations without needs of computing and storing the Jacobian matrices or their approximate matrices. It is proved that the obtained search directions in our developed algorithm are sufficiently decent with respect to the so-called approximate modified residues. Under mild assumptions, global and local convergence results of the developed algorithm are proved. Numerical tests indicate that the developed algorithm outperforms the other similar ones available in the literature.

A scalable second order optimizer with an adaptive trust region for neural networks

We introduce Tadam (Trust region ADAptive Moment estimation), a new optimizer based on the trust region of the second-order approximation of the loss using the Fisher information matrix. Despite the enhanced gradient estimations offered by second-order approximations, their practical implementation requires sizable batch sizes to estimate the second-order approximation matrices and perform matrix inversions. Consequently, integrating second-order approximations entails additional memory consumption and imposes substantial computational demands due to the inversion of large matrices. In light of these challenges, we have devised a second-order approximation algorithm that mitigates these issues by judiciously approximating the pertinent large matrix, requiring only a marginal increase in memory usage while minimizing the computational burden. Tadam approximates the loss up to the second order using the Fisher information matrix. Since estimating the Fisher information matrix is expensive in both memory and time, Tadam approximates the Fisher information matrix and reduces the computational burdens to the O(N) level. Furthermore, Tadam employs an adaptive trust region scheme to reduce approximate errors and guarantee stability. Tadam evaluates how well it minimizes the loss function and uses this information to adjust the trust region dynamically. In addition, Tadam adjusts the learning rate internally, even if we provide the learning rate as a fixed constant. We run several experiments to measure Tadam’s performance against Adam, AMSGrad, Radam, and Nadam, which have the same space and time complexity as Tadam. The test results show that Tadam outperforms the benchmarks and finds reasonable solutions fast and stably.

On the Application of Mosaic-Skeleton Approximations of Matrices in Electrodynamics Problems with Impedance Boundary Conditions

On the Application of Mosaic-Skeleton Approximations of Matrices in Electrodynamics Problems with Impedance Boundary Conditions