Rank Revealing QR Factorization Research Articles

Subspace learning for feature selection via rank revealing QR factorization: Fast feature selection

The identification of informative and distinguishing features from high-dimensional data has gained significant attention in the field of machine learning. Recently, there has been growing interest in employing matrix factorization-based techniques, such as non-negative matrix factorization, for feature selection. The primary objective of feature selection using matrix factorization is to extract a lower-dimensional subspace that captures the essence of the original space. This study introduces a novel unsupervised feature selection technique that leverages rank revealing QR (RRQR) factorization. Compared to singular value decomposition (SVD) and non-negative matrix factorization (NMF), RRQR is more computationally efficient. The uniqueness of this technique lies in the utilization of the permutation matrix of QR for feature selection. Additionally, we integrate QR factorization into the objective function of NMF to create a new unsupervised feature selection method. Furthermore, we propose a hybrid feature selection algorithm by combining RRQR and a Genetic algorithm. The algorithm eliminates redundant features using RRQR factorization and selects the most distinguishing subset of features using the Genetic algorithm. Experimental comparisons with state-of-the-art feature selection algorithms in supervised, unsupervised, and semi-supervised settings demonstrate the reliability and robustness of the proposed algorithm The evaluation is conducted on eight microarray datasets using KNN, SVM, and C4.5 classifiers. The experimental results indicate that the proposed method achieves comparable performance to the state-of-the-art feature selection methods. Our empirical findings demonstrate that the proposed method exhibits a significantly lower computational cost compared to other techniques.

Seq-SVF: An unsupervised data-driven method for automatically identifying hidden governing equations

In this work, an unsupervised data-driven method based on sequential singular value filtering (Seq-SVF) is proposed to simultaneously identify multiple partial differential equations from observed data considering potential noises. This method is aimed to extend the Sparse Identification of Nonlinear Dynamics (SINDy) to the identification of general nonlinear partial differential equations by transforming the paradigm based on regression to an unsupervised paradigm. To discover the complex coupled equations of vector or tensor forms without prior knowledge, the techniques of singular value decomposition (SVD) and strong rank-revealing QR factorization (sRRQR) are applied to the data matrix, which ensures that the method can automatically identify the number and the corresponding linearly independent terms as the left-hand terms of governing equations. To balance the complexity and the precision of modeling, a strategy for filtering singular values is designed to determine the sparse structure of governing equations from a large number of nonlinear basis functions. We show the success of the method to extract explicit and succinct models from many complex linear, nonlinear, and multiphysics mechanical systems, and the examples show more accuracy compared with using traditional sparse learning methods.

Sparse representation learning using [formula omitted] compressed sensing and rank-revealing QR factorization

Compressive sensing can be conceptualized for a classification problem in the context of representation learning, which is frequently applied for signal reconstruction via a few measurements. A novel compressive sensing-based approach is presented to substantially reduce the number of required samples for classification. This technique considers the pixels of image as sensors and it seeks to identify the most optimum sensors as decision variables in feature space. In this study, Spatial sensor locations are acquired through learning to identify the regions within an image where the most discriminative information for classification is embedded. ℓ1−2 minimization, as nonconvex but Lipschitz continuous, is solved to obtain the least non-zero elements of the full measurement vector to completely reconstruct the discriminative vector in feature space. Optimal sensors are localized from the training-set and subsequent test images are categorized based on learned sensors. The algorithm consists of three primary components: sparse minimization, feature space, and discrimination vector. ℓ1−2 minimization, rank-revealing QR factorization (RRQR) and SVM are considered for enhanced sparsity exploitation, feature space extraction and discrimination vector acquiring, respectively. The proposed method is evaluated on four different experiments and is compared against a state-of-art technique. The results demonstrate the superiority of proposed method to the compared method.

Polynomial chaos expansion approximation for dimension-reduction model-based reliability analysis method and application to industrial robots

Polynomial chaos expansion (PCE) is considered an excellent method for accurately and efficiently reliability analysis in various engineering problems. However, it becomes practically infeasible in scenarios characterized by high-dimensional input random variables and thousands of training data. To solve this problem, this paper proposes a new PCE-based surrogate-assisted method. To start with, the interacting variables are screened out from original high-dimensional input random variables by contribution-degree analysis (CDA). The original high-dimensional performance function is decomposed into a lower-dimensional component function composed of the interacting variables and multiple one-dimensional component function containing only one non-interacting variable. Then, PCE combined with a sample points selection strategy is used to fit the lower-dimensional component function, and a full PCE is utilized for fitting each one-dimensional component function. Three methods, namely the Hadamard's inequality of the design matrix, the rank revealing QR factorization of the design matrix and the maximum entropy, are implemented to realize sample point selection. Four examples are investigated to demonstrated the effectiveness of the proposed method.

Data selection and data-enabled predictive control for a fuel cell system

Data-enabled predictive control is applied for power tracking of a fuel cell system and algorithmic aspects are investigated. For the system realization, the column subset selection algorithms norm sampling, iterative norm sampling, leverage score sampling, and selection based on the strong rank-revealing QR factorization are compared. Regularization using the 1-norm and the squared 2-norm on the column combination vector g is evaluated in terms of runtime and closed-loop results. The results indicate that column selection based on a strong rank-revealing QR (sRRQR) decomposition and leverage score sampling lead to consistently small closed-loop costs even with comparatively few columns in the data matrix resulting in reduced computation time. The closed-loop performance with 1-norm and squared 2-norm is similar, with lower turnaround times for the latter.

HIFIR: Hybrid Incomplete Factorization with Iterative Refinement for Preconditioning Ill-Conditioned and Singular Systems

We introduce a software package called Hybrid Incomplete Factorization with Iterative Refinement (HIFIR) for preconditioning sparse, unsymmetric, ill-conditioned, and potentially singular systems. HIFIR computes a hybrid incomplete factorization (HIF) , which combines multilevel incomplete LU factorization with a truncated, rank-revealing QR (RRQR) factorization on the final Schur complement. This novel hybridization is based on the new theory of ϵ-accurate approximate generalized inverse (AGI) . It enables near-optimal preconditioners for consistent systems and enables flexible GMRES to solve inconsistent systems when coupled with iterative refinement. In this article, we focus on some practical algorithmic and software issues of HIFIR. In particular, we introduce a new inverse-based rook pivoting (IBRP) into ILU, which improves the robustness and the overall efficiency for some ill-conditioned systems by significantly reducing the size of the final Schur complement for some systems. We also describe the software design of HIFIR in terms of its efficient data structures for supporting rook pivoting in a multilevel setting, its template-based generic programming interfaces for mixed-precision real and complex values in C++, and its user-friendly high-level interfaces in MATLAB and Python. We demonstrate the effectiveness of HIFIR for ill-conditioned or singular systems arising from several applications, including the Helmholtz equation, linear elasticity, stationary incompressible Navier–Stokes (INS) equations, and time-dependent advection-diffusion equation.

Deviation maximization for rank-revealing QR factorizations

In this paper, we introduce a new column selection strategy, named here “Deviation Maximization”, and apply it to compute rank-revealing QR factorizations as an alternative to the well-known block version of the QR factorization with the column pivoting method, called QP3 and currently implemented in LAPACK’s xgeqp3 routine. We show that the resulting algorithm, named QRDM, has similar rank-revealing properties of QP3 and better execution times. We present experimental results on a wide data set of numerically singular matrices, which has become a reference in the recent literature.

Feature Selection Using Rank Revealing Qr Factorization and Genetic Algorithm: Unsupervised and Hybrid Supervised Approaches

Feature Selection Using Rank Revealing Qr Factorization and Genetic Algorithm: Unsupervised and Hybrid Supervised Approaches

Robust face recognition based on a new Kernel-PCA using RRQR factorization

In the last ten years, many variants of the principal component analysis were suggested to fight against the curse of dimensionality. Recently, A. Sharma et al. have proposed a stable numerical algorithm based on Householder QR decomposition (HQR) called QR PCA. This approach improves the performance of the PCA algorithm via a singular value decomposition (SVD) in terms of computation complexity. In this paper, we propose a new algorithm called RRQR PCA in order to enhance the QR PCA performance by exploiting the Rank-Revealing QR Factorization (RRQR). We have also improved the recognition rate of RRQR PCA by developing a nonlinear extension of RRQR PCA. In addition, a new robust RBF Lp-norm kernel is proposed in order to reduce the effect of outliers and noises. Extensive experiments on two well-known standard face databases which are ORL and FERET prove that the proposed algorithm is more robust than conventional PCA, 2DPCA, PCA-L1, WTPCA-L1, LDA, and 2DLDA in terms of face recognition accuracy.

An adaptive mesh refinement approach based on optimal sparse sensing

We introduce a new approach for adaptive mesh refinement in which adaptivity is driven by low rank decomposition and optimal sensing of the dynamically evolving flow field. This method seeks an ordered set of locations for mesh adaptation from the instantaneous data-driven basis of an online proper orthogonal decomposition of the velocity, which organizes features into sparse optimal orthogonal modes based on an energy norm. The sensing is achieved via a computationally expedient discrete empirical interpolation method using rank-revealing QR factorization (Drmac and Gugercin SIAM J Sci Comput 38(2):A631–A648, 2016). The methodology is applicable to a wide range of numerical discretizations, and is tested on a spatiotemporally evolving incompressible turbulent jet, a complex wind turbine wake, and supersonic flow over a forward-facing step. The proposed approach is demonstrated to predict accurate velocity statistics and yield significantly smaller grids in comparison to gradient-based methods. The algorithm is seen to focus refinement in the vicinity of dynamically significant regions such as those characterized by high turbulence kinetic energy, coherent structures and shock interactions. Moreover, the approach does not require parameters or thresholds, which may be difficult to obtain for complex flows, to be known a priori to facilitate mesh adaptation.

Skeletonization-Scheme-Based Adaptive Near Field Sampling for Radio Frequency Source Reconstruction

Radio frequency (RF) source reconstruction is useful for RF radiation analysis and interference diagnosis in Internet of Things. Near field (NF) sampling is a critical step of RF source reconstruction. Accurate RF source reconstruction usually requires a large number of NF samples, which results in tremendous effort of NF sampling. This article presents an adaptive NF sampling method based on the skeletonization scheme. First, sources to be reconstructed are related to NF samples through integral equation (IE). Second, the IE is discretized with the method of moments, and thus the interaction matrix between source and field points is found. Third, strong rank-revealing QR factorization is applied to the interaction matrix, which results in a permutation matrix, a row skeleton matrix, and a transformation matrix. Finally, a small number of skeleton sampling points are selected by analyzing the permutation matrix. The fields at skeleton sampling points can be used to calculate the fields at other sampling points through the transformation matrix. Hence, one only needs to perform NF sampling at a small number of skeleton sampling points, which significantly reduces the expenditure of NF sampling. Simulations using synthetic and measurement data are presented to show the effectiveness and advantages of the proposed sampling method.

Synthesis, kinetic analysis and modelling of galacto-oligosaccharides formation

To design and optimize chemical reactors and bio reactors a profound understanding of kinetics and in particular suitable mechanistic rate models is needed. In chemical reaction engineering the methodology from Christiansen considering elementary kinetics assuming rate-determining steps has been successfully applied. It is hardly considered for enzymatic reactions. In this work, kinetics of formation of the prebiotics galacto-oligosaccharides (GOS) from lactose were investigated with β-galactosidase using batch and dynamic experiments. The rates of the main and most relevant side reactions were quantified. Preliminary investigations were performed to determine the influence of pH, temperature, the amount of enzyme and the initial lactose concentration. To acquire mechanistic understanding instructive perturbation experiments were carried out dosing the substrate lactose and/or the products galactose and glucose either initially or dynamically during fermentation. Based on a postulated catalytic cycle, mechanistic kinetic models were developed considering hydrolysis of lactose, formation of GOS and inhibiting effects of the side products glucose and galactose. A sensitivity analysis was performed applying the subset selection method including model reduction based on singular value decomposition combined with rank revealing QR factorization. The parameterized mechanistic models allowed a good description of batch and dynamic experiments in a broad range of operating conditions.

A Novel Measurement Matrix Optimization Approach for Hyperspectral Unmixing

Each pixel in the hyperspectral unmixing process is modeled as a linear combination of endmembers, which can be expressed in the form of linear combinations of a number of pure spectral signatures that are known in advance. However, the limitation of Gaussian random variables on its computational complexity or sparsity affects the efficiency and accuracy. This paper proposes a novel approach for the optimization of measurement matrix in compressive sensing (CS) theory for hyperspectral unmixing. Firstly, a new Toeplitz-structured chaotic measurement matrix (TSCMM) is formed by pseudo-random chaotic elements, which can be implemented by a simple hardware; secondly, rank revealing QR factorization with eigenvalue decomposition is presented to speed up the measurement time; finally, orthogonal gradient descent method for measurement matrix optimization is used to achieve optimal incoherence. Experimental results demonstrate that the proposed approach can lead to better CS reconstruction performance with low extra computational cost in hyperspectral unmixing.

Literature survey on low rank approximation of matrices

Low rank approximation of matrices has been well studied in literature. Singular value decomposition, QR decomposition with column pivoting, rank revealing QR factorization, Interpolative decomposition, etc. are classical deterministic algorithms for low rank approximation. But these techniques are very expensive ( operations are required for matrices). There are several randomized algorithms available in the literature which are not so expensive as the classical techniques (but the complexity is not linear in n). So, it is very expensive to construct the low rank approximation of a matrix if the dimension of the matrix is very large. There are alternative techniques like Cross/Skeleton approximation which gives the low-rank approximation with linear complexity in n. In this article we review low rank approximation techniques briefly and give extensive references of many techniques.

Low Rank Approximation of a Sparse Matrix Based on LU Factorization with Column and Row Tournament Pivoting

In this paper we present an algorithm for computing a low rank approximation of a sparse matrix based on a truncated LU factorization with column and row permutations. We present various approaches for determining the column and row permutations that show a trade-off between speed versus deterministic/probabilistic accuracy. We show that if the permutations are chosen by using tournament pivoting based on QR factorization, then the obtained truncated LU factorization with column/row tournament pivoting, LU_CRTP, satisfies bounds on the singular values which have similarities with the ones obtained by a communication avoiding rank revealing QR factorization. Experiments on challenging matrices show that LU_CRTP provides a good low rank approximation of the input matrix and it is less expensive than the rank revealing QR factorization in terms of computational and memory usage costs, while also minimizing the communication cost. We also compare the computational complexity of our algorithm with randomized a...

Optimal survey design for big data

The volume of data routinely acquired in many industrial geoscientific settings has increased to a point that it poses serious technical challenges to store and process. Optimal experimental design is a tool that can address some of these challenges. First, before data acquisition, optimal design can forecast optimum acquisition geometries and can thus be used to minimize the acquired data volume and reduce the cost of the data life cycle. Second, after data acquisition, optimal design can optimally select the best data to process, thus minimizing the data volume throughput in processing workflows. The catch is that optimal survey design is itself subject to the computational difficulties of big data. We have developed a parallelizable dimensionality reduction method, based on sparse rank-revealing QR-factorization to reduce computing and storage costs for optimal survey design. We implemented the distributable procedure for a guided Bayesian survey design and developed it as part of a workflow to optimize a marine seismic survey of more than 100 million source-receiver pairs in the Gulf of Mexico. The method improved computation times by more than an order of magnitude and reduced memory requirements by nearly two orders of magnitude. Although the procedure involves approximations in exchange for subsequent computational efficiency, the marine survey design results were identical to those achieved without compression. The distributable reduction method may be generally useful for matrix reduction problems in which the matrix data are distributed over multiple machines.

Adaptive truncation of matrix decompositions and efficient estimation of NMR relaxation distributions

The two most successful methods of estimating the distribution of nuclear magnetic resonance relaxation times from two dimensional data are data compression followed by application of the Butler–Reeds–Dawson algorithm, and a primal-dual interior point method using preconditioned conjugate gradient. Both of these methods have previously been presented using a truncated singular value decomposition of matrices representing the exponential kernel. In this paper it is shown that other matrix factorizations are applicable to each of these algorithms, and that these illustrate the different fundamental principles behind the operation of the algorithms. These are the rank-revealing QR (RRQR) factorization and the LDL factorization with diagonal pivoting, also known as the Bunch–Kaufman–Parlett factorization. It is shown that both algorithms can be improved by adaptation of the truncation as the optimization process progresses, improving the accuracy as the optimal value is approached. A variation on the interior method viz, the use of barrier function instead of the primal-dual approach, is found to offer considerable improvement in terms of speed and reliability. A third type of algorithm, related to the algorithm known as Fast iterative shrinkage-thresholding algorithm, is applied to the problem. This method can be efficiently formulated without the use of a matrix decomposition.

Communication Avoiding Rank Revealing QR Factorization with Column Pivoting

In this paper we introduce CARRQR, a communication avoiding rank revealing QR factorization with tournament pivoting. We show that CARRQR reveals the numerical rank of a matrix in an analogous way to QR factorization with column pivoting (QRCP). Although the upper bound of a quantity involved in the characterization of a rank revealing factorization is worse for CARRQR than for QRCP, our numerical experiments on a set of challenging matrices show that this upper bound is very pessimistic, and CARRQR is an effective tool in revealing the rank in practical problems. Our main motivation for introducing CARRQR is that it minimizes data transfer, modulo polylogarithmic factors, on both sequential and parallel machines, while previous factorizations as QRCP are communication suboptimal and require asymptotically more communication than CARRQR. Hence CARRQR is expected to have a better performance on current and future computers, where communication is a major bottleneck that highly impacts the performance of an algorithm.

Kinetics of 1-dodecene hydroformylation in a thermomorphic solvent system using a rhodium-biphephos catalyst

The hydroformylation of 1-dodecene on a rhodium-biphephos catalyst complex exploiting a thermomorphic multicomponent solvent system was studied experimentally in a batch reactor in order to describe the kinetics of the main and the most relevant side reactions. The formation of the active catalyst was studied in preliminary experiments. Based on a postulated catalytic cycle mechanistic kinetic models were developed considering isomerization, hydrogenation and hydroformylation reactions as well as the formation of not catalytically active Rh-species. The complex overall network was decomposed to support parameter estimation. The isomerization of 1-dodecene, the hydrogenations of iso- and 1-dodecene and the hydroformylations of iso-dodecene and 1-dodecene were investigated as a function of temperature, total pressure and partial pressures of carbon monoxide and hydrogen, respectively. These four sub-networks of increasing size and the total network were analyzed sequentially in order to identify kinetic models and to estimate the corresponding parameters applying model reduction techniques based on singular value decomposition combined with rank revealing QR factorization.

Improving Multifrontal Methods by Means of Block Low-Rank Representations

Matrices coming from elliptic Partial Differential Equations (PDEs) have been shown to have a low-rank property: well defined off-diagonal blocks of their Schur complements can be approximated by low-rank products. Given a suitable ordering of the matrix which gives to the blocks a geometrical meaning, such approximations can be computed using an SVD or a rank-revealing QR factorization. The resulting representation offers a substantial reduction of the memory requirement and gives efficient ways to perform many of the basic dense algebra operations. Several strategies have been proposed to exploit this property. We propose a low-rank format called Block Low-Rank (BLR), and explain how it can be used to reduce the memory footprint and the complexity of direct solvers for sparse matrices based on the multifrontal method. We present experimental results that show how the BLR format delivers gains that are comparable to those obtained with hierarchical formats such as Hierarchical matrices (H matrices) and Hierarchically Semi-Separable (HSS matrices) but provides much greater flexibility and ease of use which are essential in the context of a general purpose, algebraic solver.