Orthogonal Matrix Decomposition Research Articles

Probabilistic perturbation bounds of matrix decompositions

AbstractIn this article, we determine probabilistic approximations of the entries of random perturbation matrices implementing the Markoff inequality. These approximations are used to derive with prescribed probability asymptotic componentwise perturbation bounds of some orthogonal and unitary matrix decompositions. We show that the probabilistic asymptotic bounds are significantly less conservative than the corresponding deterministic perturbation bounds. As case studies, we consider the determining of probabilistic perturbation bounds of the QR decomposition, the singular value decomposition and the Schur decomposition of a matrix using an unified method for asymptotic componentwise perturbation analysis of these decompositions. It is demonstrated that the probability bounds of the orthogonal transformations, singular values and eigenvalues are much tighter than the corresponding deterministic asymptotic bounds. The probabilistic bounds derived are appropriate for perturbation analysis of large‐order matrix problems.

Stochastic nested primal-dual method for nonconvex constrained composition optimization

In this paper we study the nonconvex constrained composition optimization, in which the objective contains a composition of two expected-value functions whose accurate information is normally expensive to calculate. We propose a STochastic nEsted Primal-dual (STEP) method for such problems. In each iteration, with an auxiliary variable introduced to track the inner layer function values we compute stochastic gradients of the nested function using a subsampling strategy. To alleviate difficulties caused by possibly nonconvex constraints, we construct a stochastic approximation to the linearized augmented Lagrangian function to update the primal variable, which further motivates to update the dual variable in a weighted-average way. Moreover, to better understand the asymptotic dynamics of the update schemes we consider a deterministic continuous-time system from the perspective of ordinary differential equation (ODE). We analyze the Karush-Kuhn-Tucker measure at the output by the STEP method with constant parameters and establish its iteration and sample complexities to find an ϵ \epsilon -stationary point, ensuring that expected stationarity, feasibility as well as complementary slackness are below accuracy ϵ \epsilon . To leverage the benefit of the (near) initial feasibility in the STEP method, we propose a two-stage framework incorporating a feasibility-seeking phase, aiming to locate a nearly feasible initial point. Moreover, to enhance the adaptivity of the STEP algorithm, we propose an adaptive variant by adaptively adjusting its parameters, along with a complexity analysis. Numerical results on a risk-averse portfolio optimization problem and an orthogonal nonnegative matrix decomposition reveal the effectiveness of the proposed algorithms.

Обчислення сингулярного розкладу матриць з використанням графічного процесора

In this research paper we present an implementation of a singular value decomposition algorithm designed specifically for the graphics processing unit. It consists of two parts: orthogonal matrix decomposition and matrix diagonalization. Presented an implementation of bidiagonalization algorithm where we calculate the main bidiagonal matrix and two orthogonal multipliers using a series of House- holder transformations, as well as diagonalization algorithm with the help of Givens rotation matrices. Bothe these parts are implemented in jCUDA environment. Experiments have been conducted, the results of which have been thoroughly investigated on the matter of time consumption and calculations error. We’ve also compared our implementation with alternatives both on central and graphic processors.

Flood forecasting and uncertainty analysis based on the combination of improved adaptive noise learning model and density estimation

AbstractThe strong randomness exhibited by runoff series means the accuracy of flood forecasting still needs to be improved. Mode mixing can be dealt with using complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), and the endpoint effect of CEEMDAN can be successfully dealt with using the mutual information criterion. To increase the computational effectiveness of broad learning (BL), orthogonal triangular matrix decomposition (QR) was used. A novel improved coupled CEEMDAN-QRBL flood forecasting model was created and applied to the prediction of daily runoff in Xiaolangdi Reservoir based on the benefit of quick calculation by the model output layer. The findings indicate that the enhanced QRBL is 28.92% more computationally efficient than the BL model, and that the reconstruction error of CEEMDAN has been decreased by 48.22%. The MAE of the improved CEEMDAN-QRBL model is reduced by 12.36% and 16.31%, and the Ens is improved by 8.81% and 3.96%, respectively, when compared with the EMD-LSTM and CEEMDAN-GRU models. The predicted values of the CEEMDAN-QRBL model have a suitable fluctuation range thanks to the use of nonparametric kernel density estimation (NPKDE), which might serve as a useful benchmark for the distribution of regional water resources.

Drought prediction based on an improved VMD-OS-QR-ELM model.

To overcome the low accuracy, poor reliability, and delay in the current drought prediction models, we propose a new extreme learning machine (ELM) based on an improved variational mode decomposition (VMD). The model first redefines the output of the hidden layer of the ELM model with orthogonal triangular matrix decomposition (QR) to construct an orthogonal triangular ELM (QR-ELM), and then introduces an online sequence learning mechanism (OS) into the QR-ELM to construct an online sequence OR-ELM (OS-QR-ELM), which effectively improves the efficiency of the ELM model. The mutual information extension method was then used to extend both ends of the original signal to improve the VMD end effect. Finally, VMD and OS-QR-ELM were combined to construct a drought prediction method based on the VMD-OS-QR-ELM. The reliability and accuracy of the VMD-OS-QR-ELM model were improved by 86.19% and 93.20%, respectively, compared with those of the support vector regression model combined with empirical mode decomposition. Furthermore, the calculation efficiency of the OS-QR-ELM model was increased by 88.65% and 85.32% compared with that of the ELM and QR-ELM models, respectively.

А Gpu-based Orthogonal Matrix Factorization Algorithm that Produces a Two-Diagonal Shape


 
 
 With the development of the Big Data sphere, as well as those fields of study that we can relate to artificial intelligence, the need for fast and efficient computing has become one of the most important tasks nowadays. That is why in the recent decade, graphics processing unit computations have been actively developing to provide an ability for scientists and developers to use thousands of cores GPUs have in order to perform intensive computations. The goal of this research is to implement orthogonal decomposition of a matrix by applying a series of Householder transformations in Java language using JCuda library to conduct a research on its benefits. Several related papers were examined. Malaschonok and Savchenko in their work have introduced an improved version of QR algorithm for this purpose [4] and achieved better results, however Householder algorithm is more promising for GPUs according to another team of researchers – Lahabar and Narayanan [6]. However, they were using Float numbers, while we are using Double, and apart from that we are working on a new BigDecimal type for CUDA. Apart from that, there is still no solution for handling huge matrices where errors in calculations might occur.
 The algorithm of orthogonal matrix decomposition, which is the first part of SVD algorithm, is researched and implemented in this work. The implementation of matrix bidiagonalization and calculation of orthogonal factors by the Hausholder method in the jCUDA environment on a graphics processor is presented, and the algorithm for the central processor for comparisons is also implemented. Research of the received results where we experimentally measured acceleration of calculations with the use of the graphic processor in comparison with the implementation on the central processor are carried out. We show a speedup up to 53 times compared to CPU implementation on a big matrix size, specifically 2048, and even better results when using more advanced GPUs. At the same time, we still experience bigger errors in calculations while using graphic processing units due to synchronization problems. We compared execution on different platforms (Windows 10 and Arch Linux) and discovered that they are almost the same, taking the computation speed into account. The results have shown that on GPU we can achieve better performance, however there are more implementation difficulties with this approach.
 
 

A geometric approach to subspace updates and orthogonal matrix decompositions under rank-one modifications

A geometric approach to subspace updates and orthogonal matrix decompositions under rank-one modifications

A new error in variables model for solving positive definite linear system using orthogonal matrix decompositions

The need to estimate a positive definite solution to an overdetermined linear system of equations with multiple right hand side vectors arises in several process control contexts. The coefficient and the right hand side matrices are respectively named data and target matrices. A number of optimization methods were proposed for solving such problems, in which the data matrix is unrealistically assumed to be error free. Here, considering error in measured data and target matrices, we present an approach to solve a positive definite constrained linear system of equations based on the use of a newly defined error function. To minimize the defined error function, we derive necessary and sufficient optimality conditions and outline a direct algorithm to compute the solution. We provide a comparison of our proposed approach and two existing methods, the interior point method and a method based on quadratic programming. Two important characteristics of our proposed method as compared to the existing methods are computing the solution directly and considering error both in data and target matrices. Moreover, numerical test results show that the new approach leads to smaller standard deviations of error entries and smaller effective rank as desired by control problems. Furthermore, in a comparative study, using the Dolan-More performance profiles, we show the approach to be more efficient.

New matrix partial order based on spectrally orthogonal matrix decomposition

We investigate partial orders on the set of complex square matrices and introduce a new order relation based on spectrally orthogonal matrix decompositions. We also establish the relation of this concept with the known orders.

Application of QR Decomposition Algorithm for Multivariate Calibration in High Overlapping Systems

ABSTRACT An accurate and stable algorithm, QR Decomposition Algorithm (QRDA) is initially presented into the chemistry field. The basic principle of QR Multivariate Calibration Algorithm (QRMCA) is investigated, and the complexes system of sodium tripolyphosphate (STPP), sodium sulfate (SS), and sodium Carbonate (SC) with serious overlapping absorption peaks is analyzed by QRMCA. According to our survey, QRDA can avoid matrix inverting, reduce the orders of matrixes, and speed up the operation of matrices. So, it has bright prospects in chemometrics. The process of the QR Decomposition Algorithm (QRDA) was implemented by an orthogonal and upper triangular matrix decomposition that is robust and convenient. Then it was applied to overcome the problem of overlapping spectra of different components in detergent-washing powder.

A unified algebraic transformation approach for parallel recursive and adaptive filtering and SVD algorithms

In this paper, a unified algebraic transformation approach is presented for designing parallel recursive and adaptive digital filters and singular value decomposition (SVD) algorithms. The approach is based on the explorations of some algebraic properties of the target algorithms' representations. Several typical modern digital signal processing examples are presented to illustrate the applications of the technique. They include the cascaded orthogonal recursive digital filter, the Givens rotation-based adaptive inverse QR algorithm for channel equalization, and the QR decomposition-based SVD algorithms. All three examples exhibit similar throughput constraints. There exist long feedback loops in the algorithms' signal flow graph representation, and the critical path is proportional to the size of the problem. Applying the proposed algebraic transformation techniques, parallel architectures are obtained for all three examples. For cascade orthogonal recursive filter, retiming transformation and orthogonal matrix decompositions (or pseudo-commutativity) are applied to obtain parallel filter architectures with critical path of five Givens rotations. For adaptive inverse QR algorithm, the commutativity and associativity of the matrix multiplications are applied to obtain parallel architectures with critical path of either four Givens rotations or three Givens rotations plus two multiply-add operations, whichever turns out to be larger. For SVD algorithms, retiming and associativity of the matrix multiplications are applied to derive parallel architectures with critical path of eight Givens rotations. The critical paths of all parallel architectures are independent of the problem size as compared with being proportional to the problem size in the original sequential algorithms. Parallelism is achieved at the expense of slight increase (or the same for the SVD case) in the algorithms' computational complexity.

Pipelined CORDIC-based cascade orthogonal IIR digital filters

CORDIC-based cascade orthogonal infinite-impulse response (IIR) digital filters possess desirable properties for VLSI implementations such as local connection, regularity, absence of limit cycle and overflow oscillations, and good finite word-length behavior. However, the achievable sample rate of these filters is limited, since these structures cannot be pipelined at finer levels (such as bit or multi-bit level) due to the presence of feedback loops. In this paper, we present a novel approach to design pipelined CORDIC-based cascade orthogonal IIR digital filters using the transfer function approach. We first present a systematic way to synthesize cascade orthogonal IIR digital filters using scalar lossless inverse scattering theory, and realize the filter transfer function as a cascade inter-connection of orthogonal sections where each section implements one real zero or a pair of complex conjugate zeroes of the transfer function. In this way, the filter achieves low sensitivity over the entire filter spectrum. Novel pipelining techniques for both coarse-grain and fine-grain pipelining of these filters are then proposed. In coarse-grain pipelining, we present a novel method based on retiming and orthogonal matrix decomposition techniques which can increase the maximum filter sample rate to O(1) level which is independent of the filter order. In fine-grain pipelining, we present a novel method based on constraint filter design and polyphase decomposition techniques which could increase the maximum filter sample rate to any desired level. The proposed architecture for coarse-grain pipelining consists of only Givens rotations, and the one for fine-grain pipelining consists of only Givens rotations and a few additions. Both architectures can be realized using CORDIC arithmetic-based processors. Finally, finite word-length simulations are carried out to compare the performance of different topologies.

Multivariable constrained predictive control (with application to high performance distillation)

Abstract A numerically computed constrained multivariable generalized predictive control algorithm is proposed. The solution of the optimization problem is achieved using an orthogonal matrix decomposition. Apart from improving the numerical integrity of the solution, Singular Value Decomposition also provides additional useful information that can be used to enhance the robustness of the closed loop control system. The proposed algorithm has been applied to two high purity distillation column physiochemical models, and the closed loop control performances studied. These have demonstrated the validity of the algorithm in respect of being able to respond to process operating constraints. In addition, the algorithm has been shown to be capable of stabilizing systems in which the nominal plant dynamic gain matrix was ill-conditioned.

Regularization of descriptor systems by output feedback

Conditions are given under which a descriptor, or generalized state-space system can be regularized by output feedback. It is shown that under these conditions, proportional and derivative output feedback controls can be constructed such that the closed-loop system is regular and has index at most one. This property ensures the solvability of the resulting system of dynamic-algebraic equations. A reduced form is given that allows the system properties as well as the feedback to be determined. The construction procedures used to establish the theory are based only on orthogonal matrix decompositions and can therefore be implemented in a numerically stable way. >

Regularization of Descriptor Systems by Derivative and Proportional State Feedback

For linear multivariable time-invariant continuous or discrete-time singular systems it is customary to use a proportional feedback control in order to achieve a desired closed loop behaviour. Derivative feedback is rarely considered. This paper examines how derivative feedback in descriptor systems can be used to alter the structure of the system pencil under various controllability conditions. It is shown that derivative and proportional feedback controls can be constructed such that the closed loop system has a given form and is also regular and has index at most 1. This property ensures the solvability of the resulting system of dynamic-algebraic equations. The construction procedures used to establish the theory are based only on orthogonal matrix decompositions and can therefore be implemented in a numerically stable way. The problem of pole placement with derivative feedback alone and in combination with proportional state feedback is also investigated. A computational algorithm for improving the “conditioning” of the regularized closed loop system is derived.

Error analysis of an algorithm for solving an underdetermined linear system

The accumulation of rounding errors in a method used to compute the solution of an underdetermined system of linear equations at the least distance from a given point is being studied. The method used is based on orthogonal matrix decompositions.