Dominant Subspace Research Articles

Dominant subspace and low-rank approximations from block Krylov subspaces without a prescribed gap

We develop a novel convergence analysis of the classical deterministic block Krylov methods for the approximation of h-dimensional dominant subspaces and low-rank approximations of matrices A∈Km×n (where K=R or C) in the case that there is no singular gap at the index h i.e., if σh=σh+1 (where σ1≥…≥σp≥0 denote the singular values of A, and p=min⁡{m,n}). Indeed, starting with a (deterministic) matrix X∈Kn×r with r≥h satisfying a compatibility assumption with some h-dimensional right dominant subspace of A, we show that block Krylov methods produce arbitrarily good approximations for both problems mentioned above. Our approach is based on recent work by Drineas, Ipsen, Kontopoulou and Magdon-Ismail on the approximation of structural left dominant subspaces. The main difference between our work and previous work on this topic is that instead of exploiting a singular gap at the prescribed index h (which is zero in this case) we exploit the nearest existing singular gaps. We include a section with numerical examples that test the performance of our main results.

Dimension reduction based on time-limited cross Gramians for bilinear systems

The cross Gramian is a useful tool in model order reduction but only applicable to square dynamical systems. Throughout this paper, time-limited cross Gramians is firstly extended to square bilinear systems that satisfies a generalized Sylvester equation, and then concepts from decentralized control are used to approximate a cross Gramian for non-square bilinear systems. In order to illustrate these cross Gramians, they are calculated efficiently based on shifted Legendre polynomials and applied to dimension reduction, which leads to a lower dimensional model by truncating the states that are associated with smaller approximate generalized Hankel singular values. In combination of the dominant subspace projection method, our reduction procedure is modified to produce a bounded-input bounded-output stable-preserved reduced model under some certain conditions. At last, the performance of numerical experiments indicates the validity of our reduction methods.

Special structural Gramian approximation methods for model order reduction of time‐delay systems

AbstractModel order reduction methods via low‐rank approximation of Gramians for time‐delay systems are developed in this paper. The main contribution is to achieve the balancing and truncation of the system by utilizing low‐rank decomposition of the Gramians combined with the low‐rank square root framework. Here, based on Laguerre expansion technique, the low‐rank factorization of the system Gramians is realized via a linear system with special structure, thus enabling an efficient implementation of the reduction process. Furthermore, the issue of stability preservation is briefly described. We employ the dominant subspaces projection model reduction method to mitigate the effects which may accidentally produce unstable reduced models. Finally, numerical results verify the performance of the approximation‐Gramian methods.

Identification of dominant subspaces for model reduction of structured parametric systems

SummaryIn this paper, we discuss a novel model reduction framework for linear structured dynamical systems. The transfer functions of these systems are assumed to have a special structure, for example, coming from second‐order linear systems or time‐delay systems, and they may also have parameter dependencies. Firstly, we investigate the connection between classic interpolation‐based model reduction methods with the reachability and observability subspaces of linear structured parametric systems. We show that if enough interpolation points are taken, the projection matrices of interpolation‐based model reduction encode these subspaces. Consequently, we are able to identify the dominant reachable and observable subspaces of the underlying system. Based on this, we propose a new model reduction algorithm combining these features and leading to reduced‐order systems. Furthermore, we discuss computational aspects of the approach and its applicability to a large‐scale setting. We illustrate the efficiency of the proposed approach with several numerical large‐scale benchmark examples.

Dominant subspaces of high-fidelity polynomial structured parametric dynamical systems and model reduction

In this work, we investigate a model order reduction scheme for high-fidelity nonlinear structured parametric dynamical systems. More specifically, we consider a class of nonlinear dynamical systems whose nonlinear terms are polynomial functions, and the linear part corresponds to a linear structured model, such as second-order, time-delay, or fractional-order systems. Our approach relies on the Volterra series representation of these dynamical systems. Using this representation, we identify the kernels and, thus, the generalized multivariate transfer functions associated with these systems. Consequently, we present results allowing the construction of reduced-order models whose generalized transfer functions interpolate these of the original system at pre-defined frequency points. For efficient calculations, we also need the concept of a symmetric Kronecker product representation of a tensor and derive particular properties of them. Moreover, we propose an algorithm that extracts dominant subspaces from the prescribed interpolation conditions. This allows the construction of reduced-order models that preserve the structure. We also extend these results to parametric systems and a special case (delay in input/output). We demonstrate the efficiency of the proposed method by means of various numerical benchmarks.

Low-rank balanced truncation of discrete time-delay systems based on Laguerre expansions

This paper introduces a novel model order reduction method based on low-rank Gramian approximations for discrete time-delay systems. Firstly, an efficient algorithm based on Laguerre functions to compute the low-rank decomposition factors of the controllability and observability Gramians for discrete time-delay systems is given, in which the low-rank factors satisfy the iterative recursive formulas of the expansion coefficients of the Laguerre functions. It effectively avoids the direct solutions of Gramians. Then, the reduced-order model of the discrete time-delay system is obtained by combining the low-rank square root method. Additionally, a modified algorithm that combines the dominant subspace projection method is introduced, which alleviates certain drawbacks of the above technique and enhances the stability in certain cases. Finally, two numerical examples are given to verify the accuracy and efficiency of our proposed algorithm.

Model reduction for stochastic systems with nonlinear drift

In this paper, we study dimension reduction techniques for large-scale controlled stochastic differential equations (SDEs). The drift of the considered SDEs contains a polynomial term satisfying a one-sided growth condition. Such nonlinearities in high dimensional settings occur, e.g., when stochastic reaction diffusion equations are discretized in space. We provide a brief discussion around existence, uniqueness and stability of solutions. (Almost) stability then is the basis for new concepts of Gramians that we introduce and study in this work. With the help of these Gramians, dominant subspace is identified leading to a balancing related highly accurate reduced order SDE. We provide an algebraic error criterion and an error analysis of the propose model reduction schemes. The paper is concluded by applying our method to spatially discretized reaction diffusion equations.

Dimension reduction based on approximate gramians via Laguerre polynomials for coupled systems

Abstract In this paper, we focus on the topic of model order reduction (MOR) for coupled systems. At first, an approximation via Laguerre polynomials expansions to controllability and observability gramians for such systems are presented, which provides a low-rank decomposition form whose factors are constructed from a recurrence formula instead of Lyapunov equations. Then, in combination of balanced truncation and dominant subspace projection method, a series of MOR algorithms are proposed that preserve the coupled structures. What’s more, some main properties of reduced-order models, such as stability preservation, are well discussed. Finally, three numerical simulations are provided to illustrate the effectiveness of our algorithms.

Model reduction for second-order systems with inhomogeneous initial conditions

In this paper, we consider the problem of finding surrogate models for large-scale second-order linear time-invariant systems with inhomogeneous initial conditions. For this class of systems, the superposition principle allows us to decompose the system behavior into three independent components. The first behavior corresponds to the transfer between the input and output having zero initial conditions. In contrast, the other two correspond to the transfer between the initial position or the initial velocity and the output when no input is applied. Based on this superposition of systems, our goal is to propose model reduction schemes that allow to preserve the second-order structure in the surrogate models. To this aim, we introduce tailored second-order Gramians for each system component and compute them numerically, solving Lyapunov equations. As a consequence, two methodologies are proposed. The first one consists in reducing each of the components independently using a suitable balanced truncation procedure. The sum of these reduced systems provides an approximation of the original system. This methodology allows flexibility on the order of the reduced-order model. The second proposed methodology consists in extracting the dominant subspaces from the sum of Gramians to build the projection matrices leading to a surrogate model. Additionally, we discuss error bounds for the overall output approximation. Finally, the proposed methods are illustrated by means of benchmark problems.

Complexity reduction of large-scale stochastic systems using linear quadratic Gaussian balancing

In this paper, we consider a model reduction technique for stabilizable and detectable stochastic systems. It is based on a pair of Gramians that we analyze in terms of well-posedness. Subsequently, dominant subspaces of the stochastic systems are identified exploiting these Gramians. An associated balancing related scheme is proposed that removes unimportant information from the stochastic dynamics in order to obtain a reduced system. We show that this reduced model preserves important features like stabilizability and detectability. Additionally, a comprehensive error analysis based on eigenvalues of the Gramian pair product is conducted. This provides an a-priori criterion for the reduction quality which we illustrate in numerical experiments.

On the accuracy of the chemically significant eigenvalue method.

We study the accuracy and convergence properties of the chemically significant eigenvalues method as proposed by Georgievskii et al. [J. Phys. Chem. A 117, 12146-12154 (2013)] and its close relative, dominant subspace truncation, for reduction of the energy-grained master equation. We formally derive the connection between both reduction techniques and provide hard error bounds for the accuracy of the latter which confirm the empirically excellent accuracy and convergence properties but also unveil practically relevant cases in which both methods are bound to fall short. We propose the use of balanced truncation as an effective alternative in these cases.

Separable Gaussian Neural Networks: Structure, Analysis, and Function Approximations

The Gaussian-radial-basis function neural network (GRBFNN) has been a popular choice for interpolation and classification. However, it is computationally intensive when the dimension of the input vector is high. To address this issue, we propose a new feedforward network-separable Gaussian neural network (SGNN) by taking advantage of the separable property of Gaussian-radial-basis functions, which splits input data into multiple columns and sequentially feeds them into parallel layers formed by uni-variate Gaussian functions. This structure reduces the number of neurons from O(Nd) of GRBFNN to O(dN), which exponentially improves the computational speed of SGNN and makes it scale linearly as the input dimension increases. In addition, SGNN can preserve the dominant subspace of the Hessian matrix of GRBFNN in gradient descent training, leading to a similar level of accuracy to GRBFNN. It is experimentally demonstrated that SGNN can achieve an acceleration of 100 times with a similar level of accuracy over GRBFNN on tri-variate function approximations. The SGNN also has better trainability and is more tuning-friendly than DNNs with RuLU and Sigmoid functions. For approximating functions with a complex geometry, SGNN can lead to results that are three orders of magnitude more accurate than those of a RuLU-DNN with twice the number of layers and the number of neurons per layer.

Laguerre-Based Low-Rank Balanced Truncation of Discrete-Time Systems

In this brief, a new model order reduction method based on low-rank Gramian approximation is proposed for discrete-time systems. The main task is to use the Laguerre functions expansions to calculate the approximate low-rank decomposition factors of the controllability and observability Gramians, which are calculated by the recurrence formulas. Then, the reduced-order systems are obtained by the low-rank square root method which approximate the original system well. In addition, due to the disadvantages of the above method that it may produce an unstable system even if the original system is stable, a modified reduction procedure is proposed, which can preserve stability in some cases. It combines the dominant subspace projection method to use the sum of both subspaces for a state space projection. The effectiveness of the proposed methods are illustrated by two numerical examples.

Subspace-Based Pilot Decontamination in User-Centric Scalable Cell-Free Wireless Networks

We consider a cell-free wireless system operated in Time Division Duplex (TDD) mode with user-centric clusters of remote radio units (RUs). Since the uplink pilot dimensions per channel coherence slot is limited, co-pilot users might incur mutual pilot contamination. In the current literature, it is assumed that the long-term statistical knowledge of all user channels is available. This enables Minimum Mean-Square Error channel estimation or simplified dominant subspace projection, which achieves significant pilot decontamination under certain assumptions on the channel covariance matrices. However, estimating the channel covariance matrix or even just its dominant subspace at all RUs forming a user cluster is not an easy task. In fact, if not properly designed, a piloting scheme for such long-term statistics estimation will also be subject to the contamination problem. In this paper, we propose a new channel subspace estimation scheme explicitly designed for cell-free wireless networks. Our scheme is based on 1) a sounding reference signal (SRS) using latin squares wideband frequency hopping, and 2) a subspace estimation method based on robust Principal Component Analysis (R-PCA). The SRS hopping scheme ensures that for any user and any RU participating in its cluster, only a few pilot measurements will contain strong co-pilot interference. These few heavily contaminated measurements are (implicitly) eliminated by R-PCA, which is designed to regularize the estimation and discount the “outlier” measurements. Our simulation results show that the proposed scheme achieves almost perfect subspace knowledge, which in turns yields system performance very close to that with ideal channel state information, thus essentially solving the problem of pilot contamination in cell-free user-centric TDD wireless networks.

Model order reduction based on low-rank decomposition of the cross Gramian

This paper considers model order reduction based on low-rank decomposition of the cross Gramian for linear systems. The proposed approach uses the low-rank factors of the cross Gramian to generate approximate balanced model for the large-scale system. Then, the reduced-order model is obtained by truncating the states corresponding to the small approximate Hankel singular values. The low-rank factors are directly constructed from the expansion coefficients of impulse responses in the space spanned by Legendre polynomials by solving block tridiagonal linear systems. In contrast to balanced truncation related approaches which require to solve Sylvester equation to compute the full cross Gramian, the proposed method needs to solve sparse linear equations, and only one singular value decomposition is applied to a low-dimensional matrix. The stability preservation of the reduced model is briefly discussed. And in combination with the dominant subspace projection method, the reduction procedure is modified to alleviate the shortcoming, which may unexpectedly lead to unstable systems even though the original one is stable. Finally, numerical experiments are given to demonstrate the effectiveness of the proposed methods.

Sampling‐based model order reduction for stochastic differential equations driven by fractional Brownian motion

AbstractIn this paper, we study large‐scale linear fractional stochastic systems representing, e.g., spatially discretized stochastic partial differential equations (SPDEs) driven by fractional Brownian motion (fBm) with Hurst parameter H > ½. Such equations are more realistic in modeling real‐world phenomena in comparison to frameworks not capturing memory effects. To the best of our knowledge, dimension reduction schemes for such settings have not been studied so far.In this work, we investigate empirical reduced order methods that are either based on snapshots (e.g. proper orthogonal decomposition) or on approximated Gramians. In each case, dominant subspaces are learned from data. Such model reduction techniques are introduced and analyzed for stochastic systems with fractional noise and later applied to spatially discretized SPDEs driven by fBm in order to reduce the computational cost arising from both the high dimension of the considered stochastic system and a large number of required Monte Carlo runs.We validate our proposed techniques with numerical experiments for some large‐scale stochastic differential equations driven by fBm.

Nonstationary fractionally integrated functional time series

We study a functional version of nonstationary fractionally integrated time series, covering the functional unit root as a special case. The time series taking values in an infinite-dimensional separable Hilbert space are projected onto a finite number of sub-spaces, the level of nonstationarity allowed to vary over them. Under regularity conditions, we derive a weak convergence result for the projection of the fractionally integrated functional process onto the asymptotically dominant sub-space, which retains most of the sample information carried by the original functional time series. Through the classic functional principal component analysis of the sample variance operator, we obtain the eigenvalues and eigenfunctions which span a sample version of the dominant sub-space. Furthermore, we introduce a simple ratio criterion to consistently estimate the dimension of the dominant sub-space, and use a semiparametric local Whittle method to estimate the memory parameter. Monte-Carlo simulation studies are given to examine the finite-sample performance of the developed techniques.

Structure-preserving model order reduction for K-power bilinear systems via Laguerre functions

ABSTRACT This paper presents a series of structure-preserving model order reduction algorithms for K-power bilinear systems via Laguerre functions. The method first aims to rewritten the K-power bilinear system as a general bilinear system and calculate the approximate low-rank factors of the cross Gramian of the bilinear system by combining the idea of Laguerre functions expansion of the matrix exponential function. After that, the approximate balanced system of the K-power bilinear system is constructed by the corresponding projection transformation of each subsystem. In order to achieve the purpose of model order reduction, the states with smaller singular values are then truncated, so as to further obtain the reduced order model. For this approach, there is a disadvantage that unstable systems may be generated although the original one is stable. To alleviate the inadequacies of this approach, we have improved the model reduction procedure, which is based upon the dominant subspace projection method. In addition, we also carried out a correlation analysis on the stability of improved algorithms. Finally, numerical experiments are employed to substantiate the effectiveness of the presented algorithms.

Strategy for Radar-Embedded Communication Waveform Design Based on Singular Value Decomposition

Radar-embedded communication (REC) is a new type of covert communication whereby communication signals are embedded in radar backscattered echoes. Conventional REC waveform design strategies generally involve transforming the radar sampling sequence into a square matrix that can be eigenvalue decomposed; however, it may lose some original characteristics of the radar signal during this process. In this paper, we propose two REC waveform design methods based on singular value decomposition (SVD), namely, SVD-NDP (non-dominant-processing waveform based on SVD) and SVD-SNDP (shaped-non-dominant-processing waveform based on SVD), and derive the similarities between the radar signal and the proposed waveforms. We then simulate the anti-detection performance, anti-interception performance, and communication reliability of the proposed waveforms and analyze the degrees of freedom and computational complexity of their design. The results show that, compared with conventional REC waveforms, the proposed waveforms could improve communication reliability without sacrificing low probability of intercept (LPI) performance and at least double the design degrees of freedom. SVD-NDP and SVD-SNDP algorithms can reduce computational complexity by more than 30% compared with conventional waveform design algorithms based on eigenvalue decomposition when the number of communication waveforms (symbols) is set to 4 (2 bits), and the proportion of the dominant subspace is set to 50%.

Gramian-based model reduction for unstable stochastic systems

This paper considers large-scale linear stochastic systems representing, e.g., spatially discretized stochastic partial differential equations. Since asymptotic stability can often not be ensured in such a stochastic setting (e.g., due to larger noise), the main focus is on establishing model order reduction (MOR) schemes applicable to unstable systems. MOR is vital to reduce the dimension of the problem in order to lower the enormous computational complexity of for instance sampling methods in high dimensions. In particular, a new type of Gramian-based MOR approach is proposed in this paper that can be used in very general settings. The considered Gramians are constructed to identify dominant subspaces of the stochastic system as pointed out in this work. Moreover, they can be computed via Lyapunov equations. However, covariance information of the underlying systems enters these equations which is not directly available. Therefore, efficient sampling-based methods relying on variance reduction techniques are established to derive the required covariances and hence the Gramians. Alternatively, an ansatz to compute the Gramians by deterministic approximations of covariance functions is investigated. An error bound for the studied MOR methods is proved yielding an a priori criterion for the choice of the reduced system dimension. This bound is new and beneficial even in the deterministic case. The paper is concluded by numerical experiments showing the efficiency of the proposed MOR schemes.