Block Diagonal Matrix Research Articles

High-Dimensional Block Diagonal Covariance Structure Detection Using Singular Vectors

The assumption of independent subvectors arises in many aspects of multivariate analysis. In most real-world applications, however, we lack prior knowledge about the number of subvectors and the specific variables within each subvector. Yet, testing all these combinations is not feasible. For example, for a data matrix containing 15 variables, there are already 1 382 958 545 possible combinations. Given that zero correlation is a necessary condition for independence, independent subvectors exhibit a block diagonal covariance matrix. This paper focuses on the detection of such block diagonal covariance structures in high-dimensional data and therefore also identifies uncorrelated subvectors. Our approach exploits the fact that the structure of the covariance matrix is mirrored by the structure of its eigenvectors. However, the true block diagonal structure is masked by noise in the sample case. To address this problem, we propose to use sparse approximations of the sample eigenvectors to reveal the sparse structure of the population eigenvectors. Notably, the right singular vectors of a data matrix with an overall mean of zero are identical to the sample eigenvectors of its covariance matrix. Using sparse approximations of these singular vectors instead of the eigenvectors makes the estimation of the covariance matrix obsolete. We demonstrate the performance of our method through simulations and provide real data examples. Supplementary materials for this article are available online.

Synthesis of a parametrically robust controller with an extended state observer

This paper considers the problem of control of a parametrically uncertain dynamic system using an extended state observer. The control system is implemented using a two-loop scheme, where the outer loop ensures the fulfillment of the selected criterion for controlling the state vector, and the inner loop compensates for or reduces the influence of the vector of the total equivalent disturbance. The goal of the study is to develop a procedure for synthesizing an extended state observer taking into account the parametric uncertainty of the plant and the requirements for the closed loop of the combined control system specified in the frequency domain. Methods of control theory, robust control, and computer modeling are used for this study. The effect of the parametric uncertainty of the plant on its controlled motion is presented as a structured disturbance described in the form of a block-diagonal matrix. The concept of structured singular values is used to determine the measure of the system's robustness. Using the methodology of optimization of structured singular values, a procedure is proposed for synthesizing the extended state observer when considering a closed loop of a combined control system taking into account the parametric uncertainty of its mathematical model. The requirements ensuring the specified performance and stability of the closed loop of the control system taking into account the spectral properties of disturbances and sensor noise are formulated in the frequency domain using frequency-dependent weighting functions. For the synthesis of combined controllers based on the minimization of structured singular values, a D-G-L-K iteration algorithm is developed. The efficiency of the proposed approach is illustrated by a numerical example. Recommendations are given for the synthesis of parametrically robust combined controllers. The practical value of the obtained results lies in the fact that the procedure developed reduces the conservatism of robust combined control systems and, as a result, improves the control performance in the case of parametric uncertainty of the plant.

Analysis and comparison of two-level KFAC methods for training deep neural networks

As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix (FIM), efficient approximations are necessary to make NGD scalable to Deep Neural Networks (DNNs). Many such approximations have been attempted. The most sophisticated of these is KFAC, which approximates the FIM as a block-diagonal matrix, where each block corresponds to a layer of the neural network. By doing so, KFAC ignores the interactions between different layers. In this work, we investigate the interest of restoring some low-frequency interactions between the layers by means of two-level methods. Inspired from domain decomposition, several two-level corrections to KFAC using different coarse spaces are proposed and assessed. The obtained results show that incorporating the layer interactions in this fashion does not really improve the performance of KFAC. This suggests that it is safe to discard the off-diagonal blocks of the FIM, since the block-diagonal approach is sufficiently robust, accurate and economical in computation time.

Robust Tensor Learning for Multi-View Spectral Clustering

Tensor-based multi-view spectral clustering methods are promising in practical clustering applications. However, most of the existing methods adopt the ℓ2,1 norm to depict the sparsity of the error matrix, and they usually ignore the global structure embedded in each single view, compromising the clustering performance. Here, we design a robust tensor learning method for multi-view spectral clustering (RTL-MSC), which employs the weighted tensor nuclear norm to regularize the essential tensor for exploiting the high-order correlations underlying multiple views and adopts the nuclear norm to constrain each frontal slice of the essential tensor as the block diagonal matrix. Simultaneously, a novel column-wise sparse norm, namely, ℓ2,p, is defined in RTL-MSC to measure the error tensor, making it sparser than the one derived by the ℓ2,1 norm. We design an effective optimization algorithm to solve the proposed model. Experiments on three widely used datasets demonstrate the superiority of our method.

ON BLOCKWISE AND REFERENCE PANEL-BASED ESTIMATORS FOR GENETIC DATA PREDICTION IN HIGH DIMENSIONS.

Genetic prediction holds immense promise for translating genetic discoveries into medical advances. As the high-dimensional covariance matrix (or the linkage disequilibrium (LD) pattern) of genetic variants often presents a block-diagonal structure, numerous methods account for the dependence among variants in predetermined local LD blocks. Moreover, due to privacy considerations and data protection concerns, genetic variant dependence in each LD block is typically estimated from external reference panels rather than the original training data set. This paper presents a unified analysis of blockwise and reference panel-based estimators in a high-dimensional prediction framework without sparsity restrictions. We find that, surprisingly, even when the covariance matrix has a block-diagonal structure with well-defined boundaries, blockwise estimation methods adjusting for local dependence can be substantially less accurate than methods controlling for the whole covariance matrix. Further, estimation methods built on the original training data set and external reference panels are likely to have varying performance in high dimensions, which may reflect the cost of having only access to summary level data from the training data set. This analysis is based on novel results in random matrix theory for block-diagonal covariance matrix. We numerically evaluate our results using extensive simulations and real data analysis in the UK Biobank.

The distribution of the product of independent variance-gamma random variables

Let X and Y be independent variance-gamma random variables with zero location parameter; then the exact probability density function of the product XY is derived. Some basic distributional properties are also derived, including formulas for the cumulative distribution function and the characteristic function, as well as asymptotic approximations for the density, tail probabilities and the quantile function. As special cases, we deduce some key distributional properties for the product of two independent asymmetric Laplace random variables as well as the product of four jointly correlated zero mean normal random variables with a particular block diagonal covariance matrix. As a by-product of our analysis, we deduce some new reduction formulas for the Meijer G-function.

Research on synthesis method of weakly coupled three-translation parallel mechanism with large bearing capacity

Aiming at the problem of poor bearing capacity in the direction of decoupled DOF of weakly coupled three-translation parallel mechanism, a class of new configurations of large load-bearing capacity weakly coupled three-translation parallel mechanisms with closed-loop unit has been proposed in this paper. Based on the screw theory, configuration evolution method, and graphical approach method, the synthesis method of these kinds of mechanisms has been proposed, and the decoupled criterion of this mechanism and the addition conditions of the kinematic pair diagram are given. The synthesis process has been developed to synthesize the single degree of freedom translational decoupled branch chain and 2T coupled branch chain. To better apply the weakly coupled mechanism to large load occasions, a closed-loop unit construction method has been proposed. Based on the theory of singular value decomposition, a theoretical formula for increasing the load-bearing capacity of a mechanism by constructing a closed-loop unit with redundant drives has been derived for the first time. The closed-loop unit is added to the 2T coupled branch chain and the single DOF decoupled branch chain to construct the large load-bearing capacity single DOF decoupled three-translation parallel mechanism. Kinematic analysis and static analysis have been carried out on the synthesized mechanism. The Jacobian matrix of the synthesized mechanism is a block diagonal matrix, and the bearing capacity is significantly stronger than that of the decoupled mechanism, even higher than that of the coupled parallel mechanism. The synthesized weak coupling three translation parallel mechanism with large load bearing characteristics has a good application prospect in the field of high precision and large load bearing.

Low-complexity GFDM transmission scheme for low earth orbit satellite communications

As a promising multi-carrier modulation scheme, Generalized Frequency Division Multiplexing (GFDM) has certain advantages in low earth orbit satellite communications. However, the cost of GFDM to obtain these advantages is the self-interference caused by non-orthogonality and the increase of computational complexity. This paper proposes two low-complexity transceiver schemes for GFDM based on matrix sparsity. Firstly, the paper summarizes two GFDM modulation structures, referred to as structure types I and II, based on modulation data. Next, the modulation matrix in both structures is divided into sub-matrices, and the modulation sub-matrix is made sparse by leveraging the properties of the Fourier transform, so as to reduce the complexity of the transmitter. Similarly, by performing sparse processing on the received matrix and converting it into a block diagonal matrix form, the complexity of the receiver can be effectively reduced. The computational costs of our proposed schemes are analyzed in detail and compared with the existing solutions that are known to have the lowest complexity. The simulation results demonstrate that the two proposed schemes can provide satisfactory low-complexity performance depending on the time–frequency structure.

Omnibus Change Detection in Block Diagonal Covariance Matrix PolSAR Data Illustrated With Simulated and Sentinel-1 Data

Omnibus Change Detection in Block Diagonal Covariance Matrix PolSAR Data Illustrated With Simulated and Sentinel-1 Data

A general expression for vibrational Hamiltonians expressed in oblique coordinates

We examine the properties of oblique coordinates. The coordinates, introduced by Zúñiga et al. [J. Phys. B: At., Mol. Opt. Phys. 52, 055101, (2019)], reduce vibrational mode-mixing and enhance the quality of vibrational assignments in quantum mechanical investigations of two-dimensional model Hamiltonians. Oblique coordinates are obtained by making non-orthogonal rotations of the original coordinates that convert the matrix representation of the quadratic Hamiltonian operator into a block-diagonal matrix where the blocks are distinguished by the total quanta of vibrational excitation. Using techniques for the polar decomposition of matrices, we present a scheme for finding these coordinates for systems of arbitrary dimensions. Several molecular examples are presented that highlight the advantages of these coordinates.

Improved modal analyses using the novel quadrilateral overlapping elements

In this work, the novel quadrilateral overlapping element incorporated with the harmonic trigonometric functions is presented to reduce the efforts in mesh division for modal analysis. The new partition of unity functions in the overlapping element are cubic polynomials. Therefore, low-order polynomials can be utilized to augment local approximation space without the linear dependence problem. Moreover, the original polynomial approximation spaces are further enriched by the harmonic trigonometric functions without introducing additional rank deficiencies in the system matrices. Thus, the presented enriched overlapping finite element method (EOFEM) provides very precise eigenfrequencies and eigenmodes by using a very coarse mesh. However, the shape functions in the presented EOFEM do not possess the Kronecker delta function property. Hence, a conforming method that modifies the shape functions rather than the variational principle is proposed to impose the essential boundary conditions. Several numerical examples in two dimensions are then investigated to elaborate on the performance of the presented EOFEM in modal analyses. Besides, a block-diagonal lumped mass matrix is presented and further extended to the structure vibration analysis by incorporating the explicit time integration method. It is proven that the enriched overlapping element is computationally more efficient than the quadratic finite element and the original overlapping element.

A consistent group-theoretic transformation for the block-diagonalization of structural matrices

Symmetry properties of physical systems may be studied through symmetry groups. In recent times, group theory has found application in the study of various problems in structural mechanics, specifically bifurcation, buckling, kinematics and vibration. Computational simplifications are achieved by decomposing the vector space of the problem into smaller subspaces that are independent of each other. When the basis vectors of a subspace are used as the symmetry-adapted variables of that subspace, a smaller problem (associated with a matrix of smaller dimensions) automatically results. However, the same decomposition may be achieved by first obtaining the structural matrix of the system, and then transforming this into a non-overlapping block-diagonal matrix, each independent block being associated with a subspace of the problem. The advantage of this approach is its greater amenability to computer programming, but it does not always give the correct results unless a very specific procedure is followed. The purpose of this contribution is to present a consistent group-theoretic approach for the block diagonalization of structural matrices.

RLEKF: An Optimizer for Deep Potential with Ab Initio Accuracy

It is imperative to accelerate the training of neural network force field such as Deep Potential, which usually requires thousands of images based on first-principles calculation and a couple of days to generate an accurate potential energy surface. To this end, we propose a novel optimizer named reorganized layer extended Kalman filtering (RLEKF), an optimized version of global extended Kalman filtering (GEKF) with a strategy of splitting big and gathering small layers to overcome the O(N^2) computational cost of GEKF. This strategy provides an approximation of the dense weights error covariance matrix with a sparse diagonal block matrix for GEKF. We implement both RLEKF and the baseline Adam in our alphaDynamics package and numerical experiments are performed on 13 unbiased datasets. Overall, RLEKF converges faster with slightly better accuracy. For example, a test on a typical system, bulk copper, shows that RLEKF converges faster by both the number of training epochs (x11.67) and wall-clock time (x1.19). Besides, we theoretically prove that the updates of weights converge and thus are against the gradient exploding problem. Experimental results verify that RLEKF is not sensitive to the initialization of weights. The RLEKF sheds light on other AI-for-science applications where training a large neural network (with tons of thousands parameters) is a bottleneck.

On solving large-scale multistage stochastic optimization problems with a new specialized interior-point approach

A novel approach based on a specialized interior-point method (IPM) is presented for solving large-scale stochastic multistage continuous optimization problems, which represent the uncertainty in strategic multistage and operational two-stage scenario trees. This new solution approach considers a split-variable formulation of the strategic and operational structures. The specialized IPM solves the normal equations by combining Cholesky factorizations with preconditioned conjugate gradients, doing so for, respectively, the constraints of the stochastic formulation and those that equate the split-variables. We show that, for multistage stochastic problems, the preconditioner (i) is a block-diagonal matrix composed of as many shifted tridiagonal matrices as the number of nested strategic-operational two-stage trees, thus allowing the efficient solution of systems of equations; (ii) its complexity in a multistage stochastic problem is equivalent to that of a very large-scale two-stage problem. A broad computational experience is reported for large multistage stochastic supply network design (SND) and revenue management (RM) problems. Some of the most difficult instances of SND had 5 stages, 839 million linear variables, 13 million quadratic variables, 21 million constraints, and 3750 scenario tree nodes; while those of RM had 8 stages, 278 million linear variables, 100 million constraints, and 100,000 scenario tree nodes. For those problems, the proposed approach obtained the solution in 1.1 days using 174 gigabytes of memory for SND, and in 1.7 days using 83 gigabytes for RM; while CPLEX v20.1 required more than 53 days and 531 gigabytes for SND, and more than 19 days and 410 gigabytes for RM.

Exact test theory in Gaussian graphical models

In this paper, we derive several statistical tests on the precision matrix with application to the determination of the structure of an undirected Gaussian graph. The exact distributions of the test statistics are obtained under the null hypotheses, while the exact distributions of the random matrices, which are used in the construction of the test statistics, are deduced under the alternative hypothesis. Moreover, we present the high-dimensional asymptotic distributions of the test statistics under the null hypothesis. The testing problems that an undirected Gaussian graph possesses a structure that corresponds to the precision matrix of an AR(1) process, to the block-diagonal precision matrix and to the precision of a factor model are discussed in detail. The performance of the proposed statistical tests is further investigated via an extensive simulation study and compared to the benchmark approach.

Efficient Spectral Graph Convolutional Network Deployment on Memristive Crossbars

Graph Neural Networks (GNNs) have attracted increasing research interest for their remarkable capability to model graph-structured knowledge. However, GNNs suffer from intensive data exchange and poor data locality, which will cause critical performance and energy bottlenecks under conventional complementary metal oxide semiconductor (CMOS)-based von-Neumann computing architectures (graphics processing unit (GPU), central processing unit (CPU)) for the “Memory Wall” issue. Fortunately, memristive crossbar-based computation has emerged as one of the most promising neuromorphic computing architectures, which has been widely studied as the computing platform for convolutional neural network (CNNs), recurrent neural network (RNNs), spiking neural network (SNNs), etc. This paper proposes the deployment of spectral graph convolutional networks (GCNs) on memristive crossbars. Further, based on the structure of GCNs (extremely high sparsity and unbalanced non-zero data distribution) and the neuromorphic characteristics of memristive crossbar circuit, we propose the acceleration method that consists of Sparse Laplace Matrix Reordering and Diagonal Block Matrix Multiplication. The simulated experiment on memristor crossbars achieves 90.3% overall accuracy on the supervised learning graph dataset (QM7), and compared with the original computation, the proposed acceleration computing framework (with half-size diagonal blocks) achieves a 27.3% reduction of memristor number. Additionally, on the unsupervised learning dataset (Karate club), our method shows no loss of accuracy with half-size diagonal block mapping and reaches a 32.2% reduction of memristor number.

Nearly Optimal Block-Jacobi Preconditioning

The goal of Jacobi preconditioning of a symmetric positive definite matrix by a diagonal matrix is to choose to minimize the condition number . In 1969, van der Sluis proved that choosing so that the diagonal entries of are all ones reduces to within a factor of the minimum possible, where the factor depends on both the dimension and the norms used to define the condition number. We extend this result in two ways to block-Jacobi preconditioning, where is a block-diagonal matrix with blocks of given sizes, and we consider instead of to maintain the symmetric positive definite (spd) property. First, we extend van der Sluis’s original bound to include block-Jacobi. Second, we define a new norm in which choosing so that the corresponding diagonal blocks of are identity matrices minimizes the condition number. We use this to show that the condition number in the 2-norm of this optimally scaled is at least as large as the condition number in the new norm, and at most a factor larger, where is the number of diagonal blocks. We give an example where the optimal 2-norm condition number nearly attains this new upper bound, which for this example is tighter than van der Sluis’s bound by a factor equal to the matrix dimension. Finally, all these results generalize to the case of one-sided scaling of a full row-rank matrix .

G-Circulant Quantum Markov Semigroups

We broaden the study of circulant Quantum Markov Semigroups (QMS). First, we introduce the notions of [Formula: see text]-circulant GKSL generator and [Formula: see text]-circulant QMS from the circulant case, corresponding to [Formula: see text], to an arbitrary finite group [Formula: see text]. Second, we show that each [Formula: see text]-circulant GKSL generator has a block-diagonal representation [Formula: see text], where [Formula: see text] is a [Formula: see text]-circulant matrix determined by some [Formula: see text]. Denoting by [Formula: see text] the subgroup of [Formula: see text] generated by the support of [Formula: see text], we prove that [Formula: see text] has its own block-diagonal matrix representation [Formula: see text] where [Formula: see text] is an irreducible [Formula: see text]-circulant matrix and [Formula: see text] is the index of [Formula: see text] in [Formula: see text]. Finally, we exploit such block representations to characterize the structure, steady states, and asymptotic evolution of [Formula: see text]-circulant QMSs.

Semi-supervised non-negative matrix tri-factorization with adaptive neighbors and block-diagonal learning

Graph-regularized non-negative matrix factorization (GNMF) is proved to be effective for the clustering of nonlinear separable data. Existing GNMF variants commonly improve model performance by adding different additional constraints or refining the model factorization form, which can lead to problems such as increased algorithm complexity or insufficient performance release. In this paper, we propose semi-supervised non-negative matrix tri-factorization with adaptive neighbors and block-diagonal (ABNMTF). Different from existing methods, in ABNMTF the similarity graph matrix is learned from the original data by adaptive neighbors k-nearest model, and a block diagonal matrix is constructed based on a few labeled data to update the similarity matrix. Our approach reconstructs the block diagonal structure into the adaptive similarity matrix, which enables simultaneous learning of the similarity matrix and label binding during factorization, engendering a distinguishable subspace representation matrix and therefore improving the clustering performance without significantly increasing the complexity of the algorithm. We also represent an optimization method to solve the ABNMTF and provide analyses of convergence and computational complexity. Extensive experiments on 8 real image datasets show that the proposed algorithm reports superior performance against several state-of-the-art approaches. Code has been made available at:https://github.com/LstinWh/ABNMTF.

Multi-view subspace clustering network with block diagonal and diverse representation

Learning multi-view data, and especially multi-view data clustering, is a popular area in computer vision and pattern recognition. The multi-view subspace clustering has achieved a better clustering quality than the single-view subspace clustering, mainly because of the complementarity of multi-view information. First, for not directly pursuing a block diagonal representation matrix of previous ℓ1 or ℓ2 regularizers in a deep subspace clustering network, a k-block diagonal regularizer is proposed to replace traditional regularizers. This block diagonal representation module is integrated into this multi-view subspace clustering network, and it can improve the clustering quality. Secondly, there exists some redundancy among the representation matrices, and a diverse representation module can be introduced into this network. This can boost the diversity of representation matrices, and make learned representation matrices more discriminative and help improve the clustering performance. In this paper, based on the deep subspace clustering network, we integrate the block diagonal and diverse representation into the network, and a multi-view subspace clustering network with the block diagonal and the diverse representation is proposed. The experimental results on the UCI Digit, Caltech101-20, COIL100 and Caltech101-7 datasets have demonstrated the superior performance of the proposed algorithm over other popular multi-view algorithms.