Block Circulant Matrices Research Articles

Limit Theorems for Spectra of Circulant Block Matrices with Large Random Blocks

This paper investigates the spectral properties of block circulant matrices with high-order symmetric (or Hermitian) blocks. We analyze cases with dependent or sparse independent entries within these blocks. Additionally, we analyze the distribution of singular values for the product of independent circulant matrices with non-Hermitian blocks.

On iterative methods based on Sherman-Morrison-Woodbury splitting

We consider a new splitting based on the Sherman-Morrison-Woodbury formula, which is particularly effective with iterative methods for the numerical solution of large linear systems and especially for systems involving matrices that are perturbations of circulant or block circulant matrices. Such matrices typically arise in the discretization of differential equations using finite element or finite difference methods. We prove the convergence of the new iteration without making any assumptions regarding the symmetry or diagonal-dominance of the matrix, which are limiting factors for most classical iterative methods.To illustrate the efficacy of the new iteration we present various applications. These include extensions of the new iteration to block matrices that arise in certain saddle point problems as well as two-dimensional finite difference discretizations. The new method exhibits fast convergence in all of the test cases we used. It has minimal storage requirements, straightforward implementation and compatibility with nearly circulant matrices via the Fast Fourier Transform. Remarkably, the new method was tested against very large matrices demonstrating extremely fast convergence. For these reasons it can be a valuable tool for the solution of various finite elements and finite differences discretizations of differential equations.

XOR count and block circulant MDS matrices over finite commutative rings

<p>Block circulant MDS matrices are used in the design of linear diffusion layers for lightweight cryptographic applications. Most of the work on construction of block circulant MDS matrices focused either on finite fields or $ GL(m, \mathbb{F}_2) $. The main objective of this paper is to extend the above study of block circulant MDS matrices to finite commutative rings. Additionally, we examine the behavior of the XOR count distribution under different reducible polynomials of equal degree over $ \mathbb{F}_2 $. We show that the determinant of a block circulant matrix over a ring can be expressed in a simple form. We construct $ 4 \times 4 $ and $ 8 \times 8 $ block circulant matrices over a ring. Furthermore, for non-negative integer $ l $, we identify the conditions under which a ring $ \mathfrak{R}_l = \frac{\mathbb{F}_2[x]}{\langle (f(x))^{2^l} \rangle} $, contains a finite field of order $ 2^m $, where $ f(x) $ is an irreducible polynomial of degree $ m $. To facilitate efficient implementation, we analyze XOR distributions within specific rings, such as $ R_1 = \frac{\mathbb{F}_2[x]}{\langle (1+x^2+x^6) \rangle} $ and $ R_2 = \frac{\mathbb{F}_2[x]}{\langle (1+x^4+x^6) \rangle} $. Our calculations reveal distinct XOR distributions when utilizing two reducible polynomials of equal degree, with XOR count distributions 776 and 764, respectively. However, when using irreducible polynomials of the same degree, the XOR count distributions remain the same. This difference is advantageous for applications in lightweight cryptography.</p>

Real block-circulant matrices and DCT-DST algorithm for transformer neural network

In the encoding and decoding process of transformer neural networks, a weight matrix-vector multiplication occurs in each multihead attention and feed forward sublayer. Assigning the appropriate weight matrix and algorithm can improve transformer performance, especially for machine translation tasks. In this study, we investigate the use of the real block-circulant matrices and an alternative to the commonly used fast Fourier transform (FFT) algorithm, namely, the discrete cosine transform–discrete sine transform (DCT-DST) algorithm, to be implemented in a transformer. We explore three transformer models that combine the use of real block-circulant matrices with different algorithms. We start from generating two orthogonal matrices, U and Q. The matrix U is spanned by the combination of the reals and imaginary parts of eigenvectors of the real block-circulant matrix, whereas Q is defined such that the matrix multiplication QU can be represented in the shape of a DCT-DST matrix. The final step is defining the Schur form of the real block-circulant matrix. We find that the matrix-vector multiplication using the DCT-DST algorithm can be defined by assigning the Kronecker product between the DCT-DST matrix and an orthogonal matrix in the same order as the dimension of the circulant matrix that spanned the real block circulant. According to the experiment's findings, the dense-real block circulant DCT-DST model with largest matrix dimension was able to reduce the number of model parameters up to 41%. The same model of 128 matrix dimension gained 26.47 of BLEU score, higher compared to the other two models on the same matrix dimensions.

New self-dual codes from $ 2 \times 2 $ block circulant matrices, group rings and neighbours of neighbours

<p style='text-indent:20px;'>In this paper, we construct new self-dual codes from a construction that involves a unique combination; <inline-formula><tex-math id="M1">\begin{document}$ 2 \times 2 $\end{document}</tex-math></inline-formula> block circulant matrices, group rings and a reverse circulant matrix. There are certain conditions, specified in this paper, where this new construction yields self-dual codes. The theory is supported by the construction of self-dual codes over the rings <inline-formula><tex-math id="M2">\begin{document}$ \mathbb{F}_2 $\end{document}</tex-math></inline-formula>, <inline-formula><tex-math id="M3">\begin{document}$ \mathbb{F}_2+u \mathbb{F}_2 $\end{document}</tex-math></inline-formula> and <inline-formula><tex-math id="M4">\begin{document}$ \mathbb{F}_4+u \mathbb{F}_4 $\end{document}</tex-math></inline-formula>. Using extensions and neighbours of codes, we construct <inline-formula><tex-math id="M5">\begin{document}$ 32 $\end{document}</tex-math></inline-formula> new self-dual codes of length <inline-formula><tex-math id="M6">\begin{document}$ 68 $\end{document}</tex-math></inline-formula>. We construct 48 new best known singly-even self-dual codes of length 96.</p>

Поиск информационных совокупностей при исправлении пакетов ошибок квазициклическими кодами

This article discusses the question of assessing the probability of finding information sets in block-permutation and block-circulant matrices. Traditionally, interference-resistant coding is considered independent errors, however, in real systems they can be grouped and generate a so-called error burst. Known estimates of the probability of finding information sets are conducted for random matrices, and for correcting error bursts widespread block-permutation low density parity check codes (LDPC-codes) or block-circulant quasi cyclic codes (QC-codes) can be used. To estimate the probability of finding information sets mathematical modeling was used. Experiments have been carried out to identify parameters for specific structures that give the greatest probability of finding information sets. The article presents the results reflecting certain features in the values of the probability of finding information sets for matrices of different types, given assumptions and hypotheses about the features. Dependence of the presence of an information set from the size and location of its search interval inside the block permutation matrix was identified. The results of this research may be used to reduce the complexity of decoding by information sets, which, when considering random matrices, is exponential.

The Equitable Chromatic Bounds on Splitting of Block Circulant Graphs

An equitable vertex coloring for the splitting of block circulant graphs is investigated. The block circulant graphs comprises block circulant matrices, where each block is itself a matrix. These blocks in each row are cyclically shifted one place to the right from those of the previous row. We approached such block circulant graphs in matrix representation and derived their independent sets using the neighbourhoods of each vertex. This classification makes the vertex coloring process to be simpler and equitable in most cases. In this framework, the equitable chromatic numbers are obtained for splitting on block circulant graphs, namely prism, antiprism, crossed and closed sun graphs.

Sampling with Derivatives in Periodic Shift-Invariant Spaces

We study the sampling problem involving derivatives in periodic shift-invariant spaces. We provide a necessary and sufficient condition for periodic nonuniform stable sampling with derivatives, based on block circulant matrices. Additionally, we derive an iterative algorithm to reconstruct a function from its irregular samples involving derivatives.

A Scalable Strategy for the Identification of Latent-Variable Graphical Models

In this article, we propose an identification method for latent-variable graphical models associated with autoregressive (AR) Gaussian stationary processes. The identification procedure exploits the approximation of AR processes through stationary reciprocal processes thus benefiting of the numerical advantages of dealing with block-circulant matrices. These advantages become more and more significant as the order of the process gets large. We show how the identification can be cast in a regularized convex program and we present numerical examples that compares the performances of the proposed method with the existing ones.

Generalized Approximate Message Passing Equalization for Multi-Carrier Faster-Than-Nyquist Signaling

Multi-carrier faster-than-Nyquist (MFTN) signaling constitutes a promising spectrally efficient non-orthogonal physical layer waveform. In this correspondence, we propose a pair of low-complexity generalized approximate message passing (GAMP)-based frequency-domain equalization (FDE) algorithms for MFTN systems operating in multipath channels. To mitigate the ill-condition of the resultant equivalent channel matrix, we construct block circulant interference matrices by inserting a few cyclic postfixes, followed by truncating the duration of the inherent two-dimensional interferences. Based on the decomposition of the block circulant matrices, we develop a novel frequency-domain received signal model using the two-dimensional fast Fourier transform for mitigating the colored noise imposed by the non-orthogonal matched filter. Moreover, we derive a GAMP-based FDE algorithm and its refined version, where the latter relies on approximations for circumventing the emergence of the ill-conditioned matrices. Our simulation results demonstrate that, for a fixed spectral efficiency, MFTN signaling can significantly improve the bit error rate (BER) performance by jointly optimizing the time- and frequency-domain packing factors. Compared to its Nyquist-signaling counterpart, our proposed MFTN systems employing the refined GAMP equalizer can achieve about 39% higher transmission rates at a negligible BER performance degradation.

Some families of integral mixed graphs

A mixed graph Gˆ is a graph where two vertices can be connected by an edge or by an arc (directed edge). The adjacency matrix, Aˆ(Gˆ), of a mixed graph has rows and columns indexed by the set of vertices of Gˆ, being its {u,v}-entry equal to 1 (respectively, −1) if the vertex u is connected by an edge (respectively, an arc) to the vertex v, and 0 otherwise. These graphs are called integral mixed graphs if the eigenvalues of its adjacency matrix are integers. In this paper, symmetric block circulant matrices are characterized, and as a consequence, the definition of a mixed graph to be a block circulant graph is presented. Moreover, using this concept and the concept of a g-circulant matrix, the construction of a family of undirected graphs that are integral block circulant graphs is shown. These results are extended using the notion of H-join operation to characterize the spectrum of a family of integral mixed graphs. Furthermore, a new binary operation called mixed asymmetric product of mixed graphs is introduced, and the notions of joining by arcs and joining by edges are used, allowing us to obtain a new integral mixed graph from two original integral mixed graphs.

Optimal block circulant preconditioners for block Toeplitz systems with application to evolutionary PDEs

In this work, we propose a preconditioned minimal residual (MINRES) method for a class of non-Hermitian block Toeplitz systems. Namely, considering an mn-by-mn non-Hermitian block Toeplitz matrix T(n,m) with m-by-m commuting Hermitian blocks, we first premultiply it by a simple permutation matrix to obtain a Hermitian matrix and then construct a Hermitian positive definite block circulant preconditioner for the modified matrix. Under certain conditions, we show that the eigenvalues of the preconditioned matrix are clustered around ±1 when n is sufficiently large. Due to the Hermitian nature of the modified matrix, MINRES with our proposed preconditioner can achieve theoretically guaranteed superlinear convergence under suitable conditions. In addition, we provide several useful properties of block circulant matrices with commuting Hermitian blocks, including diagonalizability and symmetrization. A generalization of our result to the multilevel block case is also provided. We in particular indicate that our work can be applied to the all-at-once systems arising from solving evolutionary partial differential equations. Numerical examples are given to illustrate the effectiveness of our preconditioning strategy.

Parametric Bilinear Iterative Generalized Approximate Message Passing Reception of FTN Multi-Carrier Signaling

A low-complexity parametric bilinear generalized approximate message passing (PBiGAMP)-based receiver is conceived for multi-carrier faster-than-Nyquist (MFTN) signaling over frequency-selective fading channels. To mitigate the inherent ill-conditioning problem of MFTN signaling, we construct a segment-based frequency-domain received signal model in the form of a block circulant linear transition matrix, which can be efficiently calculated by applying a two dimensional fast Fourier transform. Based on the eigenvalue decomposition of the block circulant matrices, we can diagonalize the covariance matrix of the complex-valued colored noise process imposed by the associated two dimensional non-orthogonal matched filtering. Building on this model, a PBiGAMP-based parametric joint channel estimation and equalization (JCEE) algorithm is proposed for MFTN systems. In this algorithm, we introduce a pair of additive terms for characterizing the interferences arising from adjacent segments and employ the exact discrete <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> probabilities of the transmitted symbols for improving the bit error rate (BER) performance. To further enhance the system’s robustness in the presence of ill-conditioned matrices, we develop a refined PBiGAMP-based JCEE algorithm by introducing a series of scaled identity matrices. Moreover, the proposed PBiGAMP-based JCEE algorithms may be readily decomposed into GAMP-based equalization algorithms, when the channel state information is perfectly known. The overall complexity of the proposed algorithms only increases logarithmically with the total number of transmitted symbols. Our simulation results demonstrate the benefits of the proposed PBiGAMP-based iterative message passing receiver conceived for MFTN signaling.

New extremal binary self-dual codes from block circulant matrices and block quadratic residue circulant matrices

In this paper, we construct self-dual codes from a construction that involves both block circulant matrices and block quadratic residue circulant matrices. We provide conditions when this construction can yield self-dual codes. We construct self-dual codes of various lengths over F2 and F2+uF2. Using extensions, neighbours and sequences of neighbours, we construct many new self-dual codes. In particular, we construct one new self-dual code of length 66 and 51 new self-dual codes of length 68.

Block circulant matrices and the spectra of multivariate stationary sequences

Abstract Given a weakly stationary, multivariate time series with absolutely summable autocovariances, asymptotic relation is proved between the eigenvalues of the block Toeplitz matrix of the first n autocovariances and the union of spectra of the spectral density matrices at the n Fourier frequencies, as n → ∞. For the proof, eigenvalues and eigenvectors of block circulant matrices are used. The proved theorem has important consequences as for the analogies between the time and frequency domain calculations. In particular, the complex principal components are used for low-rank approximation of the process; whereas, the block Cholesky decomposition of the block Toeplitz matrix gives rise to dimension reduction within the innovation subspaces. The results are illustrated on a financial time series.

Exploring GPU acceleration of Deep Neural Networks using Block Circulant Matrices

Training a Deep Neural Network (DNN) is a significant computing task since it places high demands on computing resources and memory bandwidth. Many approaches have been proposed to compress the network, while maintaining high model accuracy, reducing the computational demands associated with large-scale DNN training. One attractive approach is to leverage Block Circulant Matrices (BCM), compressing the linear transformation layers, e.g., convolutional and fully-connected layers, that heavily rely on performing General Matrix Multiplications (GEMM). By using BCMs, we can reduce the weight storage for a linear transformation layer from O(N2) to O(N). BCMs are also more efficient in terms of computational complexity, improving algorithmic complexity from O(N2) to O(Nlog(N)).Previous work has only evaluated DNNs using BCMs targeting FPGAs for inference. There has been little prior work that considers the potential benefits of using BCMs for accelerating DNN training on GPUs. In this paper, we explore acceleration of DNNs using BCM on a state-of-the-art GPU. First, we identify the challenges posed by using BCMs. Next, we perform both general and GPU-specific optimizations that impact: (i) the decomposition and interaction of individual operations, and (ii) the overall GPU kernel design. We modify the algorithmic steps to remove redundant computations, while maintaining mathematical integrity. We also leverage multiple GPU kernel optimizations, considering performance factors, such as occupancy, data sharing/reuse patterns, and memory coalescing. We evaluate the performance of DNN training on an NVIDIA Tesla V100, providing insights into the benefits of our proposed kernel optimizations on a state-of-the-art GPU. Based on our results, we can achieve average speedups of 1.31× and 2.79× for the convolutional layers and fully-connected layers, respectively for AlexNet. We can also achieve average speedups of 1.33× and 3.66× for the convolutional layers and fully-connected layers, respectively for VGGNet-16.

Some Results on the Field of Values of Matrix Polynomials

In this paper, the notions of pseudofield of values and joint pseudofield of values of matrix polynomials are introduced and some of their algebraic and geometrical properties are studied. Moreover, the relationship between the pseudofield of values of a matrix polynomial and the pseudofield of values of its companion linearization is stated, and then some properties of the augmented field of values of basic A-factor block circulant matrices are investigated.

The tensor t‐function: A definition for functions of third‐order tensors

SummaryA definition for functions of multidimensional arrays is presented. The definition is valid for third‐order tensors in the tensor t‐product formalism, which regards third‐order tensors as block circulant matrices. The tensor function definition is shown to have similar properties as standard matrix function definitions in fundamental scenarios. To demonstrate the definition's potential in applications, the notion of network communicability is generalized to third‐order tensors and computed for a small‐scale example via block Krylov subspace methods for matrix functions. A complexity analysis for these methods in the context of tensors is also provided.

Symmetry group factorization reveals the structure-function relation in the neural connectome of Caenorhabditis elegans

The neural connectome of the nematode Caenorhabditis elegans has been completely mapped, yet in spite of being one of the smallest connectomes (302 neurons), the design principles that explain how the connectome structure determines its function remain unknown. Here, we find symmetries in the locomotion neural circuit of C. elegans, each characterized by its own symmetry group which can be factorized into the direct product of normal subgroups. The action of these normal subgroups partitions the connectome into sectors of neurons that match broad functional categories. Furthermore, symmetry principles predict the existence of novel finer structures inside these normal subgroups forming feedforward and recurrent networks made of blocks of imprimitivity. These blocks constitute structures made of circulant matrices nested in a hierarchy of block-circulant matrices, whose functionality is understood in terms of neural processing filters responsible for fast processing of information.

Matrix Analysis of Repetitive Circulant Structures: New-block and Near Block Matrices

In many scientific fields and several problems, Block Circulant Matrices (BCM) have been used for a long period of time. Each row of the BCM is a cyclic shift of its upper row to the right. BCM has been studied widely and there are closed-form solutions for problems of BCM. In these problems, the properties of near-BCM and BCM lead to a significant decrease in computational cost and efforts. In other words, these matrices are useful to perform some computational operations at the low cost. This study introduces a method for transforming a structure into a new type of Block Circulant Structure (BCS) by applying minor modifications. Furthermore, transformation of structural matrices into Block Circulant Matrices is discussed, and the properties of these matrices are then described in details. The methods introduce calculating eigenvalues and eigenvectors of these matrices instead of calculating the inverse of their matrices. To achieve this goal, the properties of near-Block and Block Circulant Matrices are used to analyze the structural stiffness matrices. In addition, the inverse of stiffness matrices for structures are calculated and utilized in structural mechanics. For clarification of efficiency and accuracy of the method, some examples are presented.