Infinite Products Of Matrices Research Articles

Phase transitions for infinite products of large non-Hermitian random matrices

Phase transitions for infinite products of large non-Hermitian random matrices

Products of infinite upper triangular quadratic matrices

Let F be a field and q(x) a quadratic polynomial in F[x] with q(0)≠0. We denote by T∞(F) the algebra of all infinite upper triangular matrices over the field F. A matrix A∈T∞(F) is called a quadratic matrix with respect to q(x) if q(A)=0. In this paper, we first investigate the subgroup in T∞(F) generated by all quadratic matrices with respect to q(x) and then present some applications.

Convergence of deep ReLU networks

We explore convergence of deep neural networks with the popular ReLU activation function, as the depth of the networks tends to infinity. To this end, we introduce the notion of activation domains and activation matrices of a ReLU network. By replacing applications of the ReLU activation function by multiplications with activation matrices on activation domains, we obtain an explicit expression of the ReLU network. We then identify the convergence of the ReLU networks as convergence of a class of infinite products of matrices. Sufficient and necessary conditions for convergence of these infinite products of matrices are studied. As a result, we establish necessary conditions for ReLU networks to converge that the sequence of weight matrices converges to the identity matrix and the sequence of the bias vectors converges to zero as the depth of ReLU networks increases to infinity. Moreover, we obtain sufficient conditions in terms of the weight matrices and bias vectors at hidden layers for pointwise convergence of deep ReLU networks. These results provide mathematical insights to convergence of deep neural networks. Experiments are conducted to mathematically verify the results and to illustrate their potential usefulness in initialization of deep neural networks.

Convergence analysis of deep residual networks

Various powerful deep neural network architectures have made great contributions to the exciting successes of deep learning in the past two decades. Among them, deep Residual Networks (ResNets) are of particular importance because they demonstrated great usefulness in computer vision by winning the first place in many deep learning competitions. Also, ResNets are the first class of neural networks in the development history of deep learning that are really deep. It is of mathematical interest and practical meaning to understand the convergence of deep ResNets. We aim at studying the convergence of deep ResNets as the depth tends to infinity in terms of the parameters of the networks. Toward this purpose, we first give a matrix–vector description of general deep neural networks with shortcut connections and formulate an explicit expression for the networks by using the notion of activation matrices. The convergence is then reduced to the convergence of two series involving infinite products of non-square matrices. By studying the two series, we establish a sufficient condition for pointwise convergence of ResNets. We also conduct experiments on benchmark machine learning data to illustrate the potential usefulness of the results.

Bipartite Flocking Control for Multi-Agent Systems With Cooperation-Competition Interactions and Random Packet Dropouts

This paper addresses the bipartite flocking control problem of multi-agent systems with random packet dropouts, in which cooperative and competitive information interactions among agents are described by a signed network. The random packet dropouts are denoted by Bernoulli random variable and the information transmission failure between adjacent agents is possible at every time step. In this case, the unreliable information transmission within multi-agent systems may lead to behavioral dispersion and instability. To solve this problem, a novel fully distributed control scheme is designed. In light of the convergence method for products of infinite sub-stochastic matrices, the bipartite flocking control can be global almost sure to achieve and some essentially algebraic conditions are established. Finally, the effectiveness of the control scheme under random packet dropouts is illustrated by a numerical simulation example.

Asynchronous bipartite containment control of second-order multi-agent systems under switching topologies

This paper investigates the bipartite containment control problem for second-order multi-agent systems (MASs) via asynchronous sampling under switching topologies, where the relationships of both competition and cooperation exist among agents. Under asynchronous setting, each agent only receives its neighbors’ information at its own clock which is independent of the others’. The objective of bipartite containment control is to make the followers converge to a convex hull containing each leader’s trajectory as well as its opposite trajectory, which is the same as it in modulus but different in sign. First, the bipartite control problem is transformed into the converge problem of the product of infinite time-varying row stochastic matrices. Then, based on the theoretical tools including graph theory, nonnegative matrix theorem, and composition of binary relation, a sufficient condition is obtained for the bipartite containment control problem. It is shown that with proper parameters, the bipartite containment control can be achieved if the union of topology graphs related to any time intervals with given length has a directed spanning forest. Finally, the validity of the theoretical result under asynchronous sampling is demonstrated by a numerical simulation.

Convergence of deep convolutional neural networks

Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep networks with the Rectified Linear Unit (ReLU) activation function and with a fixed width. This does not cover the important convolutional neural networks where the widths are increased from layer to layer. For this reason, we first study convergence of general ReLU networks with increased widths and then apply the results obtained to deep convolutional neural networks. It turns out the convergence reduces to convergence of infinite products of matrices with increased sizes, which has not been considered in the literature. We establish sufficient conditions for convergence of such infinite products of matrices. Based on the conditions, we present sufficient conditions for pointwise convergence of general deep ReLU networks with increasing widths, and as well as pointwise convergence of deep ReLU convolutional neural networks.

The Ando-Li-Mathias geometric mean and infinite product of hollow matrices

In this paper we present a class of convergent infinite products of hollow matrices of the form These sequences of hollow matrices are obtained from the Ando-Li-Mathias matrix geometric mean of positive definite matrices. We further provide a three dimensional analytic manifold and an analytic self-map realizing such sequences.

Cucker-Smale flocking over cooperation-competition networks

This paper offers a novel distributed control scheme for the Cucker-Smale model to examine the leader-follower flocking behavior on networks with both cooperative and competitive interactions among agents. The core idea of the proposed scheme is to characterize the influence of cooperation and competition between agents through positive and negative weight functions with respect to the interaction distance, respectively. Based on the tool of infinite products of super-stochastic matrices, we establish mathematical inequalities about the intensities of cooperation and competition, and further obtain the algebraic conditions that depend on factors such as the initial states of agents, topological structure and weight functions. Moreover, we also discuss the influence of these factors on the least exponential convergence rate and the final position errors among agents. The obtained theoretical results are illustrated through simulation examples.

Collective Behavior of Multileader Multiagent Systems With Random Interactions Over Signed Digraphs

Signed digraphs have been developed in multiagent systems to explain cooperative and competitive interactions among agents. This article explores the collective behavior for a group of generic linear agents interacting by the random interaction manner on signed digraphs in the presence of multiple leaders. In the setting of random interactions, the information interaction between neighboring agents may fail at each time step, which is described by a Bernoulli random variable. The global almost sure convergence of the system is solved by using the convergence analysis method of products of infinite substochastic matrices. Moreover, an application about opinion dynamics of social networks is given. In the end, the theoretical research results are verified through simulation experiments.

Certifying Unstability of Switched Systems Using Sum of Squares Programming

The joint spectral radius (JSR) of a set of matrices characterizes the maximal asymptotic growth rate of an infinite product of matrices of the set. This quantity appears in a number of applications including the stability of switched and hybrid systems. A popular method used for the stability analysis of these systems searches for a Lyapunov function with convex optimization tools. We investigate dual formulations for this approach and leverage these dual programs for developing new analysis tools for the JSR. We show that the dual of this convex problem searches for the occupations measures of trajectories with high asymptotic growth rate. We both show how to generate a sequence of guaranteed high asymptotic growth rate and how to detect cases where we can provide lower bounds on the JSR. All results of this paper are presented for the general case of constrained switched systems, that is, systems for which the switching signal is constrained by an automaton.

Dynamic leader-following consensus for asynchronous sampled-data multi-agent systems under switching topology

This paper investigates the leader-following consensus problem for asynchronous sampled-data multi-agent systems with an active leader and under switching topology, in which the asynchrony means that each agent’s update actions are independent of the others’. First, the dynamic leader-following consensus problem for asynchronous sampled-data systems is transformed into the convergence problem of products of infinite general sub-stochastic matrices (PIGSSM), where the general sub-stochastic matrices are matrices with row sum no more than 1 but their elements are not necessarily nonnegative. We develop a method to cope with the corresponding convergence problem by matrix decomposition. In particular, we split the general sub-stochastic matrix into a sub-stochastic matrix which is a nonnegative matrix with row sum no more than 1, and a matrix with negative elements and row sum 0. Then based on a graphical approach and matrix analysis technique, we present a sufficient condition for the achievement of dynamic leader-following consensus in the asynchronous setting. Finally, simulation examples are demonstrated to verify the theoretical results.

An Entropy-Based Bound for the Computational Complexity of a Switched System

The joint spectral radius (JSR) of a set of matrices characterizes the maximal asymptotic growth rate of an infinite product of matrices of the set. This quantity appears in a number of applications including the stability of switched and hybrid systems. A popular method used for the stability analysis of these systems searches for a Lyapunov function with convex optimization tools. We analyse the accuracy of this method for constrained switched systems, a class of systems that has attracted increasing attention recently. We provide a new guarantee for the upper bound provided by the sum of squares implementation of the method. This guarantee relies on the p-radius of the system and the entropy of the language of allowed switching sequences. We end this paper with a method to reduce the computation of the JSR of low rank matrices to the computation of the constrained JSR of matrices of small dimension.

Persistent Flows in Deterministic Chains

This paper studies, the role of persistent flows in the convergence of infinite backward products of stochastic matrices of deterministic chains over networks with nonreciprocal interactions between agents. An arc describing the interaction strength between two agents is said to be persistent if its weight function has an infinite $l_1$ norm; convergence of the infinite backward products to a rank-one matrix of a deterministic chain of stochastic matrices is equivalent to achieving consensus at the node states. We discuss two balance conditions on the interactions between agents, which generalize the arc-balance and cut-balance conditions in the literature, respectively. The proposed conditions require that such a balance should be satisfied over each time window of a fixed length instead of at each time instant. We prove that in both cases global consensus is reached if and only if the persistent graph, which consists of all the persistent arcs, contains a directed spanning tree. The convergence rates of the system to consensus are also provided in terms of the interactions between agents having taken place. The results are obtained under a weak condition without assuming the existence of a positive lower bound of all the nonzero weights of arcs and are compared with the existing results. Illustrative examples are provided to validate the results and show the critical importance of the nontrivial lower boundedness of the self-confidence of the agents.

Generalized Sarymsakov Matrices

Within the set of stochastic, indecomposable, aperiodic (SIA) matrices, the class of Sarymsakov matrices is the largest known subset that is closed under matrix multiplication, and more critically whose compact subsets are all consensus sets. This paper shows that a larger subset with these two properties can be obtained by generalizing the standard definition for Sarymsakov matrices. The generalization is achieved by introducing the notion of the SIA index of a stochastic matrix, whose value is 1 for Sarymsakov matrices, and then exploring those stochastic matrices with larger SIA indices. In addition to constructing the larger set, this paper introduces another class of generalized Sarymsakov matrices, which contains matrices that are not SIA, and studies their products. Sufficient conditions are provided for an infinite product of matrices from this class, converging to a rank-one matrix. Finally, as an application of the results just described and to confirm their usefulness, a necessary and sufficient combinatorial condition, the “avoiding set condition,” for deciding whether or not a compact set of stochastic matrices is a consensus set is revisited. In addition, a necessary and sufficient combinatorial condition is established for deciding whether or not a compact set of doubly stochastic matrices is a consensus set.

On the Associativity of Infinite Matrix Multiplication

A natural definition of the product of infinite matrices mimics the usual formulation of multiplication of finite matrices with the caveat (in the absence of any sense of convergence) that the intersection of the support of each row of the first factor with the support of each column of the second factor must be finite. Multiplication is hence not completely defined, but restricted to a specific relation on infinite matrices. In order for the product of three infinite matrices A, B, and C to behave in an associative manner, the middle factor, B, must link A and C in three ways: (i) and must both be defined; (ii) and must both be defined; and, finally, (iii) must equal . In this article, these conditions are studied and are characterized in various ways by means of summability notions akin to those of formal calculus.

Cucker-Smale Flocking Control of Leader-Follower Multiagent Systems With Intermittent Communication

This paper investigates the Cucker-Smale flocking control problem of leader-follower multiagent systems in the presence of periodic intermittent communication, in which networked agents update the states by communicating with their neighbors during active periods and remain stable during dormancy periods. A distributed update algorithm that relies on the position error between agents is designed. Based on this algorithm, the problem of Cucker-Smale flocking with periodic intermittent communication is first transformed into a convergence problem of infinite products of nonnegative matrices. Then, this convergence problem is theoretically solved by using hybrid tools including nonnegative matrix analysis and graph theory. Finally, a sufficient condition, which relates to the weight function and the period length of intermittent communication is established. Moreover, numerical examples are presented to demonstrate the validity of the theoretical result.

Containment control for high‐order multi‐agent systems with heterogeneous time delays

This study addresses the containment control problem for high-order multi-agent systems with heterogeneous time delays. Here the authors consider two cases: the leaders are stationary and the leaders are dynamic. Based on the distributed containment control protocols with proper gain parameters, they utilise the properties of the product of infinite matrices to derive some sufficient conditions. The obtained results guarantee that all the followers can asymptotically converge to a convex hull spanned by those of the leaders under the proper gain parameters if the communication topology contains a directed spanning forest rooted at the leaders. Finally, some numerical simulations are presented to illustrate the effectiveness of their theoretical results.

Distributed containment control for asynchronous discrete-time second-order multi-agent systems with switching topologies

A distributed containment control problem for asynchronous discrete-time second-order multi-agent systems with switching topologies is studied in this paper, where asynchrony means that each agent only receives the state information of its neighbors at certain discrete time instants determined by its own clock that is independent of other agents. Based on a novel containment control protocol, the asynchronous system is transformed into a matrix-vector form, which implies that the asynchronous containment control problem can be converted to a convergence problem of the product of infinite time-varying nonnegative matrices whose all row sums are less than or equal to 1. Then the relations between switching communication topologies and the composite of binary relation are exploited to solve this convergence problem. Finally, we obtain a sufficient condition that all the followers can enter and keep moving in the convex hull formed by the leaders if the union of the effective communication topologies across any time intervals with some given length contains a spanning forest rooted at the leaders. Moreover, some simulation examples are presented for illustration.

Geometric representation of the infimax S-adic family

We construct geometric realizations for the infimax family of substitutions by generalizing the Rauzy-Canterini-Siegel method for a single substitution to the S-adic case. The composition of each countably infinite subcollection of substitutions from the family has an asymptotic fixed sequence whose shift orbit closure is an infimax minimal set $\Delta^+$. The subcollection of substitutions also generates an infinite Bratteli-Vershik diagram with prefix-suffix labeled edges. Paths in the diagram give the Dumont-Thomas expansion of sequences in $\Delta^+$ which in turn gives a projection onto the asymptotic stable direction of the infinite product of the Abelianization matrices. The projections of all sequences from $\Delta^+$ is the generalized Rauzy fractal which has subpieces corresponding to the images of symbolic cylinder sets. The intervals containing these subpieces are shown to be disjoint except at endpoints, and thus the induced map derived from the symbolic shift translates them. Therefore the process yields an Interval Translation Map, and the Rauzy fractal is proved to be its attractor.