Row Stochastic Matrix Research Articles

On the distributed optimization over directed networks

In this paper, we propose a distributed algorithm, called Directed-Distributed Subgradient Descent (D-DSD), to solve multi-agent optimization problems over directed graphs. Existing algorithms mostly deal with similar problems under the assumption of undirected networks, i.e., requiring the weight matrices to be doubly-stochastic. The row-stochasticity of the weight matrix guarantees that all agents reach consensus, while the column-stochasticity ensures that each agent’s local (sub)gradient contributes equally to the global objective. In a directed graph, however, it may not be possible to construct a doubly-stochastic weight matrix in a distributed manner. We overcome this difficulty by augmenting an additional variable for each agent to record the change in the state evolution. In each iteration, the algorithm simultaneously constructs a row-stochastic matrix and a column-stochastic matrix instead of only a doubly-stochastic matrix. The convergence of the new weight matrix, depending on the row-stochastic and column-stochastic matrices, ensures agents to reach both consensus and optimality. The analysis shows that the proposed algorithm converges at a rate of O(lnkk), where k is the number of iterations.

Two-way learning with one-way supervision for gene expression data

BackgroundA family of parsimonious Gaussian mixture models for the biclustering of gene expression data is introduced. Biclustering is accommodated by adopting a mixture of factor analyzers model with a binary, row-stochastic factor loadings matrix. This particular form of factor loadings matrix results in a block-diagonal covariance matrix, which is a useful property in gene expression analyses, specifically in biomarker discovery scenarios where blood can potentially act as a surrogate tissue for other less accessible tissues. Prior knowledge of the factor loadings matrix is useful in this application and is reflected in the one-way supervised nature of the algorithm. Additionally, the factor loadings matrix can be assumed to be constant across all components because of the relationship desired between the various types of tissue samples. Parameter estimates are obtained through a variant of the expectation-maximization algorithm and the best-fitting model is selected using the Bayesian information criterion. The family of models is demonstrated using simulated data and two real microarray data sets. The first real data set is from a rat study that investigated the influence of diabetes on gene expression in different tissues. The second real data set is from a human transcriptomics study that focused on blood and immune tissues. The microarray data sets illustrate the biclustering family’s performance in biomarker discovery involving peripheral blood as surrogate biopsy material.ResultsThe simulation studies indicate that the algorithm identifies the correct biclusters, most optimally when the number of observation clusters is known. Moreover, the biclustering algorithm identified biclusters comprised of biologically meaningful data related to insulin resistance and immune function in the rat and human real data sets, respectively.ConclusionsInitial results using real data show that this biclustering technique provides a novel approach for biomarker discovery by enabling blood to be used as a surrogate for hard-to-obtain tissues.

Understanding Symmetric Smoothing Filters: A Gaussian Mixture Model Perspective.

Many patch-based image denoising algorithms can be formulated as applying a smoothing filter to the noisy image. Expressed as matrices, the smoothing filters must be row normalized, so that each row sums to unity. Surprisingly, if we apply a column normalization before the row normalization, the performance of the smoothing filter can often be significantly improved. Prior works showed that such performance gain is related to the Sinkhorn-Knopp balancing algorithm, an iterative procedure that symmetrizes a row-stochastic matrix to a doubly stochastic matrix. However, a complete understanding of the performance gain phenomenon is still lacking. In this paper, we study the performance gain phenomenon from a statistical learning perspective. We show that Sinkhorn-Knopp is equivalent to an expectation-maximization (EM) algorithm of learning a Gaussian mixture model of the image patches. By establishing the correspondence between the steps of Sinkhorn-Knopp and the EM algorithm, we provide a geometrical interpretation of the symmetrization process. This observation allows us to develop a new denoising algorithm called Gaussian mixture model symmetric smoothing filter (GSF). GSF is an extension of the Sinkhorn-Knopp and is a generalization of the original smoothing filters. Despite its simple formulation, GSF outperforms many existing smoothing filters and has a similar performance compared with several state-of-the-art denoising algorithms.

Numerical ranges of row stochastic matrices

In this paper, we consider properties of the numerical range of an n-by-n row stochastic matrix A. It is shown that the numerical radius of A satisfies 1≤w(A)≤(1+n)/2, and, moreover, w(A)=1 (resp., w(A)=(1+n)/2) if and only if A is doubly stochastic (resp.,A=[01⋮10]jth for some j, 1≤j≤n). A complete characterization of the A's for which the zero matrix of size n−1 can be dilated to A is also given. Finally, for each n≥2, we determine the smallest rectangular region in the complex plane whose sides are parallel to the x- and y-axis and which contains the numerical ranges of all n-by-n row stochastic matrices.

Containment control of discrete-time general linear multi-agent systems under dynamic digraph based on trajectory analysis

This paper devotes itself to the containment tracking problem of general linear time-varying discrete-time multi-agent systems (MASs). The convergence analysis is based on trajectory analysis rather than Lyapunov method. Nonnegative matrix theory, in particular the row-stochastic matrix properties, together with the relation between matrix and graph topology, are explored to handle the containment control problem. Based on a technical lemma concerning norm estimation of infinite product of row-stochastic matrices, we show that the followers can exponentially tend to the dynamical convex hull spanned by the leaders, i.e. the containment can be achieved under the very relaxed conditions: strictly instability of the individual-system of each agent (without interactions with other agents) is allowable and the graph topology is only required to be jointly having directed spanning forest frequently enough. Moreover, the least convergence rate is explicitly specified. Simulation is also provided to demonstrate the effectiveness of our result.

Ordering graphs in a normalized singular value measure

A proposed measure of network cohesion for graphs arising from interrelated economic activity is studied. The measure is the largest singular value of a rowstochastic matrix derived from the adjacency matrix. It is shown here that among graphs on n vertices, the star universally gives the (strictly) largest measure. Other universal comparisons among graphs with larger measures are difficult to make, but one is conjectured, and a selection of empirical evidence is given.

Distributed Control and Generation Estimation Method for Integrating High-Density Photovoltaic Systems

The presence of distributed generators (DGs) such as photovoltaic systems (PVs) is increasing significantly in distribution networks, and in order to accommodate a higher penetration of DGs, technical issues arising from fluctuation and unpredictability of their power output must be addressed. It is beneficial if DGs of high penetration can be dispatched when necessary. To this end, a distributed control and generation estimation approach is developed to dispatch multiple DGs, each of which consists of a PV and a controllable load. A strongly connected digraph with a row stochastic adjacency matrix is a sufficient requirement for the communication topology. A distributed weights adjustment algorithm adaptively makes the adjacency matrix doubly stochastic so that the aggregated power generation capacity can be estimated. Then, the expected consensus operational point of the DGs is calculated by those DGs that can obtain power dispatch command from the supervisory control and data acquisition system and is propagated to the rest of the DGs with a consensus algorithm. With this method, all the DGs operate at the same ratio of available power, while their aggregated power meets the power dispatch command. Simulations in the IEEE standard 34-bus distribution network verify the effectiveness of the proposed approach.

Product of Random Stochastic Matrices

The paper deals with the convergence properties of the products of random (row-)stochastic matrices. The limiting behavior of such products is studied from a dynamical system point of view. In particular, by appropriately defining a dynamic associated with a given sequence of random (row-)stochastic matrices, we prove that the dynamics admits a class of time-varying Lyapunov functions, including a quadratic one. Then, we discuss a special class of stochastic matrices, a class P*, which plays a central role in this work. We then study cut-balanced chains and using some geometric properties of these chains, we characterize the stability of a subclass of cut-balanced chains. As a special consequence of this stability result, we obtain an extension of a central result in the non-negative matrix theory stating that, for any aperiodic and irreducible row-stochastic matrix A, the limit lim <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k→∞</sub> A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sup> exists and it is a rank one stochastic matrix. We show that a generalization of this result holds not only for sequences of stochastic matrices but also for independent random sequences of such matrices.

Fuzzy spectral clustering by PCCA+: application to Markov state models and data classification

Given a row-stochastic matrix describing pairwise similarities between data objects, spectral clustering makes use of the eigenvectors of this matrix to perform dimensionality reduction for clustering in fewer dimensions. One example from this class of algorithms is the Robust Perron Cluster Analysis (PCCA+), which delivers a fuzzy clustering. Originally developed for clustering the state space of Markov chains, the method became popular as a versatile tool for general data classification problems. The robustness of PCCA+, however, cannot be explained by previous perturbation results, because the matrices in typical applications do not comply with the two main requirements: reversibility and nearly decomposability. We therefore demonstrate in this paper that PCCA+ always delivers an optimal fuzzy clustering for nearly uncoupled, not necessarily reversible, Markov chains with transition states.

The Projection Method for Reaching Consensus in Discrete-time Multiagent Systems

In the coordination problems for multi-agent systems, a well-known condition of achieving consensus is the presence of a spanning arborescence in the communication digraph. The paper deals with the discrete consensus problem in the case where this condition is not satisfied. Let P be a row stochastic influence matrix, whereas TP is the subspace of initial opinions that ensure consensus in DeGroot's iterative pooling model. We propose a method of coordination that consists of: (1) transformation of the vector of initial opinions into the closest vector in TP and (2) subsequent DeGroot process with the matrix P.

Socialization Networks and the Transmission of Interethnic Attitudes

This paper studies the dynamics of interethnic attitudes, focusing on the role played by the network of social influences, to which agents are exposed. A cultural evolution framework is introduced, where attitudes are modeled as continuous cultural traits transmitted intergenerationally. Oblique (out of family) socialization is modeled as a network of influences, which are represented by a row stochastic matrix. Properties of the dynamics are also examined. The main result states that any time independent oblique socialization matrix ensures convergence. In particular, I find that while the vertical (parental) socialization has the role of ensuring convergence without influencing the kind of steady state reached, the structure of the oblique socialization matrix does not affect convergence properties. However, it determines the class of the steady state, which is observed at the end of the dynamic. Specific oblique socialization mechanisms and steady states relevant for interethnic issues are then introduced showing how this model links them together. A method to potentially link any oblique socialization mechanism to a consequential steady state class is then proposed. Finally, data from the US areconsidered to show how it is possible to infer properties of the oblique socialization network.

Hierarchical mixture models for biclustering in microarray data

In the last few years, model-based clustering techniques have become widely used in the context of microarray data analysis. In this empirical context, a potential purpose for statistical approaches is the identification of clusters of genes that are co-expressed under subsets of experimental conditions. We discuss a hierarchical mixture model to combine advantages of allowing for dependence within gene clusters and for simultaneous clustering of genes and experimental conditions. Thanks to the adopted hierarchical structure, we may distinguish gene clusters from mixture components, where the latter may represent intra-cluster gene-specific extra-Gaussian departures. To cluster experimental conditions, instead, we suggest a suitable parameterization of component-specific means by using a binary row stochastic matrix representing condition membership. The performance of the proposed approach is discussed on both simulated and real datasets.

Linear Preservers of Majorization

For vectors X,Y ∈ R n ,w e sayX is left matrix majorized by Y and write X ≺� Y if for some row stochastic matrix R, X = RY. Also, we write X ∼� Y, when X ≺� Y ≺� X. A linear operator T : R p → R n is said to be a linear preserver of a given relation ≺ if X ≺ Y on Rp implies that TX≺ TY on R n . In this note we study linear preservers of ∼� from R p to R n . In particular, we characterize all linear preservers of ∼� from R 2 to R n , and also, all linear preservers of ∼� from R p to R p .

Row stochastic inverse eigenvalue problem

In this paper, we give sufficient conditions or realizability criteria for the existence of a row stochastic matrix with a given spectrum Λ = {λ1, ..., λ n } = Λ1 ∪ ⋯ ∪ Λ m ∪ Λm+1, m > 0; where (p k is an integer greater than 1), λk 1= λ k > 0, 1 = λ1 ≥ ω k > 0, k = 1, ..., m; Λm+1= {λ m +1}, ωm+1≡ λ1 + ..., +λ n ≤ λ1, ω k ≥ λ k , ω1 ≥ λ k , k = 2, ..., m + 1. In the case when p1, ..., p m are all equal to 2, Λ becomes a list of 2m + 1 real numbers for any positive integer m, and our result gives sufficient conditions for a list of 2m + 1 real numbers to be realizable by a row stochastic matrix. AMS classification: 15A18.

The Structure of linear preservers of left matrix majorization on R^T

For vectors X,Y ∈ Rn, Y is said to be left matrix majorized by X (Y ≺� X )i f for some row stochastic matrix R, Y = RX. A linear operator T:R p → R n is said to be a linear preserver of ≺� if Y ≺� X on R p implies that TY ≺� TXon R n . The linear operators T:R p → R n ( n<p (p−1)) which preserve ≺� have been characterized. In this paper, linear operators T:R p → R n which preserve ≺� are characterized without any condition on n and p.

Biclustering of Gene Expression Data by an Extension of Mixtures of Factor Analyzers

A challenge in microarray data analysis concerns discovering local structures composed by sets of genes that show homogeneous expression patterns across subsets of conditions. We present an extension of the mixture of factor analyzers model (MFA) allowing for simultaneous clustering of genes and conditions. The proposed model is rather flexible since it models the density of high-dimensional data assuming a mixture of Gaussian distributions with a particular omponent-specific covariance structure. Specifically, a binary and row stochastic matrix representing tissue membership is used to cluster tissues (experimental conditions), whereas the traditional mixture approach is used to define the gene clustering. An alternating expectation conditional maximization (AECM) algorithm is proposed for parameter estimation; experiments on simulated and real data show the efficiency of our method as a general approach to biclustering. The Matlab code of the algorithm is available upon request from authors.

Comments on "Jordan Canonical Form of the Google Matrix"

The Google matrix is a Web hyperlink matrix which is given by $P(\alpha)=\alpha P+(1-\alpha)E$, where $P$ is a row stochastic matrix, $E$ is a row stochastic rank-one matrix, and $0<\alpha<1$. In this paper we explore the analytic expression of the Jordan canonical form and point out that a theorem due to Serra-Capizzano (cf. Theorem 2.3 in [SIAM J. Matrix Anal. Appl., 27 (2005), pp. 305-312]) can be used for estimating the condition number of the PageRank vector as a function of $\alpha$ now viewed in the complex field. Furthermore, we give insight into a more efficient scaling matrix in order to minimize the condition number.

Linear preservers of left matrix majorization

For X,Y ∈ Mnm(R )( =Mnm), we say that Y is left (resp. right) matrix majorized by X and write Y ≺� X (resp. Y ≺r X )i fY = RX (resp. Y = XR) for some row stochastic matrix R. A linear operator T:Mnm → Mnm is said to be a linear preserver of a given relation ≺ on Mnm if Y ≺ X implies that TY ≺ TX. The linear preservers of ≺� or ≺r are fully characterized by A.M. Hasani and M. Radjabalipour. Here, we launch an attempt to extend their results to the case where the domain and the codomain of T are not necessarily identical. We begin by characterizing linear preservers T : Mp1 → Mn1 of ≺� .

On linear preservers of (right) matrix majorization

An n × m matrix A is said to be matrix majorized (or more precisely matrix majorized from the right) by an n × m matrix B, and write A ≺ B, if there exists a row stochastic matrix R such that A = BR. We characterize the linear operators that preserve the matrix majorization ≺.

The structure of linear operators strongly preserving majorizations of matrices

A matrix majorization relation A ≺r B (resp., A ≺ℓ B) on the collection Mn of all n × n real matrices is a relation A = BR (resp., A = RB) for some n × n row stochastic matrix R (depending on A and B). These right and left matrix majorizations have been considered by some authors under the names “matrix majorization” and “weak matrix majorization,” respectively. Also, a multivariate majorization A ≺rmul B (resp., A ≺ℓmul B) is a relation A = BD (resp., A = DB) for some n × n doubly stochastic matrix D (depending on A and B). The linear operators T : Mn → Mn which strongly preserve each of the above mentioned majorizations are characterized. Recall that an operator T : Mn → Mn strongly preserves a relation R on Mn when R(T(X), T(Y )) if and only if R(X, Y ). The results are the sharpening of well-known representations T X = CXD or T X = CXtD for linear operators preserving invertible matrices.