Stochastic approximation of eigenvectors and eigenvalues of the Q-symmetric expectation of a random matrix

Abstract

We establish an almost sure convergence theorem of the stochastic approximation process of Oja for estimating eigenvectors of the Q-symmetric expectation of a random matrix, under a correlation model between the incoming random matrices. This theorem generalizes previous theorems and extends them to the case where the metric Q is unknown and estimated online in parallel. We apply it to streaming principal component analysis of a random vector Z, when a mini-batch of observations of Z is used at each step or all the observations up to the current step. We deal with the case of streaming generalized canonical correlation analysis, with a metric estimated online in parallel.