Abstract
This paper is concerned with a new expression of the so-called Pennington-Worah distribution, characterizing the asymptotic empirical eigenvalue distribution of some non linear random matrix ensembles. More precisely consider $M= \frac{1} {m} YY^{*}$ with $Y=f(WX)$ where $W$ and $X$ are random rectangular matrices with i.i.d. centered entries. The function $f$ is applied pointwise and can be seen as an activation function in (random) neural networks. The asymptotic empirical distribution of this ensemble has been computed in [16] and [3]. Here it is related to the Marcenko-Pastur distribution and information plus noise matrices.
Highlights
The scope of this article is to describe the limiting empirical eigenvalue distribution (e.e.d.) of some non linear random matrix ensembles considered in [16]. Such ensembles have been introduced as new approaches to understand deep learning using the theory of random matrices: we refer the reader to the above cited article as well as [9], [8], [14], and [13] for a more complete introduction to the subject
This can be related to the results of [2]
The intuition comes from the combinatorial argument we give in subsection 2.2
Summary
The scope of this article is to describe the limiting empirical eigenvalue distribution (e.e.d.) of some non linear random matrix ensembles considered in [16]. Theorem 1.4 has a similar flavor to that of [12], where kernel matrices are considered In both cases, the limiting empirical eigenvalue distribution can be related to that of a linear model of random matrices. This is in the same vein as [14] where the same phenomenon arises. Theorem 1.4 states that μf is related to the rectangular free convolution of the pushforward for both the Marchenko-Pastur distribution and the product Wishart distribution (see [6] Chapter 3 e.g.) This can be related to the results of [2]. The intuition comes from the combinatorial argument we give in subsection 2.2
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
