Chaos in heterogeneous neural networks: II. Multiple activity modes

Abstract

We study the activity of a recurrent neural network consisting of multiple cell groups through the structure of its correlations by showing how the rules that govern the strengths of connections between the different cell groups shape the average autocorrelation found in each group. We derive an analytical expression for the number of independent autocorrelation modes the network can concurrently sustain. Each mode corresponds to a non-zero component of the network’s autocorrelation, when it is projected on a specific set of basis vectors. In a companion abstract we derive a formula for the first mode, and hence the entire network, to become active. When the network is just above the critical point where it becomes active all groups of cells have the same autocorrelation function up to a constant multiplicative factor. We derive here a formula for this multiplicative factor which is in fact the ratio of the average firing rate of each group. As the effective synaptic gain grows a second activity mode appears, the autocorrelation functions of each group have different shapes, and the network becomes doubly chaotic. We generalize this result to understand how many modes of activity can be found in a heterogeneous network based on its connectivity structure. Finally, we use our theory to understand the dynamics of a clustered network where cells from the same group are strongly connected compared to cells from different groups. We show how this structure can lead to a one or more activity modes and interesting switching effects in the identity of the dominant cluster. To model the heteregenous network we include N neurons that are divided into D groups. The synaptic weight between neurons i,j is drawn from a centered distribution with standard deviations summarized in a D×D rule matrix N-1/2Gc(i)d(j)where c(i) is the group neuron i belongs to. The network obeys the standard rate dynamics (d/dt)xi=- xi+∑j=1...NJij tanhxj. The global behavior of the network changes according to the real part of the eigenvalues of a D×D matrix M whose c,d element is Mcd= N-1Nc(Gcd)2. When M's largest eigenvalue, Λ1, become larger than 1 the network become chaotic. The ratios of the components of the leading eigenvector V1are the ratios of the autocorrelations functions of the different groups (Figure 1 A). When Λ2 becomes larger than 1 the network is doubly chaotic (Figure 1 B). In general, the autocorrelation vector has a non-zero projection only on eigenvectors of M with eigenvalues greater than 1 (Figure 1 C) and hence the number of active modes in the network is equal to the number of eigenvalues of M that are larger than 1. Figure 1 (A) For an example network with 1200 neurons divided to 3 equally sized groups we plot the autocorrelation function averaged over neurons belonging to the same group. G was chosen such that one eigenvalue of M is greater than 1. Independent of the time ...