State-dependent filtering as a mechanism toward visual robustness

We previously found the effect of colorfulness-adaptation in natural images. It was observed to be stronger in natural images than unnatural images, suggesting the influence of naturalness on the adaptation. However, what characteristics of images and what levels of visual system were involved were not examined enough. This research investigates whether the effect of colorfulness-adaptation is associated with spatial frequency components in natural images. If adaptation was a mechanism in early cortical level, the effect would be strong for adaptation and test images sharing similar spatial frequency components. In the experiment, we examined how the colorfulness impression of a test image changed following adaptation images with different levels of saturation. We selected several types of natural image from a standard image database for test and adaptation images. We also processed them to make shuffled images with spatial frequency component differed from the originals and phase-scrambled images with the component similar to the originals, for both adaptation and test images. Observers evaluated whether a test image was colorful or faded. Results show that the colorfulness perception of the test images was influenced by the saturation of the adaptation images. The effect was the strongest for the combination of natural (original) adaptation and natural test images regardless of image types. The effect for the combination of phase-scrambled images was weaker than those of original images and stronger than those of shuffled images. They suggest that not only the spatial frequency components of an image but also the recognition of images would contribute to colorfulness-adaptation.

A comparison of methods used to detect changes in neuronal discharge patterns

Aim: This study evaluates the assumption that global impression is created based on low spatial frequency components of posterior-anterior chest radiographs. Background: Expert radiologists precisely and rapidly allocate visual attention on pulmonary nodules chest radiographs. Moreover, the most frequent accurate decisions are produced in the shortest viewing time, thus, the first hundred milliseconds of image perception seems be crucial for correct interpretation. Medical image perception model assumes that during holistic analysis experts extract information based on low spatial frequency (SF) components and creates a mental map of suspicious location for further inspection. The global impression results in flagged regions for detailed inspection with foveal vision. Method: Nine chest experts and nine non-chest radiologists viewed two sets of randomly ordered chest radiographs under 2 timing conditions: (1) 300ms; (2) free search in unlimited time. The same radiographic cases of 25 normal and 25 abnormal digitalized chest films constituted two image sets: low-pass filtered and unfiltered. Subjects were asked to detect nodules and rank confidence level. MRMC ROC DBM analyses were conducted. Results: Experts had improved ROC AUC while high SF components are displayed (p=0.03) or while low SF components were viewed under unlimited time (p=0.02) compared with low SF 300mSec viewings. In contrast, non-chest radiologists showed no significant changes when high SF are displayed under flash conditions compared with free search or while low SF components were viewed under unlimited time compared with flash. Conclusion: The current medical image perception model accurately predicted performance for non-chest radiologists, however chest experts appear to benefit from high SF features during the global impression.

The right and left mammograms of a patient are assumed to be bilaterally symmetric for image readings. The detection of asymmetry in bilateral mammograms is a reliable indicator for detecting possible breast abnormalities. The purpose of this study was to examine the potential usefulness of a new method in terms of spatial frequency components for exploration of similarity and abnormality between the right and left mammograms. A total of 98 normal and 119 abnormal cases with calcifications were used for this study. Each case included two mediolateral oblique views. The spatial frequency components were determined from the symmetric regions in the right and left mammograms by Fourier transform. The degrees of conformity between the two spatial frequency components in the right and left mammograms were calculated for the same and different patients. The degrees of conformity were also examined for cases with and without calcifications for the same patient to show if the proposed method was useful for indicating the existence of calcifications or not. The average degrees of conformity and the standard deviations for the same and different patients were 0.911 ± 0.0165 and 0.857 ± 0.0328, respectively. The degrees of conformity calculated from abnormal cases (0.836 ± 0.0906) showed statistically lower values compared with those measured from normal cases (0.911 ± 0.0165). Our results indicated that histogram analysis of spatial frequency components could be useful as a similarity measure between bilateral mammograms for the same patient and abnormal signs in a mammogram.

We propose and demonstrate a novel image processing technique called spatial amplification where a set of selected spatial frequency components of an incident image is coherently amplified. In this technique, an incident image is first Fourier transformed onto a photorefractive crystal where it interacts with a nonuniform pump beam to selectively enhance the energy of certain spatial frequency components via two-beam coupling,1 and is then reimaged onto an output plane. The overlap of the pump and the Fourier transformed beam inside the crystal defines the type of image processing that is performed. For example, if only the higher-order frequency components corresponding to vertical lines are amplified, the process will yield vertical edge enhancement. With this description of spatial amplification, one can readily see its similarity to the well-known technique of spatial filtering. However, because it does not remove any spatial frequency components as is common in spatial filtering techniques, there are several advantages of spatial amplification such as high energy efficiency, high dynamic range, and cascadability. In our experiments, a photorefractive BaTiO3 crystal is used as the amplifying medium. The lack of phase crosstalk between the two interacting beams is recognized as the essential part of our technique.

In this paper, we propose an improved coding scheme for optical wireless communication systems using a LED array transmitter and a high-speed camera as the receiver on a vehicle. Previously, we have proposed a hierarchical coding scheme which allocated the data to spatial frequency components depending on the priority. In that scheme, the high-priority data can be received even if the receiver was far from the transmitter. We confirmed the advantage of the hierarchical coding scheme, but the bit error performance was not sufficient. In this paper, we divide the data into spatial frequency components, and use error correcting code for each spatial frequency components' data. Experimental evaluation demonstrates the improvement in BER performance. This improvement implies that the system range increased compared to the previous method.

Most digital color cameras use doubly refractive crystal devices as an optical low-pass filter. The filter reduces frequency components lower than the Nyquist frequency, and images are blurred. We previously presented a demosaicking method that simultaneously removes blurs caused by the optical low-pass filter. For the sharpening-demosaicking approach, the Bayer's RGB color filter array is not necessarily proper, and we studied another color-filter array, namely the WRB filter array, where the W-filtering means that all the visible light passes through it. Our prototypal sharpening-demosaicking method employed the iterative algorithm, and restored only spatial frequency components of color signals lower than the Nyquist frequency corresponding to the mosaicking pattern of the W filters. However, the same recovery problem is solved by a non-iterative method in the Discrete Fourier Transform domain. Moreover, our prototypal method often produced ringing artifacts near sharp color edges. To suppress those artifacts, we introduce the total-variation-based super-resolution into the sharpening-demosaicking approach. This super-resolution approach restores spatial frequency components higher than the Nyquist frequency from observed blurry spatial frequency components so that without producing ringing artifacts it can demosaic and sharpen images while preserving 1D image structures that intensity values are almost constant along the edge direction.

The impairment of cognitive function in Alzheimer’s disease is clearly correlated to synapse loss. However, the mechanisms underlying this correlation are only poorly understood. Here, we investigate how the loss of excitatory synapses in sparsely connected random networks of spiking excitatory and inhibitory neurons alters their dynamical characteristics. Beyond the effects on the activity statistics, we find that the loss of excitatory synapses on excitatory neurons reduces the network’s sensitivity to small perturbations. This decrease in sensitivity can be considered as an indication of a reduction of computational capacity. A full recovery of the network’s dynamical characteristics and sensitivity can be achieved by firing rate homeostasis, here implemented by an up-scaling of the remaining excitatory-excitatory synapses. Mean-field analysis reveals that the stability of the linearised network dynamics is, in good approximation, uniquely determined by the firing rate, and thereby explains why firing rate homeostasis preserves not only the firing rate but also the network’s sensitivity to small perturbations.

In conventional halftone screen technology, orthogonal screens are widely used. The screen angles for three colors of cyan (C), black (K) and magenta (M) are 15°, 45° and 75°. Another one yellow (Y) is 0° or 30°. Since Y crosses to M (C) with the shallow angle difference of 15° when Y is 0°, a low frequency moiré occurs in red (green), and it becomes an image defect. In the xerography process that uses the multiplex transfer of color toners, more remarkable moiré especially occurs as compared with offset printing. In order to reduce this moiré, “four-color harmonics screen” which sets spatial frequency components in four colors as a harmonics relation, has been developed. This is achieved by sharing a part of spatial frequency between two colors using a nonorthogonal screen. There are two kinds of combination of four colors that have such a relation. One is a ring coupling that shares one spatial frequency component between two colors respectively, and combines four colors to the shape of a ring. Another is a star coupling. In this case, three colors share three spatial frequency components of remainder one color respectively. In this paper, two screen-sets are introduced as each example of screen designing. In conventional halftoning, a color moire could not sufficiently be prevented, if a high-frequency screen like 300 lpi was not used. However, a color moiré can be prevented by “four-color harmonics screen” even if the screen-frequency is 150 to 200 lpi.

An Optimal Decision Population Code that Accounts for Correlated Variability Unambiguously Predicts a Subject’s Choice

Article Figures and data Abstract Editor's evaluation Introduction Results Discussion Materials and methods Appendix 1 Appendix 2 Appendix 3 Appendix 4 Appendix 5 Data availability References Decision letter Author response Article and author information Metrics Abstract To rapidly process information, neural circuits have to amplify specific activity patterns transiently. How the brain performs this nonlinear operation remains elusive. Hebbian assemblies are one possibility whereby strong recurrent excitatory connections boost neuronal activity. However, such Hebbian amplification is often associated with dynamical slowing of network dynamics, non-transient attractor states, and pathological run-away activity. Feedback inhibition can alleviate these effects but typically linearizes responses and reduces amplification gain. Here, we study nonlinear transient amplification (NTA), a plausible alternative mechanism that reconciles strong recurrent excitation with rapid amplification while avoiding the above issues. NTA has two distinct temporal phases. Initially, positive feedback excitation selectively amplifies inputs that exceed a critical threshold. Subsequently, short-term plasticity quenches the run-away dynamics into an inhibition-stabilized network state. By characterizing NTA in supralinear network models, we establish that the resulting onset transients are stimulus selective and well-suited for speedy information processing. Further, we find that excitatory-inhibitory co-tuning widens the parameter regime in which NTA is possible in the absence of persistent activity. In summary, NTA provides a parsimonious explanation for how excitatory-inhibitory co-tuning and short-term plasticity collaborate in recurrent networks to achieve transient amplification. Editor's evaluation Many brain circuits, particularly those found in mammalian sensory cortices, need to respond rapidly to stimuli while at the same time avoiding pathological, runaway excitation. Over several years, many theoretical studies have attempted to explain how cortical circuits achieve these goals through interactions between inhibitory and excitatory cells. This study adds to this literature by showing how synaptic short-term depression can stabilise strong positive feedback in a circuit under a variety of plausible scenarios, allowing strong, rapid and stimulus-specific responses. https://doi.org/10.7554/eLife.71263.sa0 Decision letter Reviews on Sciety eLife's review process Introduction Perception in the brain is reliable and strikingly fast. Recognizing a familiar face or locating an animal in a picture only takes a split second (Thorpe et al., 1996). This pace of processing is truly remarkable since it involves several recurrently connected brain areas each of which has to selectively amplify or suppress specific signals before propagating them further. This processing is mediated through circuits with several intriguing properties. First, excitatory-inhibitory (EI) currents into individual neurons are commonly correlated in time and co-tuned in stimulus space (Wehr and Zador, 2003; Froemke et al., 2007; Okun and Lampl, 2008; Hennequin et al., 2017; Rupprecht and Friedrich, 2018; Znamenskiy et al., 2018). Second, neural responses to stimulation are shaped through diverse forms of short-term plasticity (STP) (Tsodyks and Markram, 1997; Markram et al., 1998; Zucker and Regehr, 2002; Pala and Petersen, 2015). Finally, mounting evidence suggests that amplification rests on neuronal ensembles with strong recurrent excitation (Marshel et al., 2019; Peron et al., 2020), whereby excitatory neurons with similar tuning preferentially form reciprocal connections (Ko et al., 2011; Cossell et al., 2015). Such predominantly symmetric connectivity between excitatory cells is consistent with the notion of Hebbian cell assemblies (Hebb, 1949), which are considered an essential component of neural circuits and the putative basis of associative memory (Harris, 2005; Josselyn and Tonegawa, 2020). Computationally, Hebbian cell assemblies can amplify specific activity patterns through positive feedback, also referred to as Hebbian amplification. Based on these principles, several studies have shown that Hebbian amplification can drive persistent activity that outlasts a preceding stimulus (Hopfield, 1982; Amit and Brunel, 1997; Yakovlev et al., 1998; Wong and Wang, 2006; Zenke et al., 2015; Gillary et al., 2017), comparable to selective delay activity observed in the prefrontal cortex when animals are engaged in working memory tasks (Funahashi et al., 1989; Romo et al., 1999). However, in most brain areas, evoked responses are transient and sensory neurons typically exhibit pronounced stimulus onset responses, after which the circuit dynamics settle into a low-activity steady-state even when the stimulus is still present (DeWeese et al., 2003; Mazor and Laurent, 2005; Bolding and Franks, 2018). Preventing run-away excitation and multi-stable attractor dynamics in recurrent networks requires powerful and often finely tuned feedback inhibition resulting in EI balance (Amit and Brunel, 1997; Compte et al., 2000; Litwin-Kumar and Doiron, 2012; Ponce-Alvarez et al., 2013; Mazzucato et al., 2019), However, strong feedback inhibition tends to linearize steady-state activity (van Vreeswijk and Sompolinsky, 1996; Baker et al., 2020). Murphy and Miller, 2009 proposed balanced amplification which reconciles transient amplification with strong recurrent excitation by tightly balancing recurrent excitation with strong feedback inhibition (Goldman, 2009; Hennequin et al., 2012; Hennequin et al., 2014; Bondanelli and Ostojic, 2020; Gillett et al., 2020). Importantly, balanced amplification was formulated for linear network models of excitatory and inhibitory neurons. Due to linearity, it intrinsically lacks the ability to nonlinearly amplify stimuli which limits its capabilities for pattern completion and pattern separation. Further, how balanced amplification relates to nonlinear neuronal activation functions and nonlinear synaptic transmission as, for instance, mediated by STP (Tsodyks and Markram, 1997; Markram et al., 1998; Zucker and Regehr, 2002; Pala and Petersen, 2015), remains elusive. This begs the question of whether there are alternative nonlinear amplification mechanisms and how they relate to existing theories of recurrent neural network processing. Here, we address this question by studying an alternative mechanism for the emergence of transient dynamics that relies on recurrent excitation, supralinear neuronal activation functions, and STP. Specifically, we build on the notion of ensemble synchronization in recurrent networks with STP (Loebel and Tsodyks, 2002; Loebel et al., 2007) and study this phenomenon in analytically tractable network models with rectified quadratic activation functions (Ahmadian et al., 2013; Rubin et al., 2015; Hennequin et al., 2018; Kraynyukova and Tchumatchenko, 2018) and STP. We first characterize the conditions under which individual neuronal ensembles with supralinear activation functions and recurrent excitatory connectivity succumb to explosive run-away activity in response to external stimulation. We then show how STP effectively mitigates this instability by re-stabilizing ensemble dynamics in an inhibition-stabilized network (ISN) state, but only after generating a pronounced stimulus-triggered onset transient. We call this mechanism NTA and show that it yields selective onset responses that carry more relevant stimulus information than the subsequent steady-state. Finally, we characterize the functional benefits of inhibitory co-tuning, a feature that is widely observed in the brain (Wehr and Zador, 2003; Froemke et al., 2007; Okun and Lampl, 2008; Rupprecht and Friedrich, 2018) and readily emerges in computational models endowed with activity-dependent plasticity of inhibitory synapses (Vogels et al., 2011). We find that co-tuning prevents persistent attractor states but does not preclude NTA from occurring. Importantly, NTA purports that, following transient amplification, neuronal ensembles settle into a stable ISN state, consistent with recent studies suggesting that inhibition stabilization is a ubiquitous feature of cortical networks (Sanzeni et al., 2020). In summary, our work indicates that NTA is ideally suited to amplify stimuli rapidly through the interaction of strong recurrent excitation with STP. Results To understand the emergence of transient responses in recurrent neural networks, we studied rate-based population models with a supralinear, power law input-output function (Figure 1A and B; Ahmadian et al., 2013; Hennequin et al., 2018), which captures essential aspects of neuronal activation (Priebe et al., 2004), while also being analytically tractable. We first considered an isolated neuronal ensemble consisting of one excitatory (E) and one inhibitory (I) population (Figure 1A). Figure 1 with 1 supplement see all Download asset Open asset Neuronal ensembles nonlinearly amplify inputs above a critical threshold. (A) Schematic of the recurrent ensemble model consisting of an excitatory (blue) and an inhibitory population (red). (B) Supralinear input-output function given by a rectified power law with exponent α=2. (C) Firing rates of the excitatory (blue) and inhibitory population (red) in response to external stimulation during the interval from 2 to 4 s (gray bar). The stimulation was implemented by temporarily increasing the input gE. (D) Phase portrait of the system before stimulation (left; C orange) and during stimulation (right; C green). (E) Characteristic function F⁢(z) for varying input strength gE. Note that the function loses its zero crossings, which correspond to fixed points of the system for increasing external input. (F) Heat map showing the evoked firing rate of the excitatory population for different parameter combinations JE⁢E and gE. The gray region corresponds to the parameter regime with unstable dynamics. The dynamics of this network are given by (1) τE⁢d⁢rEd⁢t=-rE+[JE⁢E⁢rE-JE⁢I⁢rI+gE]+αE , (2) τI⁢d⁢rId⁢t=-rI+[JI⁢E⁢rE-JI⁢I⁢rI+gI]+αI , where rE and rI are the firing rates of the excitatory and inhibitory population, τE and τI represent the corresponding time constants, JX⁢Y denotes the synaptic strength from the population Y to the population X, where X,Y∈{E,I}, gE and gI are the external inputs to the respective populations. Finally, αE and αI, the exponents of the respective input-output functions, are fixed at two unless mentioned otherwise. For ease of notation, we further define the weight matrix J of the compound system as follows: (3) J=[JEE−JEIJIE −JII]. We were specifically interested in networks with strong recurrent excitation that can generate positive feedback dynamics in response to external inputs gE. Therefore, we studied networks with (4) det(J)=−JEEJII+JIEJEI<0. In contrast, networks in which recurrent excitation is met by strong feedback inhibition such that det(J)>0 are unable to generate positive feedback dynamics provided that inhibition is fast enough (Ahmadian et al., 2013). Importantly, we assumed that most inhibition originates from recurrent connections (Franks et al., 2011; Large et al., 2016) and, hence, we kept the input to the inhibitory population gI fixed unless mentioned otherwise. Nonlinear amplification of inputs above a critical threshold We initialized the network in a stable low-activity state in the absence of external stimulation, consistent with spontaneous activity in cortical networks (Figure 1C). However, an input gE of sufficient strength, destabilized the network (Figure 1C). Importantly, this behavior is distinct from linear network models in which the network stability is independent of inputs (Materials and methods). The transition from stable to unstable dynamics can be understood by examining the phase portrait of the system (Figure 1D). Before stimulation, the system has a stable and an unstable fixed point (Figure 1D, left). However, both fixed points disappear for an input gE above a critical stimulus strength (Figure 1D, right). To further understand the system’s bifurcation structure, we consider the characteristic function (5) F(z)=JEE[z]+αE−JEI[det(J)⋅JEI−1[z]+αE+JEI−1JIIz−JEI−1JIIgE+gI]+αI−z+gE, where z denotes the total current into the excitatory population and det(J) represents the determinant of the weight matrix (Kraynyukova and Tchumatchenko, 2018; Materials and methods). The characteristic function reduces the original two-dimensional system to one dimension, whereby the zero crossings of the characteristic function correspond to the fixed points of the original system (Eq. (1)-(2)). We use this correspondence to visualize how the fixed points of the system change with the input gE. Increasing gE shifts F⁢(z) upwards, which eventually leads to all zero crossings disappearing and the ensuing unstable dynamics (Figure 1E; Materials and methods). Importantly, for any weight matrix J with negative determinant, there exists a critical input gE at which all fixed points disappear (Materials and methods). While for weak recurrent E-to-E connection strength JE⁢E, the transition from stable dynamics to unstable is gradual, in that it happens at higher firing rates (Figure 1F), it becomes more abrupt for stronger JE⁢E. Thus, our analysis demonstrates that individual neuronal ensembles with negative determinant det(J) nonlinearly amplify inputs above a critical threshold by switching from initially stable to unstable dynamics. Short-term plasticity, but not spike-frequency adaptation, can re-stabilize ensemble dynamics Since unstable dynamics are not observed in neurobiology, we wondered whether neuronal spike frequency adaptation (SFA) or STP could re-stabilize the ensemble dynamics while keeping the nonlinear amplification character of the system. Specifically, we considered SFA of excitatory neurons, E-to-E short-term depression (STD), and E-to-I short-term facilitation (STF). We focused on these particular mechanisms because they are ubiquitously observed in the brain. Most pyramidal cells exhibit SFA (Barkai and Hasselmo, 1994) and most synapses show some form of STP (Markram et al., 1998; Zucker and Regehr, 2002; Pala and Petersen, 2015). Moreover, the time scales of these mechanisms are well-matched to typical timescales of perception, ranging from milliseconds to seconds (Tsodyks and Markram, 1997; Fairhall et al., 2001; Pozzorini et al., 2013). When we simulated our model with SFA (Eqs. (21)–(23)), we observed different network behaviors depending on the adaptation strength. When adaptation strength was weak, SFA was unable to stabilize run-away excitation (Figure 2A; Materials and methods). Increasing the adaptation strength eventually prevented run-away excitation, but to give way to oscillatory ensemble activity (Figure 2—figure supplement 1). Finally, we confirmed analytically that SFA cannot stabilize excitatory run-away dynamics at a stable fixed point (Materials and methods). In particular, while the input is present, strong SFA creates a stable limit cycle with associated oscillatory ensemble activity (Figure 2—figure supplement 1; Materials and methods), which was also shown in previous modeling studies (van Vreeswijk and Hansel, 2001), but is not typically observed in sensory systems (DeWeese et al., 2003; Rupprecht and Friedrich, 2018). Figure 2 with 12 supplements see all Download asset Open asset Short-term plasticity, but not spike-frequency adaptation, re-stabilizes ensemble dynamics. (A) Firing rates of the excitatory (blue) and inhibitory population (red) in the presence of spike-frequency adaptation (SFA). During stimulation (gray bar) additional input is injected into the excitatory population. The inset shows a cartoon of how SFA affects spiking neuronal dynamics in response to a step current input. (B) Left: Same as (A) but in the presence of E-to-E short-term depression (STD). Right: Same as left but inactivating inhibition in the period marked in purple. (C) 3D plot of the excitatory activity rE, inhibitory activity rI, and the STD variable x of the network in B left. The orange and green points mark the fixed points before/after and during stimulation. (D) Characteristic function F⁢(z) in networks with E-to-E STD. Different brightness levels correspond to different time points in B left. (E) Same as (B) but in the presence of E-to-I short-term facilitation (STF). (F) Inhibition-stabilized network (ISN) index, which corresponds to the largest real part of the eigenvalues of the Jacobian matrix of the E-E subnetwork with STD, as a function of time for the network with E-to-E STD in B left. For values above zero (dashed line), the ensemble is an ISN. (G) Analytical solution of non-ISN (magenta), ISN (green), paradoxical, and non-paradoxical regions for different parameter combinations JE⁢E and the STD variable x. The solid line separates the non-ISN and ISN regions, whereas the dashed line separates the non-paradoxical and paradoxical regions. (H) The normalized firing rates of the excitatory (blue) and inhibitory population (red) when injecting additional excitatory current into the inhibitory population before stimulation (left; orange bar in B), and during stimulation (right; green bar in B). Initially, the ensemble is in the non-ISN regime and injecting excitatory current into the inhibitory population increases its firing rate. During stimulation, however, the ensemble is an ISN. In this case, excitatory current injection into the inhibitory population results in a reduction of its firing rate, also known as the paradoxical effect. Next, we considered STP, which is capable of saturating the effective neuronal input-output function (Mongillo et al., 2012; Zenke et al., 2015; Eqs. (37)–(39), Eqs. (41)–(43)). We first analyzed the stimulus-evoked network dynamics when we added STD to the recurrent E-to-E connections. Strong depression of synaptic efficacy resulted in a brief onset transient after which the ensemble dynamics quickly settled into a stimulus-evoked steady-state with slightly higher activity than the baseline (Figure 2B, left). After stimulus removal, the ensemble activity returned back to its baseline level (Figure 2B, left; Figure 2C). Notably, the ensemble dynamics settled at a stable steady state with a much higher firing rate, when inhibition was inactivated during stimulus presentation (Figure 2B, right). This shows that STP is capable of creating a stable high-activity fixed point, which is fundamentally different from the SFA dynamics discussed above. This difference in ensemble dynamics can be readily understood by analyzing the stability of the three-dimensional dynamical system (Materials and methods). We can gain a more intuitive understanding by considering self-consistent solutions of the characteristic function F⁢(z). Initially, the ensemble is at the stable low activity fixed point. But the stimulus causes this fixed point to disappear, thus giving way to positive feedback which creates the leading edge of the onset transient (Figure 2B). However, because E-to-E synaptic transmission is rapidly reduced by STD, the curvature of F⁢(z) changes and a stable fixed point is created, thereby allowing excitatory run-away dynamics to terminate and the ensemble dynamics settle into a steady-state at low activity levels (Figure 2D). We found that E-to-I STF leads to similar dynamics (Figure 2E, left; Appendix 1) with the only difference that this configuration requires inhibition for network stability (Figure 2E, right), whereas E-to-E STD stabilizes activity even without inhibition, albeit at physiologically implausibly high activity levels. Importantly, the re-stabilization through either form of STP did not impair an ensemble’s ability to amplify stimuli during the initial onset phase. Crucially, transient amplification in supralinear networks with STP occurs above a critical threshold (Figure 2—figure supplement 2), and requires recurrent excitation JEE to be sufficiently strong (Figure 2—figure supplement 2C, D). To quantify the amplification ability of these networks, we calculated the ratio of the evoked peak firing rate to the input strength, henceforth called the ‘Amplification index’. We found that amplification in STP-stabilized supralinear networks can be orders of magnitude larger than in linear networks with equivalent weights and comparable stabilized supralinear networks (SSNs) without STP (Figure 2—figure supplement 3). We stress that the resulting firing rates are parameter-dependent (Figure 2—figure supplement 4) and their absolute value can be high due to the high temporal precision of the onset peak and its short duration. In experiments, such high rates manifest themselves as precisely time-locked spikes with millisecond resolution (DeWeese et al., 2003; Wehr and Zador, 2003; Bolding and Franks, 2018; Gjoni et al., 2018). Recent studies suggest that cortical networks operate as inhibition-stabilized networks (ISNs) (Sanzeni et al., 2020; Sadeh and Clopath, 2021), in which the excitatory network is unstable in the absence of feedback inhibition (Tsodyks et al., 1997; Ozeki et al., 2009). To that end, we investigated how ensemble re-stabilization relates to the network operating regime at baseline and during stimulation. a network is an ISN or not is by the real part of the leading of the Jacobian of the subnetwork (Tsodyks et al., We the leading in our model STP and referred to it as (Materials and Appendix We found that in networks with STP the ISN can from negative to positive during external stimulation, that the ensemble can transition from a non-ISN to an ISN (Figure Notably, this behavior is distinct from linear network models in which the network operating regime is independent of the input (Materials and methods). this between non-ISN to ISN however, was parameter and we also found network that were in the ISN regime at baseline and during stimulation (Figure 2—figure supplement Thus, re-stabilization was by the network state and consistent with observed ISN states (Sanzeni et al., 2020). studies have shown that one characteristic of in excitatory and inhibitory networks is that injecting excitatory current into inhibitory neurons inhibitory firing which is also known as the paradoxical (Tsodyks et al., 1997; and 2020). it is whether in networks with STP, inhibitory stabilization paradoxical response and We analyzed the of being an ISN and the of paradoxical response in networks with STP (Materials and Appendix Appendix 3). we found that in networks with E-to-E STD, the paradoxical inhibitory whereas inhibitory stabilization does not paradoxical response (Figure Materials and methods), suggesting that paradoxical is a sufficient but not for being an ISN. In contrast, in networks with E-to-I inhibitory stabilization and paradoxical each Appendix 3). Therefore, paradoxical can be as a for inhibition stabilization for networks with STP we considered By injecting excitatory current into the inhibitory population, we found that the network did not exhibit the paradoxical before stimulation (Figure left; Figure 2—figure supplement In contrast, injecting excitatory inputs into the inhibitory population during stimulation reduced their activity (Figure Figure 2—figure supplement in our non-paradoxical response does not non-ISN (Figure 2—figure supplement Materials and methods). We the inhibition stabilization of the ensemble by the ensemble behavior when a transient to excitatory population activity is while inhibition is before stimulation and during stimulation. Before stimulation, the firing rate of the excitatory population slightly increases and then to its baseline after the transient (Figure 2—figure supplement During stimulation, however, the transient leads to a transient of the excitatory firing rate (Figure 2—figure supplement results further that the ensemble shown in our is initially a non-ISN before stimulation and can transition to an ISN with stimulation. By the input level at the baseline in the the ensemble can be initially an ISN (Figure 2—figure supplement recent studies that cortical circuits in the operate as in the absence of sensory stimulation (Sanzeni et al., 2020). the that the supralinear input-output function of our captures some aspects of (Priebe et al., 2004), it is and thus high firing This is in to where firing rates are due to neuronal While this to analytically study the system and to gain a understanding of the ensemble dynamics, we wondered whether our were also when we the firing To that end, we the same while the firing rate at In the absence of additional SFA or STP the firing rate a stable high-activity state in the ensemble dynamics which the unstable dynamics in the the ensemble this high-activity steady-state when with an external input above a critical threshold and persistent activity after stimulus (Figure 2—figure supplement While weak SFA did not change this strong SFA resulted in oscillatory behavior during stimulation consistent with previous work (Figure 2—figure supplement Vreeswijk and Hansel, 2001), but did not in stable commonly observed in In the presence of E-to-E STD or E-to-I however, the ensemble transient evoked activity at stimulation onset that was comparable to the Importantly, the ensemble did not show persistent activity after the stimulation (Figure 2—figure supplement Finally, we confirmed that all of these were similar in a spiking neural network model (Figure 2—figure supplement Materials and methods). In summary, we found that neuronal ensembles can and amplify inputs by switching from stable to unstable dynamics before being through STP We call this mechanism nonlinear transient amplification in to balanced amplification and Miller, 2009; Hennequin et al., from population dynamics with supralinear neuronal activation functions with STP. While we that there be nonlinear transient amplification in this we our analysis to the above. NTA is by a onset a subsequent ISN steady-state while the stimulus and a to a baseline activity state after the stimulus is Thus, NTA is ideally suited to rapidly and nonlinearly amplify sensory inputs through recurrent excitation, (Ko et al., 2011; Cossell et al., 2015), while avoiding persistent activity. inhibition the parameter regime of NTA in the absence of persistent activity to we have focused on a neuronal However, to process information in the several ensembles with different stimulus and in the same This creates can to multi-stable persistent attractor dynamics, which are not commonly observed and could have effects on the processing of subsequent solution to this could be EI co-tuning, which in network models with inhibitory synapses (Vogels et al., and has observed in several sensory systems (Wehr and Zador, 2003; Froemke et al., 2007; Okun and Lampl, 2008; Rupprecht and Friedrich, 2018). To characterize the conditions under which neuronal ensembles nonlinearly amplify stimuli without persistent we analyzed the of two we considered networks with two excitatory ensembles and between and co-tuned inhibition (Figure In the of inhibition, one inhibitory population both excitatory (Figure left). In contrast, in networks with co-tuned inhibition, each ensemble is by a of an excitatory and an inhibitory population which can have for instance, due to ensembles (Figure right). Figure 3 with 1 supplement see all Download asset Open asset inhibition the parameter regime of NTA in the absence of persistent activity. (A) Schematic of two neuronal ensembles with inhibition and with co-tuned inhibition (B) Firing rate

Decision letter: Nonlinear transient amplification in recurrent neural networks with short-term plasticity

The increasing complexity and variety of image sensors has been the source of interest in the development of data compression for images. Image data has become one of the most active topics of research in digital image processing as a result [1]. The continued evolution of digital circuitry has caused the focus of data compression research to lie in digital implementations. However, there is also a potential for optical computations in image data compression, as was demonstrated in the concepts of interpolated DPCM [2]. The method of DPCM data compression is one of the most thoroughly studied techniques. DPCM achieves data compression by separating the image information into two parts: the low-spatial frequencies and the high-spatial frequencies. Low-spatial frequencies are re-tained by exploiting their predictability; high-spatial frequencies are retained at fewer significant bits, and substantial data compression is achieved. Interpolated DPCM is a mechanism for separating an image into low- and high- spatial frequency components, with a similar amount of data compression being achieved. The computations to achieve the sep-aration can be implemented by simple incoherent optical devices [2].

Since the introduction of sine wave gratings in electrophysiology [Campbell and Green, J. Physiol. 181, 573 (1965)], properties of spatial visual processing have been investigated and expressed in terms of spatial frequency channels. Such channels have also been found in human psycho­physics, leading to models of spatial vision such as that of Wilson and Bergen [Vision Res. 19, 19 (1979)]. The physiological substrate of such models in man is unknown but is generally thought to be situated early along the visual pathway. To test the nature of spatial frequency processing in striate visual cortex in primates, visually evoked potentials were recorded subdurally in alert rhesus monkeys. Two-dimensional (checkerboard) stimuli, in which spatial frequency components could be modulated independently, were presented. Our data show that neuronal responses to modulation of one spatial frequency component are highly dependent on the presence of (stationary or modulated) components with spatial frequencies at intervals up to two octaves. These interactions occur both for transient and for steady-state stimuli and are mutual, although not symmetric. They can best be modeled as mutual adaptations of localized spatial filters; a few populations of such filters, with bandwidths of 1–2 octaves, could account for the data.

Hebb's Dream: The Resurgence of Cell Assemblies