Sigmoidal Nonlinearity Research Articles

Periodic solutions to a mean-field model for electrocortical activity

We consider a continuum model of electrical signals in the human cortex, which takes the form of a system of semilinear, hyperbolic partial differential equations for the inhibitory and excitatory membrane potentials and the synaptic inputs. The coupling of these components is represented by sigmoidal and quadratic nonlinearities. We consider these equations on a square domain with periodic boundary conditions, in the vicinity of the primary transition from a stable equilibrium to time-periodic motion through an equivariant Hopf bifurcation. We compute part of a family of standing wave solutions, emanating from this point.

Sparseness, antisparseness and anything in between: the operating point of a neuron determines its computational repertoire.

A recent model of intrinsic plasticity coupled to Hebbian synaptic plasticity proposes that adaptation of a neuron's threshold and gain in a sigmoidal response function to achieve a sparse, exponential output firing rate distribution facilitates the discovery of heavy-tailed or super- gaussian sources in the neuron's inputs. We show that the exponential output distribution is irrelevant to these dynamics and that, furthermore, while sparseness is sufficient, it is not necessary. The intrinsic plasticity mechanism drives the neuron's threshold large and positive, and we prove that in such a regime, the neuron will find supergaussian sources; equally, however, if the threshold is large and negative (an antisparse regime), it will also find supergaussian sources. Away from such extremes, the neuron can also discover subgaussian sources. By examining a neuron with a fixed sigmoidal nonlinearity and considering the synaptic strength fixed-point structure in the two-dimensional parameter space defined by the neuron's threshold and gain, we show that this space is carved up into sub- and supergaussian-input-finding regimes, possibly with regimes of simultaneous stability of sub- and supergaussian sources or regimes of instability of all sources; a single gaussian source may also be stabilized by the presence of a nongaussian source. A neuron's operating point (essentially its threshold and gain coupled with its input statistics) therefore critically determines its computational repertoire. Intrinsic plasticity mechanisms induce trajectories in this parameter space but do not fundamentally modify it. Unless the trajectories cross critical boundaries in this space, intrinsic plasticity is irrelevant and the neuron's nonlinearity may be frozen with identical receptive field refinement dynamics.

A novel model incorporating two variability sources for describing motor evoked potentials.

Motor evoked potentials (MEPs) play a pivotal role in transcranial magnetic stimulation (TMS), e.g., for determining the motor threshold and probing cortical excitability. Sampled across the range of stimulation strengths, MEPs outline an input-output (IO) curve, which is often used to characterize the corticospinal tract. More detailed understanding of the signal generation and variability of MEPs would provide insight into the underlying physiology and aid correct statistical treatment of MEP data. A novel regression model is tested using measured IO data of twelve subjects. The model splits MEP variability into two independent contributions, acting on both sides of a strong sigmoidal nonlinearity that represents neural recruitment. Traditional sigmoidal regression with a single variability source after the nonlinearity is used for comparison. The distribution of MEP amplitudes varied across different stimulation strengths, violating statistical assumptions in traditional regression models. In contrast to the conventional regression model, the dual variability source model better described the IO characteristics including phenomena such as changing distribution spread and skewness along the IO curve. MEP variability is best described by two sources that most likely separate variability in the initial excitation process from effects occurring later on. The new model enables more accurate and sensitive estimation of the IO curve characteristics, enhancing its power as a detection tool, and may apply to other brain stimulation modalities. Furthermore, it extracts new information from the IO data concerning the neural variability-information that has previously been treated as noise.

Steady State and Transient MSE Analysis of Convexly Constrained Mixture Methods

We investigate convexly constrained mixture methods to adaptively combine outputs of two adaptive filters running in parallel to model a desired unknown system. We compare several algorithms with respect to their mean-square error in the steady state, when the underlying unknown system is nonstationary with a random walk model. We demonstrate that these algorithms are universal such that they achieve the performance of the best constituent filter in the steady state if certain algorithmic parameters are chosen properly. We also demonstrate that certain mixtures converge to the optimal convex combination filter such that their steady-state performances can be better than the best constituent filter. We also perform the transient analysis of these updates in the mean and mean-square error sense. Furthermore, we show that the investigated convexly constrained algorithms update certain auxiliary variables through sigmoid nonlinearity, hence, in this sense, related.

Modeling and Prediction of Monthly Total Ozone Concentrations by Use of an Artificial Neural Network Based on Principal Component Analysis

In the work discussed in this paper we considered total ozone time series over Kolkata (22°34′10.92″N, 88°22′10.92″E), an urban area in eastern India. Using cloud cover, average temperature, and rainfall as the predictors, we developed an artificial neural network, in the form of a multilayer perceptron with sigmoid non-linearity, for prediction of monthly total ozone concentrations from values of the predictors in previous months. We also estimated total ozone from values of the predictors in the same month. Before development of the neural network model we removed multicollinearity by means of principal component analysis. On the basis of the variables extracted by principal component analysis, we developed three artificial neural network models. By rigorous statistical assessment it was found that cloud cover and rainfall can act as good predictors for monthly total ozone when they are considered as the set of input variables for the neural network model constructed in the form of a multilayer perceptron. In general, the artificial neural network has good potential for predicting and estimating monthly total ozone on the basis of the meteorological predictors. It was further observed that during pre-monsoon and winter seasons, the proposed models perform better than during and after the monsoon.

A neurocomputing approach to the forecasting of monthly maximum temperature over Kolkata, India using total ozone concentration as predictor

The association between the monthly total ozone concentration and monthly maximum temperature over Kolkata (22.56° N, 88.30° E), India, has been explored in this paper. For this, the predictability of monthly maximum temperature based on the total ozone as predictor is investigated using Artificial Neural Network. The presence of persistence and similar cyclic patterns are revealed through autocorrelation and cross-correlation coefficients. Common cycles of length 12 and 6 have been identified through periodogram. Hence, a predictive model has been generated by Artificial Neural Network in the form of Multi Layer Perceptron (MLP) using scaled conjugate gradient learning with sigmoid non-linearity. After training and testing the network, an MLP with total ozone of month n as predictor and maximum temperature of month ( n + 1) as the target output is found as the best model. Performance of the model has been judged statistically. Finally, the MLP model has been compared with linear and non-linear regressions and the efficiency of MLP has been established over the regression models.

To honor Fechner and obey Stevens: Relationships between psychophysical and neural nonlinearities.

G. T. Fechner (1860/1966) famously described two kinds of psychophysics: Outer psychophysics captures the black box relationship between sensory inputs and perceptual magnitudes, whereas inner psychophysics contains the neural transformations that Fechner's outer psychophysics elided. The relationship between the two has never been clear. Moreover, psychophysical power laws are found in almost every sensory system, yet the vast majority of neurons show sigmoid nonlinearities. Here, we selectively review the literatures on psychophysical and physiological nonlinearities and show how they can be placed within a framework for understanding the relationship between inner and outer psychophysics: a neural organization with a logical structure commensurate to outer psychophysical theory. In theoretical treatments of Stevens's law, the power law is a consequence of combining a Weber's law scaling of inputs with a Weber's law-like scaling of sensation magnitudes, yielding an exponent that is the ratio of the Weber constants. A neural derivation using physiological sigmoid nonlinearities should be commensurate to this internal logic. There is a class of models in which two nonlinear neural mechanisms (e.g., a sensory channel and the cortical numerosity mechanism tapped by magnitude estimation) are coupled through feedback, yielding power law behavior as an emergent property of the system, with an exponent that is a ratio of neural coupling strengths. Rather than a discrepancy between psychophysics and physiology, these models suggest complementarity between inner and outer psychophysics, because the Weber constants required for outer psychophysics modeling can be derived from the sigmoid nonlinearities of inner psychophysics.

Identification of the best architecture of a multilayer perceptron in modeling daily total ozone concentration over Kolkata, India

Autoregressive neural network (AR-NN) models of various orders have been generated in this work for the daily total ozone (TO) time series over Kolkata (22.56°N, 88.5°E). Artificial neural network in the form of multilayer perceptron (MLP) is implemented in order to generate the AR-NN models of orders varying from 1 to 13. An extensive variable selection method through multiple linear regression (MLR) is implemented while developing the AR-NNs. The MLPs are characterized by sigmoid non-linearity. The optimum size of the hidden layer is identified in each model and prediction are produced by validating it over the test cases using the coefficient of determination (R 2) and Willmott’s index (WI). It is observed that AR-NN model of order 7 having 6 nodes in the hidden layer has maximum prediction capacity. It is further observed that any increase in the orders of AR-NN leads to less accurate prediction.

Multivariate forecast of winter monsoon rainfall in India using SST anomaly as a predictor: Neurocomputing and statistical approaches

In this article, the complexities in the relationship between rainfall and sea surface temperature (SST) anomalies during the winter monsoon over India were evaluated statistically using scatter plot matrices and autocorrelation functions. Linear, as well as polynomial trend equations were obtained, and it was observed that the coefficient of determination for the linear trend was very low and it remained low even when polynomial trend of degree six was used. An exponential regression equation and an artificial neural network with extensive variable selection were generated to forecast the average winter monsoon rainfall of a given year using the rainfall amounts and the SST anomalies in the winter monsoon months of the previous year as predictors. The regression coefficients for the multiple exponential regression equation were generated using Levenberg-Marquardt algorithm. The artificial neural network was generated in the form of a multilayer perceptron with sigmoid non-linearity and genetic-algorithm based variable selection. Both of the predictive models were judged statistically using the Willmott's index, percentage error of prediction, and prediction yields. The statistical assessment revealed the potential of artificial neural network over exponential regression.

Multiperiodicity of Periodically Oscillated Discrete-Time Neural Networks With Transient Excitatory Self-Connections and Sigmoidal Nonlinearities

The existing approaches to the multistability and multiperiodicity of neural networks rely on the strictly excitatory self-interactions of neurons or require constant interconnection weights. For periodically oscillated discrete-time neural networks (DTNNs), it is difficult to discuss multistable dynamics when the connection weights are periodically oscillated around zero. By using transient excitatory self-interactions of neurons and sigmoidal nonlinearities, we develop an approach to investigate multiperiodicity and attractivity of periodically oscillated DTNNs with time-varying and distributed delays. It shows that, under some new criteria, there exist multiplicity results of periodic solutions which are locally or globally exponentially stable. Computer numerical simulations are performed to illustrate the new theories.

Univariate modelling of summer-monsoon rainfall time series: Comparison between ARIMA and ARNN

The present article reports studies to develop a univariate model to forecast the summer monsoon (June–August) rainfall over India. Based on the data pertaining to the period 1871–1999, the trend and stationarity within the time series have been investigated. After revealing the randomness and non-stationarity within the time series, the autoregressive integrated moving average (ARIMA) models have been attempted and the ARIMA(0,1,1) has been identified as a suitable representative model. Consequently, an autoregressive neural network (ARNN) model has been attempted and the neural network has been trained as a multilayer perceptron with the extensive variable selection procedure. Sigmoid non-linearity has been used while training the network. Finally, a three-three-one architecture of the ARNN model has been obtained and after thorough statistical analysis the supremacy of ARNN has been established over ARIMA(0,1,1). The usefulness of ARIMA(0,1,1) has also been described.

التعامل مع التعقيدات الزلزالية باستخدام التقنيات الحاسوبية العصبية وتحديد قدرها الزلزالي ببعض المتوقعات منخفضة المقارنة

Earthquake, being one of the most hazardous geophysical events, has fascinated the scientists all over the globe to investigate an efficient predictive methodology. In India, earthquake is not a rare event. Purpose of the present paper is to view the earthquake as a complex system and then to estimate the magnitude of earthquake over Indian subcontinent using artificial neural network with backpropagation learning. The day, month, and year of occurrence of the earthquake, latitude of the place, and the longitude of the place are considered as the predictor variables. After considering various sizes of the hidden layer, it has been found that artificial neural network in the form of multilayer perceptron with two hidden nodes and sigmoid nonlinearity can be used to anticipate the magnitude of earthquake over Indian subcontinent. The paper concludes with an outline for future work.

What is the "contrast" in contrast adaptation?

Event Abstract Back to Event What is the "contrast" in contrast adaptation? During natural vision, retinal ganglion cells encode enormous variations of light intensity. In response, they adapt to the luminance distribution?s mean and width, the latter usually quantified by contrast (C = standard deviation / mean). This contrast adaptation compresses the large dynamic range of stimuli into a narrow range of firing rates. Here we describe ganglion cell responses via a linear-nonlinear model and ask, how should the nonlinearity adapt to different stimulus ensembles to efficiently fill the cells "response bandwidth" To study this, we passed each stimulus through a fixed linear filter followed by rectification and a nonlinearity, the output of which is taken to model an OFF ganglion cell?s firing rate. We then sought the nonlinearity that maximized information transfer subject to a constraint on the peak firing rate. The predicted nonlinearity maps the distribution of filter outputs to a uniform distribution of firing rates. An added mean rate constraint gives a truncated exponential rate distribution. First, we considered a Gaussian stimulus ensemble. For this ensemble, the model predicts a sigmoidal nonlinearity with a gain (slope at inflection point) proportional to 1/C. To test this, we made patch-clamp recordings of guinea pig retinal ganglion cells subject to full-field white noise stimulation at several contrasts. We found very close agreement between the predicted and measured nonlinearities at each contrast level. Natural light fluctuations are strongly non-Gaussian: they have temporal correlations and a skewed distribution. To study how this affects the optimal nonlinearity, we sorted a natural time series (van Hateren, 1997) into segments of fixed contrast. For all segment lengths we found that the distribution of contrasts was wide and skewed (peak C ~ 0.5 and 10% of segments with C > 1.5). In addition, for any given contrast, the light intensity distribution was also highly skewed. As above, we filtered and rectified segments of a given contrast; the resulting optimal gain was proportional to 1/C^0.5. This behavior differs from the Gaussian results in that it predicts less adaptation to increasing contrast. This is because skewed stimuli feature large but rare intensity fluctuations that strongly affect C, but excessive adaptation to these outliers would render the cell insensitive to the much more frequent small intensities. These considerations led us to ask whether a measure of contrast which is robust to outliers would be more natural for skewed intensity distributions. One possibility is the q-width, namely the range around the mean containing q% of intensity values. For Gaussian stimuli the q-width is proportional to the standard deviation. We found that the optimal gain predicted from the natural time series went approximately as the reciprocal of the 60-width. Thus, using this measure, the predicted gains have the same functional form for both naturally skewed and Gaussian distributions. Experiments in preparation will measure contrast adaptation in response to naturalistic stimuli, allowing us to test the prediction that contrast gain control is insensitive to outliers. Conference: Computational and systems neuroscience 2009, Salt Lake City, UT, United States, 26 Feb - 3 Mar, 2009. Presentation Type: Poster Presentation Topic: Poster Presentations Citation: (2009). What is the "contrast" in contrast adaptation?. Front. Syst. Neurosci. Conference Abstract: Computational and systems neuroscience 2009. doi: 10.3389/conf.neuro.06.2009.03.144 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 02 Feb 2009; Published Online: 02 Feb 2009. Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Google Google Scholar PubMed Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Persistent Neural States: Stationary Localized Activity Patterns in Nonlinear Continuous n-Population, q-Dimensional Neural Networks

Neural continuum networks are an important aspect of the modeling of macroscopic parts of the cortex. Two classes of such networks are considered: voltage and activity based. In both cases, our networks contain an arbitrary number, n, of interacting neuron populations. Spatial nonsymmetric connectivity functions represent cortico-cortical, local connections, and external inputs represent nonlocal connections. Sigmoidal nonlinearities model the relationship between (average) membrane potential and activity. Departing from most of the previous work in this area, we do not assume the nonlinearity to be singular, that is, represented by the discontinuous Heaviside function. Another important difference from previous work is that we relax the assumption that the domain of definition where we study these networks is infinite, that is, equal to [Formula: see text] or [Formula: see text]. We explicitly consider the biologically more relevant case of a bounded subset Ω of [Formula: see text], a better model of a piece of cortex. The time behavior of these networks is described by systems of integro-differential equations. Using methods of functional analysis, we study the existence and uniqueness of a stationary (i.e., time-independent) solution of these equations in the case of a stationary input. These solutions can be seen as ‘persistent’; they are also sometimes called bumps. We show that under very mild assumptions on the connectivity functions and because we do not use the Heaviside function for the nonlinearities, such solutions always exist. We also give sufficient conditions on the connectivity functions for the solution to be absolutely stable, that is, independent of the initial state of the network. We then study the sensitivity of the solutions to variations of such parameters as the connectivity functions, the sigmoids, the external inputs, and, last but not least, the shape of the domain of existence Ω of the neural continuum networks. These theoretical results are illustrated and corroborated by a large number of numerical experiments in most of the cases 2 ⩽ n ⩽ 3, 2 ⩽ q ⩽ 3.

A granular extension of the fuzzy-ARTMAP (FAM) neural classifier based on fuzzy lattice reasoning (FLR)

The fuzzy lattice reasoning (FLR) classifier was introduced lately as an advantageous enhancement of the fuzzy-ARTMAP (FAM) neural classifier in the Euclidean space R N . This work extends FLR to space F N , where F is the granular data domain of fuzzy interval numbers (FINs) including (fuzzy) numbers, intervals, and cumulative distribution functions. Based on a fundamentally improved mathematical notation this work proposes novel techniques for dealing, rigorously, with imprecision in practice. We demonstrate a favorable comparison of our proposed techniques with alternative techniques from the literature in an industrial prediction application involving digital images represented by histograms. Additional advantages of our techniques include a capacity to represent statistics of all orders by a FIN, an introduction of tunable (sigmoid) nonlinearities, a capacity for effective data processing without any data normalization, an induction of descriptive decision-making knowledge (rules) from the training data, and the potential for input variable selection.

Persistent Neural States: Stationary Localized Activity Patterns in Nonlinear Continuousn-Population,q-Dimensional Neural Networks

Neural continuum networks are an important aspect of the modeling of macroscopic parts of the cortex. Two classes of such networks are considered: voltage and activity based. In both cases, our networks contain an arbitrary number, n, of interacting neuron populations. Spatial nonsymmetric connectivity functions represent cortico-cortical, local connections, and external inputs represent nonlocal connections. Sigmoidal nonlinearities model the relationship between (average) membrane potential and activity. Departing from most of the previous work in this area, we do not assume the nonlinearity to be singular, that is, represented by the discontinuous Heaviside function. Another important difference from previous work is that we relax the assumption that the domain of definition where we study these networks is infinite, that is, equal to R or R2. We explicitly consider the biologically more relevant case of a bounded subset omega of Rq, q = 1, 2, 3, a better model of a piece of cortex. The time behavior of these networks is described by systems of integro-differential equations. Using methods of functional analysis, we study the existence and uniqueness of a stationary (i.e., time-independent) solution of these equations in the case of a stationary input. These solutions can be seen as 'persistent'; they are also sometimes called bumps. We show that under very mild assumptions on the connectivity functions and because we do not use the Heaviside function for the nonlinearities, such solutions always exist. We also give sufficient conditions on the connectivity functions for the solution to be absolutely stable, that is, independent of the initial state of the network. We then study the sensitivity of the solutions to variations of such parameters as the connectivity functions, the sigmoids, the external inputs, and, last but not least, the shape of the domain of existence omega of the neural continuum networks. These theoretical results are illustrated and corroborated by a large number of numerical experiments in most of the cases 2 < or = n < or = 3, 2 < or = q < or = 3.

Chaotic dynamics on large networks

Many systems in nature are governed by a large number of agents that interact nonlinearly through complex feedback loops. When the networks are sufficiently large and interconnected, they typically exhibit self-organization and chaos. This paper examines the prevalence and degree of chaos on large unweighted recurrent networks of ordinary differential equations with sigmoidal nonlinearities and unit coupling. The largest Lyapunov exponent is used as the signature and measure of the chaos, and the study includes the effects of damping, asymmetries in the distribution of coupling strengths, network symmetry, and sparseness of connections. Minimum conditions and optimal network architectures are determined for the existence of chaos. The results have implications for the design of social and other networks in the real world in which weak chaos is deemed desirable or as a way of understanding why certain networks might exist on "the edge of chaos."

Absolute Stability and Complete Synchronization in a Class of Neural Fields Models

Neural fields are an interesting option for modeling macroscopic parts of the cortex involving several populations of neurons, like cortical areas. Two classes of neural field equations are considered: voltage- and activity-based. The spatio-temporal behavior of these fields is described by nonlinear integro-differential equations. The integral term, computed over a compact subset of $\mathbb{R}^q$, $q=1,2,3$, involves space and time varying, possibly nonsymmetric, intracortical connectivity kernels. Contributions from white matter afferents are represented as external input. Sigmoidal nonlinearities arise from the relation between average membrane potentials and instantaneous firing rates. Using methods of functional analysis, we characterize the existence and uniqueness of a solution of these equations for general, homogeneous (i.e., independent of the spatial variable), and spatially locally homogeneous inputs. In all cases we give sufficient conditions on the connectivity functions for the solutions to be absolutely stable, that is to say, asymptotically independent of the initial state of the field. These conditions bear on some compact operators defined from the connectivity kernels, the maximal slope of the sigmoids, and the time constants used in describing the temporal shape of the postsynaptic potentials. Numerical experiments are presented to illustrate the theory. To our knowledge this is the first time that such a complete analysis of the problem of the existence and uniqueness of a solution of these equations has been obtained. Another important contribution is the analysis of the absolute stability of these solutions—more difficult but more general than the linear stability analysis which it implies. The reason we have been able to complete this work is our use of the functional analysis framework and the theory of compact operators in a Hilbert space which has allowed us to provide simple mathematical answers to some of the questions raised by modelers in neuroscience.

Feed forward Artificial Neural Network model to predict the average summer-monsoon rainfall in India

In the present research, possibility of predicting average summer-monsoon rainfall over India has been analyzed through Artificial Neural Network models. In formulating the Artificial Neural Network based predictive model, three layered networks have been constructed with sigmoid non-linearity. The models under study are different in the number of hidden neurons. After a thorough training and test procedure, neural net with three nodes in the hidden layer is found to be the best predictive model.

Nonlinear modeling of protein expressions in protein arrays

This paper addresses the problem of estimating the expressions or concentrations of proteins from measurements obtained from protein arrays and illustrates the methodology on lysate microarray data. With several families of parametric models we design a number of algorithms for the estimation of a highly nonlinear calibration curve as well as the concentrations themselves. The model families include polynomial and sigmoidal nonlinearities for the calibration curve and homoscedastic or heteroscedastic models for the noise. The accuracy of the estimation methods is tested on simulated data and applied to real lysate array data. The results are generally very good, provided that strongly nonlinear models are used.