Sigmoidal Nonlinearity Research Articles

Comparative dynamics analysis of the environmental policy efficacy in a nonlinear Cournot duopoly with differentiated goods and emission charges

The price and the exchanged quantity volatility observed in real-world markets may be explained, according to the existing empirical literature, in terms of the endogenous fluctuations generated by the presence of nonlinearities. We then replace with a sigmoid adaptive best response mechanism the linear partial adjustment best response rule considered in Mamada and Perrings (2020), where the effect produced by quadratic emission charges on the dynamics of a Cournot duopoly model with homogeneous goods was investigated. Moreover, the sigmoid nonlinearity, in addition to being well suited to describe the bounded output variations caused by physical, historical and institutional constraints, makes the model able to generate interesting, non-divergent dynamic outcomes, despite the linearity of the demand function and of marginal costs. Additionally, following the suggestion in Mamada and Perrings (2020), we deal with the more general case of differentiated products. Beyond analytically studying the stability of the unique steady state, coinciding with the Nash equilibrium, and the effect produced by the main parameters on the stability region, we perform two comparative dynamics investigations which allow to evaluate the environmental policy efficacy when the Nash equilibrium is not stable and thus the standard comparative statics technique does not fit for the purpose. In particular, the former analysis, which is based on a comparison of emissions for different levels of charges, shows that, also in case the Nash equilibrium is not stable, the considered environmental policy may be effective both with complements and with substitutes. The latter investigation, consisting in a comparison of emissions along non-stationary trajectories and along the equilibrium path, in the proposed experiments highlights that emissions are larger along non-stationary trajectories. This gives us the opportunity to show how to act on the level of the asymptotes of the sigmoid adjustment mechanism to reduce output variations, reaching at one time a complete stabilization of the system and limiting pollution.

Input-size dependence of the baroreflex neural arc transfer characteristics during Gaussian white noise inputs.

Although Gaussian white noise (GWN) inputs offer a theoretical framework for identifying higher-order nonlinearity, an actual application to the data of the neural arc of the carotid sinus baroreflex did not succeed in fully predicting the well-known sigmoidal nonlinearity. In the present study, we assumed that the neural arc can be approximated by a cascade of a linear dynamic (LD) component and a nonlinear static (NS) component. We analyzed the data obtained using GWN inputs with a mean of 120 mmHg and standard deviations (SDs) of 10, 20, and 30 mmHg for 15 min each in anesthetized rats (n = 7). We first estimated the linear transfer function from carotid sinus pressure to sympathetic nerve activity (SNA) and then plotted the measured SNA against the linearly predicted SNA. The predicted and measured data pairs exhibited an inverse sigmoidal distribution when grouped into 10 bins based on the size of the linearly predicted SNA. The sigmoidal nonlinearity estimated via the LD-NS model showed a midpoint pressure (104.1 ± 4.4 mmHg for SD of 30 mmHg) lower than that estimated by a conventional stepwise input (135.8 ± 3.9 mmHg, P < 0.001). This suggests that the NS component is more likely to reflect the nonlinearity observed during pulsatile inputs that are physiological to baroreceptors. Furthermore, the LD-NS model yielded higher R2 values compared with the linear model and the previously suggested second-order Uryson model in the testing dataset.NEW & NOTEWORTHY We examined the input-size dependence of the baroreflex neural arc transfer characteristics during Gaussian white noise inputs. A linear dynamic-static nonlinear model yielded higher R2 values compared with a linear model and captured the well-known sigmoidal nonlinearity of the neural arc, indicating that the nonlinear dynamics contributed to determining sympathetic nerve activity. Ignoring such nonlinear dynamics might reduce our ability to explain underlying physiology and significantly limit the interpretation of experimental data.

Instance Segmentation Evaluation For Traffic Signs

This research paper focuses on developing a traffic sign recognition system based on the You Only Look At Coefficients (YOLACT) model, a one-stage instance segmentation model that offers high performance in terms of accuracy and reliability. However, the performance of YOLACT is influenced by various conditions such as day/night and different angles of objects. Therefore, this study aims to evaluate the impact of different angles and environments on the performance of the system. The paper discusses the framework, backbone structure, prototype generation branch, mask coefficient, and mask assembly used in the system. ResNet-101 and ResNet-50 were used as the backbone structure to extract feature maps of objects in the input image. The prototype generation branch generates prototype masks using fully convolutional networks (FCN), and the mask coefficient branch generates the Mask assembly using the sigmoid nonlinearity. Two models, YOLACT and Mask-RCNN, were evaluated in terms of mean precision (mAP) and frames per second (FPS) with the front view dataset. The results show that YOLACT outperforms Mask-RCNN in terms of accuracy and speed. For an image resolution of 550x550, YOLACT with Resnet-101 was considered the best model in this article since it achieves over 80% precision, recall, specificity, and accuracy in various conditions such as day, night, left and right, and forward-looking angles.

Identification of fractional Hammerstein model for electrical stimulated muscle: An application of fuzzy-weighted differential evolution

Fractional order representations model complex systems with less parameters, improved accuracy and enhanced robustness in control systems but the system identification of fractional order systems is highly complicated and challenging task because of substantial nonlinearity of the input, unknown linear/nonlinear parameters of the blocks, and unknown fractional order. In this paper, parameter estimation technique is proposed based on fuzzy-weighted differential evolution algorithm for fractional electrically stimulated muscle models, which is generalization of fractional Hammerstein controlled autoregressive model vital for rehabilitation of spinal cord injury (SCI) patients. The system identification problem of fractional electrically stimulated muscle models (FESMMs) is formulated via mean square error approximation between the true and estimated response of FESMMs. The parameters of FESMMs are identified by employing fuzzy weighted differential evolution algorithm optimization knacks of with polynomial, cubic spline and sigmoidal input nonlinearities on various noise variances in the system dynamics. Comparison of results from actual to calculated responses indicates closeness up to 4 decimal places of accuracy for FESMM with polynomial type nonlinearity, up to 6 decimal places for FESMM with sigmoidal type kernel, and up to 5 decimal for FESMM with cubic spline type nonlinearity for different low and high signal to noise scenarios i.e., σ2=0.0022,0.022,0.22, which verify the precision, efficacy, stability and reliability of the fuzzy weighted differential evolution algorithm for system identification of FESMMs in rehabilitation scenarios of SCI.

The Role of Frustration in Collective Decision-Making Dynamical Processes on Multiagent Signed Networks

In this article, we consider a collective decision-making process in a network of agents described by a nonlinear interconnected dynamical model with sigmoidal nonlinearities and signed interaction graph. The decisions are encoded in the equilibria of the system. The aim is to investigate this multiagent system when the signed graph representing the community is not structurally balanced and in particular as we vary its frustration, i.e., its distance to structural balance. The model exhibits bifurcations, and a “social effort” parameter, added to the model to represent the strength of the interactions between the agents, plays the role of bifurcation parameter in our analysis. We show that, as the social effort increases, the decision-making dynamics exhibit a pitchfork bifurcation behavior where, from a deadlock situation of “no decision” (i.e., the origin is the only globally stable equilibrium point), two possible (alternative) decision states for the community are achieved (corresponding to two nonzero locally stable equilibria). The value of social effort for which the bifurcation is crossed (and a decision is reached) increases with the frustration of the signed network.

Dynamic Modeling of a Nonlinear Two-Wheeled Robot Using Data-Driven Approach

A system identification of a two-wheeled robot (TWR) using a data-driven approach from its fundamental nonlinear kinematics is investigated. The fundamental model of the TWR is implemented in a Simulink environment and tested at various input/output operating conditions. The testing outcome of TWR’s fundamental dynamics generated 12 datasets. These datasets are used for system identification using simple autoregressive exogenous (ARX) and non-linear auto-regressive exogenous (NLARX) models. Initially the ARX structure is heuristically selected and estimated through a single operating condition. We conclude that the single ARX model does not satisfy TWR dynamics for all datasets in term of fitness. However, NLARX fitted the 12 estimated datasets and 2 validation datasets using sigmoid nonlinearity. The obtained results are compared with TWR’s fundamental dynamics and predicted outputs of the NLARX showed more than 98% accuracy at various operating conditions.

Neural Network Identifiability for a Family of Sigmoidal Nonlinearities

This paper addresses the following question of neural network identifiability: Does the input–output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? The existing literature on the subject (Sussman in Neural Netw 5(4):589–593, 1992; Albertini et al. in Artificial neural networks for speech and vision, 1993; Fefferman in Rev Mat Iberoam 10(3):507–555, 1994) suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided that the networks under consideration satisfy certain “genericity conditions.” The results in Sussman (1992) and Albertini et al. (1993) apply to networks with a single hidden layer and in Fefferman (1994) the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.

Equilibria and their Stability in Networks with Steep Sigmoidal Nonlinearities

In this paper we investigate equilibria of continuous differential equation models of network dynamics. The motivation comes from gene regulatory networks where each directed edge represents either...

Finite-time synchronization of multi-scroll chaotic systems with sigmoid non-linearity and uncertain terms

In this article, the finite-time synchronization of multi-scroll chaotic systems with sigmoid non-linearity in the presence of uncertain stochastic parameters has been studied. The finite-time synchronization is a popular topic from a practical point of view in engineering, which has many applications in the areas like chaos-based communication, image and voice encryptions. The present chaotic system scrolls (up to five) are incorporated with the help of hyperbolic tangent functions (sigmoid non-linearity), but in real applications, there is no limit of multi-scroll in a chaotic system. Finite-time synchronization is achieved with the help of Lyapunov stability using some lemmas and definitions. The Lyapunov exponents are also calculated for multi-scroll chaotic systems with uncertainties and it confirms that the multi-scroll system is chaotic with uncertainties. A drive has been taken to achieve finite-time synchronization of multi-scroll chaotic systems with uncertain parameters, where the multi-scrolls are generated by nonlinear functions.

Initializing photonic feed-forward neural networks using auxiliary tasks

Photonics is among the most promising emerging technologies for providing fast and energy-efficient Deep Learning (DL) implementations. Despite their advantages, these photonic DL accelerators also come with certain important limitations. For example, the majority of existing photonic accelerators do not currently support many of the activation functions that are commonly used in DL, such as the ReLU activation function. Instead, sinusoidal and sigmoidal nonlinearities are usually employed, rendering the training process unstable and difficult to tune, mainly due to vanishing gradient phenomena. Thus, photonic DL models usually require carefully fine-tuning all their training hyper-parameters in order to ensure that the training process will proceed smoothly. Despite the recent advances in initialization schemes, as well as in optimization algorithms, training photonic DL models is still especially challenging. To overcome these limitations, we propose a novel adaptive initialization method that employs auxiliary tasks to estimate the optimal initialization variance for each layer of a network. The effectiveness of the proposed approach is demonstrated using two different datasets, as well as two recently proposed photonic activation functions and three different initialization methods. Apart from significantly increasing the stability of the training process, the proposed method can be directly used with any photonic activation function, without further requiring any other kind of fine-tuning, as also demonstrated through the conducted experiments.

Asymmetric Sigmoidal Activation Function for Feed-Forward Artificial Neural Networks

Artificial neural networks of the feed – forward kind, are an established technique under the supervised learning paradigm for the solution of learning tasks. The mathematical result that allows one to assert the usefulness of this technique is that these networks can approximate any continuous function to the desired degree. The requirement imposed on these networks is to have non-linear functions of a specific kind at the hidden nodes of the network. In general, sigmoidal non-linearities, called activation functions, are generally used. In this paper we propose an asymmetric activation function. The networks using the proposed activation function are compared against those using the generally used logistic and the hyperbolic tangent activation function for the solution of 12 function approximation problems. The results obtained allow us to infer that the proposed activation function, in general, reaches deeper minima of the error measures and has better generalization error values.

Backtracking search heuristics for identification of electrical muscle stimulation models using Hammerstein structure

The electrical muscle stimulation models (EMSMs) are effectively described through Hammerstein structure and are used to restore the functionality of paralyzed muscles after spinal cord injury (SCI). In the present study, global search efficacy of evolutionary computing paradigm through backtracking search algorithm (BSA) is exploited for parameter estimation of EMSMs. The approximation theory in mean squared error sense is used for the construction of a merit function for EMSMs based on deviation between optimal and approximated parameters. Variants of BSA are designed based on memory size and population dynamics for the minimization problem of EMSMs having cubic spline as well as sigmoidal nonlinearities. Comparative studies by means of rigorous statistics establish the worth of scheme for effective, accurate, reliable, robust and stable identification of EMSMs in rehabilitation scenarios of SCI.

Multiscroll chaotic system with sigmoid nonlinearity and its fractional order form with synchronization application

In this paper, a multiscroll snap oscillator with hyperbolic tangent function is proposed. There is no limitation in the number of scrolls and it can be increased by proper choice of a specific function. The Lyapunov exponents of the proposed system are obtained to testify the chaotic behavior of the system. Fractional order multiscroll system is derived from its integer order model by using the Adams–Bashforth–Moulton algorithm. A new scheme is applied in order to investigate the synchronization of the multiscroll systems. The main objective of the paper is to propose a multiscroll attractor and show that the number of scrolls can be controlled by the only nonlinear function. Such systems are less investigated in the literatures and has many real time applications like image and voice encryption, random number generators, chaos based communication systems and so on.

Phase Locking of Auditory-Nerve Fibers Reveals Stereotyped Distortions and an Exponential Transfer Function with a Level-Dependent Slope.

Phase locking of auditory-nerve-fiber (ANF) responses to the fine structure of acoustic stimuli is a hallmark of the auditory system's temporal precision and is important for many aspects of hearing. Period histograms from phase-locked ANF responses to low-frequency tones exhibit spike-rate and temporal asymmetries, but otherwise retain an approximately sinusoidal shape as stimulus level increases, even beyond the level at which the mean spike rate saturates. This is intriguing because apical cochlear mechanical vibrations show little compression, and mechanoelectrical transduction in the receptor cells is thought to obey a static sigmoidal nonlinearity, which might be expected to produce peak clipping at moderate and high stimulus levels. Here we analyze phase-locked responses of ANFs from cats of both sexes. We show that the lack of peak clipping is due neither to ANF refractoriness nor to spike-rate adaptation on time scales longer than the stimulus period. We demonstrate that the relationship between instantaneous pressure and instantaneous rate is well described by an exponential function whose slope decreases with increasing stimulus level. Relatively stereotyped harmonic distortions in the input to the exponential can account for the temporal asymmetry of the period histograms, including peak splitting. We show that the model accounts for published membrane-potential waveforms when assuming a power-of-three, but not a power-of-one, relationship to exocytosis. Finally, we demonstrate the relationship between the exponential transfer functions and the sigmoidal pseudotransducer functions obtained in the literature by plotting the maxima and minima of the voltage responses against the maxima and minima of the stimuli.SIGNIFICANCE STATEMENT Phase locking of auditory-nerve-fiber responses to the temporal fine structure of acoustic stimuli is important for many aspects of hearing, but the mechanisms underlying phase locking are not fully understood. Intriguingly, period histograms retain an approximately sinusoidal shape across sound levels, even when the mean rate has saturated. We find that neither refractoriness nor spike-rate adaptation is responsible for this behavior. Instead, the peripheral auditory system operates as though it contains an exponential transfer function whose slope changes with stimulus level. The underlying mechanism is distinct from the comparatively weak cochlear mechanical compression in the cochlear apex, and likely resides in the receptor cells.

MLP based predictive model for surface ozone concentration over an urban area in the Gangetic West Bengal during pre-monsoon season

The present paper reports a comparative study among two neurocomputing models in the form of Multilayer Perceptron (MLP) models and non-linear regression for the prediction of surface ozone (O3) during pre-monsoon season over Gangetic West Bengal (GWB), India considering NOx, SO2, PM10 and temperature as predictors. Learning the MLPs through gradient descent (GD) with tanhyperbolic and sigmoid nonlinearities, we found that all the models under consideration have almost the same degrees of prediction efficiency for O3 over GWB during pre-monsoon season with the said predictors. However, the MLP model with tanhyperbolic activation function is found to produce a significantly higher correlation and Willmott's index of agreement between actual and predicted O3 than the other models. Finally, MLP with GD learning characterized by tanhyperbolic nonlinearity is identified to have significant efficiency in surface ozone prediction over the region as mentioned above.

On-line detection of qualitative dynamical changes in nonlinear systems: The resting-oscillation case

Motivated by neuroscience applications, we introduce the concept of qualitative detection, that is, the problem of determining on-line the current qualitative dynamical behavior (e.g., resting, oscillating, bursting, spiking etc.) of a nonlinear system. The approach is thought for systems characterized by i) large parameter variability and redundancy, ii) a small number of possible robust, qualitatively different dynamical behaviors and, iii) the presence of sharply different characteristic timescales. These properties are omnipresent in neurosciences and hamper quantitative modeling and fitting of experimental data. As a result, novel control theoretical strategies are needed to face neuroscience challenges like on-line epileptic seizure detection. The proposed approach aims at detecting the current dynamical behavior of the system and whether a qualitative change is likely to occur without quantitatively fitting any model nor asymptotically estimating any parameter. We talk of qualitative detection. We rely on the qualitative properties of the system dynamics, extracted via singularity and singular perturbation theories, to design low dimensional qualitative detectors. We introduce this concept on a general class of singularly perturbed systems and then solve the problem for an analytically tractable class of two-dimensional systems with a single unknown sigmoidal nonlinearity and two sharply separated timescales. Numerical results are provided to show the performance of the designed qualitative detector.

Harmonics added to a flickering light can upset the balance between ON and OFF pathways to produce illusory colors

The neural signals generated by the light-sensitive photoreceptors in the human eye are substantially processed and recoded in the retina before being transmitted to the brain via the optic nerve. A key aspect of this recoding is the splitting of the signals within the two major cone-driven visual pathways into distinct ON and OFF branches that transmit information about increases and decreases in the neural signal around its mean level. While this separation is clearly important physiologically, its effect on perception is unclear. We have developed a model of the ON and OFF pathways in early color processing. Using this model as a guide, we can produce imbalances in the ON and OFF pathways by changing the shapes of time-varying stimulus waveforms and thus make reliable and predictable alterations to the perceived average color of the stimulus-although the physical mean of the waveforms does not change. The key components in the model are the early half-wave rectifying synapses that split retinal photoreceptor outputs into the ON and OFF pathways and later sigmoidal nonlinearities in each pathway. The ability to systematically vary the waveforms to change a perceptual quality by changing the balance of signals between the ON and OFF visual pathways provides a powerful psychophysical tool for disentangling and investigating the neural workings of human vision.

Qualitative parameter estimation for a class of relaxation oscillators

Motivated by neuroscience applications, we introduce the concept of qualitative estimation as an adaptation of classical parameter estimation to nonlinear systems characterized by i) the presence of possibly many redundant parameters, ii) a small number of possible qualitatively different behaviors, iii) the presence of sharply different characteristic timescales and, consequently, iv) the generic impossibility of quantitatively modeling and fitting experimental data. As a first application, we illustrate these ideas on a class of nonlinear systems with a single unknown sigmoidal nonlinearity and two sharply separated timescales. This class of systems is shown to exhibit either global asymptotic stability or relaxation oscillations depending on a single ruling parameter and independently of the exact shape of the nonlinearity. We design and analyze a qualitative estimator that estimates the distance of the ruling parameter from the unknown critical value at which the transition between the two behaviors happens without using any quantitative fitting of the measured data.

Local Dynamics in Trained Recurrent Neural Networks.

Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task-related neural dynamics, we study trained recurrent neural networks. We develop a mean field theory for reservoir computing networks trained to have multiple fixed point attractors. Our main result is that the dynamics of the network's output in the vicinity of attractors is governed by a low-order linear ordinary differential equation. The stability of the resulting equation can be assessed, predicting training success or failure. As a consequence, networks of rectified linear units and of sigmoidal nonlinearities are shown to have diametrically different properties when it comes to learning attractors. Furthermore, a characteristic time constant, which remains finite at the edge of chaos, offers an explanation of the network's output robustness in the presence of variability of the internal neural dynamics. Finally, the proposed theory predicts state-dependent frequency selectivity in the network response.

A Mathematical Motivation for Complex-Valued Convolutional Networks

A complex-valued convolutional network (convnet) implements the repeated application of the following composition of three operations, recursively applying the composition to an input vector of nonnegative real numbers: (1) convolution with complex-valued vectors, followed by (2) taking the absolute value of every entry of the resulting vectors, followed by (3) local averaging. For processing real-valued random vectors, complex-valued convnets can be viewed as data-driven multiscale windowed power spectra, data-driven multiscale windowed absolute spectra, data-driven multiwavelet absolute values, or (in their most general configuration) data-driven nonlinear multiwavelet packets. Indeed, complex-valued convnets can calculate multiscale windowed spectra when the convnet filters are windowed complex-valued exponentials. Standard real-valued convnets, using rectified linear units (ReLUs), sigmoidal (e.g., logistic or tanh) nonlinearities, or max pooling, for example, do not obviously exhibit the same exact correspondence with data-driven wavelets (whereas for complex-valued convnets, the correspondence is much more than just a vague analogy). Courtesy of the exact correspondence, the remarkably rich and rigorous body of mathematical analysis for wavelets applies directly to (complex-valued) convnets.