Discrete Neural Network Research Articles

Design, Analysis, and Application of a Discrete Error Redefinition Neural Network for Time-Varying Quadratic Programming.

Time-varying quadratic programming (TV-QP) is widely used in artificial intelligence, robotics, and many other fields. To solve this important problem, a novel discrete error redefinition neural network (D-ERNN) is proposed. By redefining the error monitoring function and discretization, the proposed neural network is superior to some traditional neural networks in terms of convergence speed, robustness, and overshoot. Compared with the continuous ERNN, the proposed discrete neural network is more suitable for computer implementation. Unlike continuous neural networks, this article also analyzes and proves how to select the parameters and step size of the proposed neural networks to ensure the reliability of the network. Moreover, how to achieve the discretization of the ERNN is presented and discussed. The convergence of the proposed neural network without disturbance is proven, and bounded time-varying disturbances can be resisted in theory. Furthermore, the comparison results with other related neural networks show that the proposed D-ERNN has a faster convergence speed, better antidisturbance ability, and lower overshoot.

Complex Dynamical Behavior of Locally Active Discrete Memristor-Coupled Neural Networks with Synaptic Crosstalk: Attractor Coexistence and Reentrant Feigenbaum Trees

In continuous neural modeling, memristor coupling has been investigated widely. Yet, there is little research on discrete neural networks in the field. Discrete models with synaptic crosstalk are even less common. In this paper, two locally active discrete memristors are used to couple two discrete Aihara neurons to form a map called DMCAN. Then, the synapse is modeled using a discrete memristor and the DMCAN map with crosstalk is constructed. The DMCAN map is investigated using phase diagram, chaotic sequence, Lyapunov exponent spectrum (LEs) and bifurcation diagrams (BD). Its rich and complex dynamical behavior, which includes attractor coexistence, state transfer, Feigenbaum trees, and complexity, is systematically analyzed. In addition, the DMCAN map is implemented in hardware on a DSP platform. Numerical simulations are further validated for correctness. Numerical and experimental findings show that the synaptic connections of neurons can be modeled by discrete memristor coupling which leads to the construction of more complicated discrete neural networks.

A Forward Learning Algorithm for Neural Memory Ordinary Differential Equations.

The deep neural network, based on the backpropagation learning algorithm, has achieved tremendous success. However, the backpropagation algorithm is consistently considered biologically implausible. Many efforts have recently been made to address these biological implausibility issues, nevertheless, these methods are tailored to discrete neural network structures. Continuous neural networks are crucial for investigating novel neural network models with more biologically dynamic characteristics and for interpretability of large language models. The neural memory ordinary differential equation (nmODE) is a recently proposed continuous neural network model that exhibits several intriguing properties. In this study, we present a forward-learning algorithm, called nmForwardLA, for nmODE. This algorithm boasts lower computational dimensions and greater efficiency. Compared with the other learning algorithms, experimental results on MNIST, CIFAR10, and CIFAR100 demonstrate its potency.

Fractional discrete neural networks with variable order: solvability, finite time stability and synchronization

Fractional discrete neural networks with variable order: solvability, finite time stability and synchronization

Dynamic Analysis, Image Encryption Application and FPGA Implementation of a Discrete Memristor-Coupled Neural Network

This paper presents a novel discrete memristor model that incorporates exponential and absolute value functions. A discrete coupled memristor neural network model is constructed based on this memristor design. The periodic and chaotic regions of the discrete neural network model are determined using bifurcation and Lyapunov exponent spectrum. Furthermore, by varying the initial values of the discrete memristive neural network, we observe the coexistence of chaos and periodic attractors, as well as periodic attractors. Additionally, an application to color image encryption based on the discrete system model is given. Security analysis is conducted in the aspects of key space, histogram analysis, correlation analysis, sensitivity analysis, Peak Signal-to-Noise Ratio (PSNR), and information entropy analysis. The analysis results show that the algorithm has a key space size of [Formula: see text], and the information entropy of baboon graph is 7.9993, which is very close to the ideal value of 8. It shows that the image encryption algorithm is feasible and effective. Finally, the implementation of the discrete memristive neural network model is realized using Field Programmable Gate Array (FPGA). The experimental implementation is conducted using the Verilog language on the Vivado 2018.3 platform, and the obtained results align with the numerical simulation results obtained through MATLAB.

Mean-squared exponential stability of distributed almost periodic sequence for nonlocal space-time discrete shunting inhibitory cellular neural networks with Brownian motions

The discrete-time and discrete-space model for stochastic nonlocal shunting inhibitory cellular neural networks with reaction diffusions are modelled in the first time. Owing to the consideration of spatial variables, the discrete model derived in this article is more complex than the traditional ordinary difference model. In accordance with the constant-variation-formula for discrete-time and discrete-space model, Banach contracting mapping principle, the method of proof by contradiction and stochastic calculus, we also obtain the existence of a unique bounded almost periodic sequence for the discrete model, which is exponentially stable in the mean-square sense. Noting that it is the first time to consider the dynamics of almost periodicity in distribution of time-space discrete neural network models. The practicability of the present results is demonstrated by means of an illustration.

Coexisting discharge and synchronization of heterogeneous discrete neural network with crosstalk memristor synapses

Synaptic crosstalk, which occurs due to the overflow of neurotransmitters between neighboring synapses, holds a crucial position in shaping the discharge characteristics and signal transmission within nervous systems. In this work, two memristors are employed to simulate biological neural synapses and bidirectionally coupled Chialvo discrete neuron and Rulkov discrete neuron. Thus, a heterogeneous discrete neural network with memristor-synapse coupling is constructed, with the crosstalk behavior between memristor synapses in the coupled state taken into account. The analysis demonstrates that the quantity and stability of fixed points within this neural network greatly depend on the strength of synaptic crosstalk. Additionally, through a thorough investigation of bifurcation diagrams, phase diagrams, Lyapunov exponents, and time sequences, we uncover the multi-stable state property exhibited by the neural network. This characteristic manifests as the coexistence of diverse discharge behaviors, which significantly change with the intensity of synaptic crosstalk. Interestingly, the introduction of control parameter into state variables can lead the bias to increase, and also the infinite stable states to occur in the neural network. Furthermore, we comprehensively study the influence of synaptic crosstalk strength on the synchronization behavior of the neural network, with consideration of various coupling strengths, initial conditions, and parameters. Our analysis, which is based on the phase difference and synchronization factor of neuronal discharge sequences, reveales that the neural network maintains phase synchronization despite the variations of the two crosstalk strengths. The insights gained from this work provide important support for elucidating the electrophysiological mechanisms behind the processing and transmission of biological neural information. Especially, the coexisting discharge phenomenon in the neural network provides an electrophysiological theoretical foundation for the clinical symptoms and diagnosis of the same neurological disease among different individuals or at different stages. And the doctors can predict the progression and prognosis of neurological disease based on the patterns and characteristics of coexisting discharge in patients, enabling them to adopt appropriate intervention measures and monitoring plans. Therefore, the research on coexisting discharge in the neural system contributes to the comprehensive treatment of nervous system disease.

A memristor-coupled heterogeneous discrete neural networks with infinite multi-structure hyperchaotic attractors

A memristor-coupled heterogeneous discrete neural networks with infinite multi-structure hyperchaotic attractors

Dynamical behavior of memristor-coupled heterogeneous discrete neural networks with synaptic crosstalk

Synaptic crosstalk is a prevalent phenomenon among neuronal synapses, playing a crucial role in the transmission of neural signals. Therefore, considering synaptic crosstalk behavior and investigating the dynamical behavior of discrete neural networks are highly necessary. In this paper, we propose a heterogeneous discrete neural network (HDNN) consisting of a three-dimensional KTz discrete neuron and a Chialvo discrete neuron. These two neurons are coupled mutually by two discrete memristors and the synaptic crosstalk is considered. The impact of crosstalk strength on the firing behavior of the HDNN is explored through bifurcation diagrams and Lyapunov exponents. It is observed that the HDNN exhibits different coexisting attractors under varying crosstalk strengths. Furthermore, the influence of different crosstalk strengths on the synchronized firing of the HDNN is investigated, revealing a gradual attainment of phase synchronization between the two discrete neurons as the crosstalk strength decreases.

Finite Time Stability Results for Neural Networks Described by Variable-Order Fractional Difference Equations

Variable-order fractional discrete calculus is a new and unexplored part of calculus that provides extraordinary capabilities for simulating multidisciplinary processes. Recognizing this incredible potential, the scientific community has been researching variable-order fractional discrete calculus applications to the modeling of engineering and physical systems. This research makes a contribution to the topic by describing and establishing the first generalized discrete fractional variable order Gronwall inequality that we employ to examine the finite time stability of nonlinear Nabla fractional variable-order discrete neural networks. This is followed by a specific version of a generalized variable-order fractional discrete Gronwall inequality described using discrete Mittag–Leffler functions. A specific version of a generalized variable-order fractional discrete Gronwall inequality represented using discrete Mittag–Leffler functions is shown. As an application, utilizing the contracting mapping principle and inequality approaches, sufficient conditions are developed to assure the existence, uniqueness, and finite-time stability of the equilibrium point of the suggested neural networks. Numerical examples, as well as simulations, are provided to show how the key findings can be applied.

Nonlinear dynamics of a three-dimensional discrete-time delay neural network

This paper examines a three-dimensional delayed discrete neural network model analytically and numerically to determine the existence of different types of bifurcations of the involved fixed points. The model exhibits different bifurcations such as pitchforks, flips, Neimark–Sackers, and flip-Neimark–Sackers. The critical coefficients are used to determine the structure of each bifurcation. The curves are calculated and plotted for each bifurcation when the parameters are changed. Further, these bifurcations are theoretically analyzed and numerically verified. From the obtained results, we observed that by drawing the curves associated with each bifurcation, the numerical simulations are consistent with the analytical results.

Bifurcations and complex dynamics of a two dimensional neural network model with delayed discrete time

This paper focuses on the different bifurcations of fixed points of a delayed discrete neural network model analytically and numerically. The conditions and critical values of different bifurcations including the pitchfork, flip, Neimark–Sacker, and flip–Neimark–Sacker are analyzed. By using the critical coefficients, the structure for each bifurcation are determined. By taking one and two parameters, the critical coefficients are calculated and the curves associated with each bifurcation are plotted. The numerical simulation results demonstrate the effectiveness and feasibility of the proposed method.

Discrete memristor applied to construct neural networks with homogeneous and heterogeneous coexisting attractors

Memristors are widely used to simulate the effects of electromagnetic radiation on neurons or as synapses to simulate excitation and inhibition between neurons. This paper constructs discrete neural network models (DNNMs) in three scenarios: without discrete memristor, discrete memristor simulate electromagnetic radiation stimulation, and discrete local active memristor simulate synapses. Research reveals that the DNNM without fixed points under electromagnetic radiation stimulation, which in turn induces hidden hyperchaotic attractors and also observes the existence of infinite coexisting homogeneous attractors. Meanwhile, the DNNM can generate heterogeneous coexisting attractors after introducing a memristor to mimic synapses. Numerical results show that discrete memristors can induce the DNNM to develop more complex chaotic dynamics. In addition, a hardware platform is designed to validate the numerical results, and the DNNMs are applied to construct the pseudo-random number generator (PRNG).

The new fractional discrete neural network model under electromagnetic radiation: Chaos, control and synchronization

This paper describes a new four-dimensional fractional discrete neural network with electromagnetic radiation model. In addition, the non-linear dynamics of the suggested model are examined, within the framework of commensurate, incommensurate and variable orders, through different numerical techniques such as Lyapunov exponent, phase portraits, bifurcation diagrams, and the 0–1 test method. The results imply that The behaviours of the proposed fractional discrete neural network model have rich and complex dynamical properties that are influenced by the variation of the system parameters, the commensurate order, the incommensurate order and the variable order. Moreover, the approximate entropy test and Co algorithm are carried out to measure complexity and validate the presence of chaos. Finally, nonlinear controllers are illustrated to stabilize and synchronize the proposed model.

Prospects for control methods in engineering systems

The article discusses the prerequisites and natural consequences of the control methods development in engineering systems: (1) a simple deviation and perturbation controller, (2) a fuzzy logic controller with a fuzzifier and a rule base, (3) a neural network controller for dynamically adjusting the coefficients of the corresponding links, (4) a discrete neural network controller with a neural approximator and controller. The experience gained by researchers and engineers since the first description of the principles of regulation in 1910 and the level of information technologies design, in particular the neural network method of machine learning and the enormous computing potential of computer devices, today can be integrated into a fundamentally new method of discrete neural network regulation.
 The review carried out in the article is aimed at identifying and demonstrating the significance of experimental and operational data, which must be properly structured and marked up at the stage of their collection and archiving. It is this approach that will allow us to quickly implement neural network controllers in engineering systems, since the most important stage for their creation is the process of learning and optimizing the architecture of neural networks.
 The principle of operation, advantages and disadvantages, the optimal stages of a neural network controller improvement based on two neural networks for the formation of a control strategy, taking into account the most probable state of the system at the next point in time, are given.

Quantum annealing for neural network optimization problems: A new approach via tensor network simulations

Here, we focus on the problem of minimizing complex classical cost functions associated with prototypical discrete neural networks, specifically the paradigmatic Hopfield model and binary perceptron. We show that the adiabatic time evolution of QA can be efficiently represented as a suitable Tensor Network. This representation allows for simple classical simulations, well-beyond small sizes amenable to exact diagonalization techniques. We show that the optimized state, expressed as a Matrix Product State (MPS), can be recast into a Quantum Circuit, whose depth scales only linearly with the system size and quadratically with the MPS bond dimension. This may represent a valuable starting point allowing for further circuit optimization on near-term quantum devices.

On Variable-Order Fractional Discrete Neural Networks: Existence, Uniqueness and Stability

Given the recent advances regarding the studies of discrete fractional calculus, and the fact that the dynamics of discrete-time neural networks in fractional variable-order cases have not been sufficiently documented, herein, we consider a novel class of discrete-time fractional-order neural networks using discrete nabla operator of variable-order. An adequate criterion for the existence of the solution in addition to its uniqueness for such systems is provided with the use of Banach fixed point technique. Moreover, the uniform stability is investigated. We provide at the end two numerical simulations illustrating the relevance of the aforementioned results.

Dynamic Behavior Analysis and Synchronization of Memristor-Coupled Heterogeneous Discrete Neural Networks

Continuous memristors have been widely studied in recent years; however, there are few studies on discrete memristors in the field of neural networks. In this paper, a four-stable locally active discrete memristor (LADM) is proposed as a synapse, which is used to connect a two-dimensional Chialvo neuron and a three-dimensional KTZ neuron, and construct a simple heterogeneous discrete neural network (HDNN). Through a bifurcation diagram and Lyapunov exponents diagram, the period and chaotic regions of the discrete neural network model are shown. Through numerical analysis, it was found that the chaotic region and periodic region of the neural network based on DLAM are significantly improved. In addition, coexisting chaos and chaos attractors, coexisting periodic and chaotic attractors, and coexisting periodic and periodic attractors will appear when the initial value of the LADM is changed. Coupled by a LADM synapse, two heterogeneous discrete neurons are gradually synchronized by changing the coupling strength. This paper lays a good foundation for the future analysis of LADMs and the related research of discrete neural networks coupled by LADMs.

THE ROLE OF STIFFNESS IN TRAINING AND GENERALIZATION OF RESNETS

Neural ordinary differential equations (NODEs) have recently regained popularity as large-depth limits of a large class of neural networks. In particular, residual neural networks (ResNets) are equivalent to an explicit Euler discretization of an underlying NODE, where the transition from one layer to the next is one time step of the discretization. The relationship between continuous and discrete neural networks has been of particular interest. Notably, analysis from the ordinary differential equation viewpoint can potentially lead to new insights for understanding the behavior of neural networks in general. In this work, we take inspiration from differential equations to define the concept of stiffness for a ResNet via the interpretation of a ResNet as the discretization of a NODE. We then examine the effects of stiffness on the ability of a ResNet to generalize, via computational studies on example problems coming from climate and chemistry models. We find that penalizing stiffness does have a unique regularizing effect, but we see no benefit to penalizing stiffness over L<sup>2</sup> regularization (penalization of network parameter norms) in terms of predictive performance.

Finding Common Ground on the Site of Onset of Amyotrophic Lateral Sclerosis.

The fundamental origin of amyotrophic lateral sclerosis (ALS) has remained an enigma since its earliest description as a relentlessly progressive degeneration with prominent neuromuscular manifestations that are associated with upper and lower motor neuron dysfunction. Although this remains the hallmark of ALS, a significant proportion of patients will also demonstrate one or more features of frontotemporal dysfunction, including a frontotemporal dementia (FTD). Understanding whether these 2 seemingly disparate syndromes are simply reflective of the co-occurrence of 2 distinct pathologic processes or the clinical manifestations of a common pathophysiologic derangement involving the brain more widely has gripped contemporary ALS researchers. Supporting a commonality of causation, both ALS and FTD show an alteration in the metabolism of TAR DNA-binding protein 43, marked by a shift in nucleocytoplasmic localization alongside a broad range of neuronal cytoplasmic inclusions consisting of pathologic aggregates of RNA-binding proteins. Similarly, several disease-associated or disease-modifying genetic variants that are shared between the 2 disorders suggest shared underlying mechanisms. In both, a prominent glial response has been postulated to contribute to non-cell-autonomous spread. A more contemporary hypothesis, however, suggests that syndromes of cortical and subcortical dysfunction are driven by impairments in discrete neural networks. This postulates that such networks, including networks subserving motor or cognitive function, possess unique and selective vulnerabilities to either single molecular toxicities or combinations thereof. The co-occurrence of one or more network dysfunctions in ALS and FTD is thus a reflection not of unique neuroanatomic correlates but rather of shared molecular vulnerabilities. The basis of such shared vulnerabilities becomes the fulcrum around which the next advances in our understanding of ALS and its possible therapy will develop.