Chaotic Itinerancy Research Articles

Seeing double with a multifunctional reservoir computer.

Multifunctional biological neural networks exploit multistability in order to perform multiple tasks without changing any network properties. Enabling artificial neural networks (ANNs) to obtain certain multistabilities in order to perform several tasks, where each task is related to a particular attractor in the network's state space, naturally has many benefits from a machine learning perspective. Given the association to multistability, in this paper, we explore how the relationship between different attractors influences the ability of a reservoir computer (RC), which is a dynamical system in the form of an ANN, to achieve multifunctionality. We construct the "seeing double" problem in order to systematically study how a RC reconstructs a coexistence of attractors when there is an overlap between them. As the amount of overlap increases, we discover that for multifunctionality to occur, there is a critical dependence on a suitable choice of the spectral radius for the RC's internal network connections. A bifurcation analysis reveals how multifunctionality emerges and is destroyed as the RC enters a chaotic regime that can lead to chaotic itinerancy.

Controlling chaotic itinerancy in laser dynamics for reinforcement learning.

Photonic artificial intelligence has attracted considerable interest in accelerating machine learning; however, the unique optical properties have not been fully used for achieving higher-order functionalities. Chaotic itinerancy, with its spontaneous transient dynamics among multiple quasi-attractors, can be used to realize brain-like functionalities. In this study, we numerically and experimentally investigate a method for controlling the chaotic itinerancy in a multimode semiconductor laser to solve a machine learning task, namely, the multiarmed bandit problem, which is fundamental to reinforcement learning. The proposed method uses chaotic itinerant motion in mode competition dynamics controlled via optical injection. We found that the exploration mechanism is completely different from a conventional searching algorithm and is highly scalable, outperforming the conventional approaches for large-scale bandit problems. This study paves the way to use chaotic itinerancy for effectively solving complex machine learning tasks as photonic hardware accelerators.

Nonlinear dynamics of a self-mixing thin-slice solid-state laser subjected to Doppler-shifted optical feedback.

Chaotic oscillations of a linearly polarized single longitudinal-mode thin-slice Nd:GdVO_{4} laser placed in a self-mixing laser Doppler velocity scheme were dynamically characterized in terms of the intensity probability distribution, joint time-frequency analysis, and short-term Fourier transformation of temporal evolutions, and the degree of disorder in the amplitude and phase of the long-term temporal evolutions. The transition from chaotic relaxation oscillations (ROs) to chaotic spiking oscillations (SOs) was explored via the chaotic itinerancy (CI) regime by increasing the feedback ratio toward the laser from a rotating scattering object. The intensity probability distribution was found to change from an exponential decay in the RO regime to an inverse power law in the SO regime, which manifests itself in self-organized critical behavior, while stochastic subharmonic frequency locking among the two periodicities of RO and SO takes place in the CI regime featuring quantum-noise (spontaneous-emission)-induced order in the amplitude and phase of the spiking oscillations. All of the experimental results were reproduced by numerical simulations of a model equation of a single-mode self-mixing solid-state laser subjected to Doppler-shifted optical feedback from a rotating scattering object.

Designing spontaneous behavioral switching via chaotic itinerancy.

Chaotic itinerancy is a frequently observed phenomenon in high-dimensional nonlinear dynamical systems and is characterized by itinerant transitions among multiple quasi-attractors. Several studies have pointed out that high-dimensional activity in animal brains can be observed to exhibit chaotic itinerancy, which is considered to play a critical role in the spontaneous behavior generation of animals. Thus, how to design desired chaotic itinerancy is a topic of great interest, particularly for neurorobotics researchers who wish to understand and implement autonomous behavioral controls. However, it is generally difficult to gain control over high-dimensional nonlinear dynamical systems. In this study, we propose a method for implementing chaotic itinerancy reproducibly in a high-dimensional chaotic neural network. We demonstrate that our method enables us to easily design both the trajectories of quasi-attractors and the transition rules among them simply by adjusting the limited number of system parameters and by using the intrinsic high-dimensional chaos.

A LSTM based prediction model for nonlinear dynamical systems with chaotic itinerancy

The prediction for chaotic trajectory from the measured data of time history, without prior knowledge of underlying dynamical model, is a challenging task in the data-driven analysis, due to its sensitivity to initial conditions. In this paper, the Long Short-Term Memory Network (LSTM) with the merge layer is proposed to predict the future states of the coupled Morris-Lecar (M-L) system with the chaotic itinerancy responses. Here, the two LSTM models with single-branch and multi-branch are constructed respectively to carry out the predictions in the multivariate loading conditions. By comparison to the network model with single-branch, the multi-branch model with adding merge layer can provide a high utilization of weights to reduce training cost greatly and receive a low prediction error, which make the multi-layer LSTM promising to estimate a high-dimensional complex dynamical behavior like transient chaotic itinerancy.

Chaos with a high-dimensional torus

Transition from quasiperiodicity with many frequencies (i.e., a high-dimensional torus) to chaos is studied by using $N$-dimensional globally coupled circle maps. First, the existence of $N$-dimensional tori with $N\geq 2$ is confirmed while they become exponentially rare with $N$. Besides, chaos exists even when the map is invertible, and such chaos has more null Lyapunov exponents as $N$ increases. This unusual form of "chaos on a torus," termed toric chaos, exhibits delocalization and slow dynamics of the first Lyapunov vector. Fractalization of tori at the transition to chaos is also suggested. The relevance of toric chaos to neural dynamics and turbulence is discussed in relation to chaotic itinerancy.

Frustration induced transient chaos, fractal and riddled basins in coupled limit cycle oscillators

This report unravels frustration as a source of transient chaotic dynamics even in a simple array of coupled limit cycle oscillators. The transient chaotic dynamics along with the multistable nature of frustrated systems facilitates the existence of complex basin structures such as fractal and riddled basins. In particular, we report the emergence of transient chaotic dynamics around the chaotic itinerancy region, where the basin of attraction near the chaotic region is riddled and it becomes fractal basin when we move away from this region. Normally the complex basin structures are observed only for oscillatory states but surprisingly it is also observed even for oscillation death states. Interestingly super persistent transient chaos is also shown to emerge in the coupled limit cycle oscillators, which manifests as a stable chaos for larger networks.

Chaotic Itinerancy Observed in Mutually Coupled Gaussian Maps

A one-dimensional Gaussian map defined by a Gaussian function describes a discrete-time dynamical system. Chaotic behavior can be observed in both Gaussian and logistic maps. This study analyzes the bifurcation structure corresponding to the fixed and periodic points of a coupled system comprising two Gaussian maps. The bifurcation structure of a mutually coupled Gaussian map is more complex than that of a mutually coupled logistic map. In a coupled Gaussian map, it was confirmed that after a stable fixed point or stable periodic points became unstable through the bifurcation, the points were able to recover their stability while the system parameters were changing. Moreover, we investigated a parameter region in which symmetric and asymmetric stable fixed points coexisted. Asymmetric unstable fixed point was generated by the [Formula: see text]-type branching of a symmetric stable fixed point. The stability of the unstable fixed point could be recovered through period-doubling and tangent bifurcations. Furthermore, a homoclinic structure related to the occurrence of chaotic behavior and invariant closed curves caused by two-periodic points was observed. The mutually coupled Gaussian map was merely a two-dimensional dynamical system; however, chaotic itinerancy, known to be a characteristic property associated with high-dimensional dynamical systems, was observed. The bifurcation structure of the mutually coupled Gaussian map clearly elucidates the mechanism of chaotic itinerancy generation in the two-dimensional coupled map. We discussed this mechanism by comparing the bifurcation structures of the Gaussian and logistic maps.

Chaotic Itinerancy in Random Dynamical System Related to Associative Memory Models

We consider a random dynamical system arising as a model of the behavior of a macrovariable related to a more complicated model of associative memory. This system can be seen as a small (stochastic and deterministic) perturbation of a determinstic system having two weak attractors which are destroyed after the perturbation. We show, with a computer aided proof, that the system has a kind of chaotic itineracy. Typical orbits are globally chaotic, while they spend a relatively long time visiting the attractor’s ruins.

Intermittent transition between synchronization and desynchronization in multi-regional business cycles

Empirical studies often conclude that multi-regional business cycles exhibit intermittent transition between synchronization and desynchronization of each regional fluctuations. In this study, we robustly observe this behavior (called chaotic itinerancy) in a model of multi-regional business cycles, in which all regions of a national economy are homogeneous and connected through producers’ behavior based on the average level announced by the government. Although a producer very slowly adjusts his/her output towards the average level, regional business cycles begin to synchronize because of the entrainment effect. Moreover, when a producer emphasizes the profit maximization more and when puts more weight on the average level in his/her decision-making, the economy is more likely to exhibit such intermittent transition. Further, it is clarified that behind intermittent transition exist cycles among periodic orbits with different number of unstable directions.

Chaotic itinerancy within the coupled dynamics between a physical body and neural oscillator networks.

Chaotic itinerancy is a phenomenon in which the state of a nonlinear dynamical system spontaneously explores and attracts certain states in a state space. From this perspective, the diverse behavior of animals and its spontaneous transitions lead to a complex coupled dynamical system, including a physical body and a brain. Herein, a series of simulations using different types of non-linear oscillator networks (i.e., regular, small-world, scale-free, random) with a musculoskeletal model (i.e., a snake-like robot) as a physical body are conducted to understand how the chaotic itinerancy of bodily behavior emerges from the coupled dynamics between the body and the brain. A behavior analysis (behavior clustering) and network analysis for the classified behavior are then applied. The former consists of feature vector extraction from the motions and classification of the movement patterns that emerged from the coupled dynamics. The network structures behind the classified movement patterns are revealed by estimating the “information networks” different from the given non-linear oscillator networks based on the transfer entropy which finds the information flow among neurons. The experimental results show that: (1) the number of movement patterns and their duration depend on the sensor ratio to control the balance of strength between the body and the brain dynamics and on the type of the given non-linear oscillator networks; and (2) two kinds of information networks are found behind two kinds movement patterns with different durations by utilizing the complex network measures, clustering coefficient and the shortest path length with a negative and a positive relationship with the duration periods of movement patterns. The current results seem promising for a future extension of the method to a more complicated body and environment. Several requirements are also discussed.

Non-local correlations via chaotic itinerancy in VCSEL with optical feedback.

Similar to edge-emitting lasers, vertical cavity surface emitting lasers (VCSELs) subjected to optical feedback are known for exhibiting erratic fluctuations of their optical power at slow and fast time scales; these are called low-frequency fluctuations (LFF). Here, we demonstrate both experimentally and numerically that the chaotic itinerancy in phase space associated with LFF has a deep connection with the creation of non-local correlations at multiple time scales between the two linear polarization modes. Our result provides a novel framework to interpret the unknown origin of spectral signatures in the optical power of chaotic lasers with optical feedback, which were observed in the past two decades.

High-frequency chaotic dynamics enabled by optical phase-conjugation.

Wideband chaos is of interest for applications such as random number generation or encrypted communications, which typically use optical feedback in a semiconductor laser. Here, we show that replacing conventional optical feedback with phase-conjugate feedback improves the chaos bandwidth. In the range of achievable phase-conjugate mirror reflectivities, the bandwidth increase reaches 27% when compared with feedback from a conventional mirror. Experimental measurements of the time-resolved frequency dynamics on nanosecond time-scales show that the bandwidth enhancement is related to the onset of self-pulsing solutions at harmonics of the external-cavity frequency. In the observed regime, the system follows a chaotic itinerancy among these destabilized high-frequency external-cavity modes. The recorded features are unique to phase-conjugate feedback and distinguish it from the long-standing problem of time-delayed feedback dynamics.

From globally coupled maps to complex-systems biology.

Studies of globally coupled maps, introduced as a network of chaotic dynamics, are briefly reviewed with an emphasis on novel concepts therein, which are universal in high-dimensional dynamical systems. They include clustering of synchronized oscillations, hierarchical clustering, chimera of synchronization and desynchronization, partition complexity, prevalence of Milnor attractors, chaotic itinerancy, and collective chaos. The degrees of freedom necessary for high dimensionality are proposed to equal the number in which the combinatorial exceeds the exponential. Future analysis of high-dimensional dynamical systems with regard to complex-systems biology is briefly discussed.

Synthetic Modeling of Autonomous Learning with a Chaotic Neural Network

We investigate the possible role of intermittent chaotic dynamics called chaotic itinerancy, in interaction with nonsupervised learnings that reinforce and weaken the neural connection depending on the dynamics itself. We first performed hierarchical stability analysis of the Chaotic Neural Network model (CNN) according to the structure of invariant subspaces. Irregular transition between two attractor ruins with positive maximum Lyapunov exponent was triggered by the blowout bifurcation of the attractor spaces, and was associated with riddled basins structure. We secondly modeled two autonomous learnings, Hebbian learning and spike-timing-dependent plasticity (STDP) rule, and simulated the effect on the chaotic itinerancy state of CNN. Hebbian learning increased the residence time on attractor ruins, and produced novel attractors in the minimum higher-dimensional subspace. It also augmented the neuronal synchrony and established the uniform modularity in chaotic itinerancy. STDP rule reduced the residence time on attractor ruins, and brought a wide range of periodicity in emerged attractors, possibly including strange attractors. Both learning rules selectively destroyed and preserved the specific invariant subspaces, depending on the neuron synchrony of the subspace where the orbits are situated. Computational rationale of the autonomous learning is discussed in connectionist perspective.

Bifurcation to chaotic low-frequency fluctuations in a laser diode with phase-conjugate feedback.

We unveil theoretically the bifurcations to chaotic low-frequency fluctuations (LFF) in a laser diode with phase-conjugate feedback (PCF). LFF occur from a chaotic itinerancy among destabilized limit-cycle attractors that correspond to the external-cavity modes (ECMs) of the PCF laser system and with a directional motion toward a self-pulsating dynamics of increasing frequency and larger output power. When increasing the feedback strength, the frequency of the fast-pulsing dynamics changes about a multiple of the external-cavity frequency, which is a unique feature of LFF in a laser diode with PCF.

Controlling deterministic output variability in a feature extracting chaotic BAM

In this work, a chaotic feature extracting BAM that is capable of generating various behaviors is introduced. These behaviors arise from different attractors, ranging from a stored fixed point to a wandering chaotic region, including variations of all stored fixed points. Variations of stored patterns are generated by the network via the setting of the variability exhibited by every extracted feature. A control method is applied in order to move the network's trajectory into the desired regions and generate chaotic itinerancy, which is reported as a salient property of the brain system. This control is achieved by adjusting the free parameters of the feature extracting units' activation functions. Moreover, it is shown that the higher the number of units applied as feature extractors, the more local features are obtained, control of which leads to greater output uncertainty. However, the structure of this model is very simple and its complex behavior is a result of the interaction among feature units. These observations imply that the proposed model can be feasibly applied in information processing, such as searching in memory, pattern recognition in the presence of noise and variability, modeling episodic memory and decision making in a changing environment.

Perception and self-organized instability

This paper considers state-dependent dynamics that mediate perception in the brain. In particular, it considers the formal basis of self-organized instabilities that enable perceptual transitions during Bayes-optimal perception. The basic phenomena we consider are perceptual transitions that lead to conscious ignition (Dehaene and Changeux, 2011) and how they depend on dynamical instabilities that underlie chaotic itinerancy (Breakspear, 2001; Tsuda, 2001) and self-organized criticality (Beggs and Plenz, 2003; Plenz and Thiagarajan, 2007; Shew et al., 2011). Our approach is based on a dynamical formulation of perception as approximate Bayesian inference, in terms of variational free energy minimization. This formulation suggests that perception has an inherent tendency to induce dynamical instabilities (critical slowing) that enable the brain to respond sensitively to sensory perturbations. We briefly review the dynamics of perception, in terms of generalized Bayesian filtering and free energy minimization, present a formal conjecture about self-organized instability and then test this conjecture, using neuronal (numerical) simulations of perceptual categorization.

Some Thoughts Connecting Deterministic Chaos, Neuronal Dynamics and Aesthetic Experience

The apparent randomness of deterministic chaos is diferentiated from stochastic randomness and linked to natural processes, time’s irreversibility and the creation of meaning. Current neuroscience research strongly suggests that chaotic dynamics govern the physiological functioning of the brain/mind. The brain/mind is conceived as a multi-attractor system functioning at a far from equilibrium state poised for instantaneous state changes and transitions. Chaotic itinerancy has been suggested as a process by which chaotic transitions among attractors may be made and dynamically integrated in a multi-attractor chaotic system such as the brain. The article outlines a theory suggesting that the general characteristics of aesthetic experience are determined by the chaotic dynamics of the brain/mind and by the dynamics of chaotic itinerancy. Two examples, a novel by W.G. Sebald and the installation art of Jenny Holzer are described in terms of this new aesthetic theory.

Hypotheses on the functional roles of chaotic transitory dynamics

In contrast to the conventional static view of the brain, recent experimental data show that an alternative view is necessary for an appropriate interpretation of its function. Some selected problems concerning the cortical transitory dynamics are discussed. For the first time, we propose five scenarios for the appearance of chaotic itinerancy, which provides typical transitory dynamics. Second, we describe the transitory behaviors that have been observed in human and animal brains. Finally, we propose nine hypotheses on the functional roles of such dynamics, focusing on the dynamics embedded in data and the dynamical interpretation of brain activity within the framework of cerebral hermeneutics.