Time-delay Reservoir Research Articles

Hardware optimization for photonic time-delay reservoir computer dynamics

Reservoir computing (RC) is one kind of neuromorphic computing mainly applied to process sequential data such as time-dependent signals. In this paper, the bifurcation diagram of a photonic time-delay RC system is thoroughly studied, and a method of bifurcation dynamics guided hardware hyperparameter optimization is presented. The time-evolution equation expressed by the photonic hardware parameters is established while the intrinsic dynamics of the photonic RC system is quantitively studied. Bifurcation dynamics based hyperparameter optimization offers a simple yet effective approach in hardware setting optimization that aims to reduce the complexity and time in hardware adjustment. Three benchmark tasks, nonlinear channel equalization (NCE), nonlinear auto regressive moving average with 10th order time lag (NARMA10) and Santa Fe laser time-series prediction tasks are implemented on the photonic delay-line RC using bifurcation dynamics guided hardware optimization. The experimental results of these benchmark tasks achieved overall good agreement with the simulated bifurcation dynamics modeling results.

Asynchronous photonic time-delay reservoir computing.

Time-delay reservoir computing uses a nonlinear node associated with a feedback loop to construct a large number of virtual neurons in the neural network. The clock cycle of the computing network is usually synchronous with the delay time of the feedback loop, which substantially constrains the flexibility of hardware implementations. This work shows an asynchronous reservoir computing network based on a semiconductor laser with an optical feedback loop, where the clock cycle (20 ns) is considerably different to the delay time (77 ns). The performance of this asynchronous network is experimentally investigated under various operation conditions. It is proved that the asynchronous reservoir computing shows highly competitive performance on the prediction task of Santa Fe chaotic time series, in comparison with the synchronous counterparts.

Impact of filtering on photonic time-delay reservoir computing.

We analyze the modification of the computational properties of a time-delay photonic reservoir computer with a change in its feedback bandwidth. For a reservoir computing configuration based on a semiconductor laser subject to filtered optoelectronic feedback, we demonstrate that bandwidth selection can lead to a flat-topped eigenvalue spectrum for which a large number of system frequencies are weakly damped as a result of the attenuation of modulational instability by feedback filtering. This spectral configuration allows for the optimization of the reservoir in terms of its memory capacity, while its computational ability appears to be only weakly affected by the characteristics of the filter.

Multi-task photonic time-delay reservoir computing based on polarization modulation.

We propose and experimentally demonstrate a multi-task photonic time-delay reservoir computing (RC) system based on polarization modulation. The key component in the system is a polarization modulator (PolM) that functions, jointly with a polarization controller (PC) and a polarizer, as an equivalent Mach-Zehnder modulator (MZM) to perform electrical to optical conversion and to provide nonlinear operation. By adjusting the bias of the equivalent MZM, the nonlinear function can be optimized for different tasks to achieve the best multi-task performance. In this paper, the task-independent information processing capacity (IPC) of the time-delay RC system is evaluated. The results show that the readout bias of the equivalent MZM leads to a different IPC which can be optimized for different tasks. Two benchmark tasks (NARMA10 and IPIX radar signal prediction) are performed experimentally. The readout bias is adjusted independently for each of the two tasks to give a minimum normalized mean square error (NMSE), which are 0.2103 and 0.0031 for the NARMA10 and IPIX radar signal prediction tasks at a speed of 1.06 Mb/s, respectively.

Adaptive Memory-Enhanced Time Delay Reservoir and its Memristive Implementation

Time Delay Reservoir (TDR) is a hardware-friendly machine learning approach from two perspectives. First, it can prevent the connection overhead of neural networks with increasing neurons. Second, through its dynamic system representation, TDR can also be implemented in hardware by different systems. However, it performs poorly on tasks that involve long-term dependency. In this work, we first introduce a higher-order delay unit, which is capable of accumulating and transferring the long history states in an adaptive manner to further enhance the reservoir memory. Particle Swarm Optimisation is applied to optimize the enhanced degree of memory adaptivity. Our experiments demonstrate its superiority both for short- and long-term memory datasets over seven existing approaches. In light of the hardware-friendly feature of TDR, we further propose a memristive implementation of our adaptive memory-enhanced TDR, where a dynamic memristor and the memristor-based delay element are applied to construct the reservoir. Through circuit simulation, the feasibility of our proposed memristive implementation is verified. The comparisons with different hardware reservoirs show that our proposed memristive implementation is effective both for short- and long-term memory datasets, while exhibiting benefits in terms of smaller circuit area and lower power consumption compared with traditional hardware reservoirs.

Enhanced prediction performance of a time-delay reservoir computing system based on a VCSEL by dual-training method.

In this paper, a new dual-training method for a time-delay reservoir computing (RC) system based on a single vertical-cavity surface-emitting laser (VCSEL) is proposed and demonstrated experimentally for the first time. The prediction performance of the RC system by using the dual-training method has been experimentally and numerically investigated. Here, the dual-training method is defined as performing a further RC based on the difference between the target value and the predicted value of the traditional single training. It is found that enhanced prediction performance of the RC system can be obtained by employing the dual-training method, compared to the traditional single training method. More specifically, the NMSE values of the RC system with the dual-training method applied can be improved to 760% compared with the single training method in experiments. Besides, the effects of injection power, bias currents, feedback strength, and frequency detuning are also considered. The proposed dual-training method is of great significance to the performance enhancement of the RC and has an important promotion effect on the application of the RC in the future.

Chain-structure time-delay reservoir computing for synchronizing chaotic signal and an application to secure communication

In this work, a chain-structure time-delay reservoir (CSTDR) computing, as a new kind of machine learning-based recurrent neural network, is proposed for synchronizing chaotic signals. Compared with the single time-delay reservoir, our proposed CSTDR computing shows excellent performance in synchronizing chaotic signal achieving an order of magnitude higher accuracy. Noise consideration and optimal parameter setting of the model are discussed. Taking the CSTDR computing as the core, a novel scheme of secure communication is further designed, in which the “smart” receiver is different from the traditional in that it can synchronize to the chaotic signal used for encryption in an adaptive manner. The scheme can solve the issues such as design constrains for identical dynamical systems and couplings between transmitter and receiver in conventional settings. To further manifest the practical significance of the scheme, the digital implementation using field-programmable gate array is conducted and tested experimentally with real-world examples including image and video transmission. The work sheds light on developing machine learning-based signal processing and communication applications.

Parallel Time-Delay Reservoir Computing With Quantum Dot Lasers

A semiconductor laser with optical feedback and optical injection is an appealing scheme to construct the time-delay reservoir computing (TDRC) networks. Quantum dot (QD) lasers are compatible to the silicon platform, and hence is helpful to develop fully on-chip TDRCs. This work theoretically demonstrates a parallel TDRC based on a Fabry-Perot QD laser with multiple longitudinal modes. These modes act as connected physical neurons, which process the input signal in parallel. The interaction strength of the modes is characterized by the cross-gain saturation effect. We show that the neuron interaction strength affects the performance of various benchmark tasks, including the memory capacity, time series prediction, nonlinear channel equalization, and spoken digit recognition. In comparison with the one-channel TDRC with the same number of nodes, the parallel TDRC runs faster and its performance is improved on multiple benchmark tasks.

Microring resonators with external optical feedback for time delay reservoir computing.

Microring resonators (MRRs) are a key photonic component in integrated devices, due to their small size, low insertion losses, and passive operation. While the MRRs have been established for optical filtering in wavelength-multiplexed systems, the nonlinear properties that they can exhibit give rise to new perspectives on their use. For instance, they have been recently considered for introducing optical nonlinearity in photonic reservoir computing systems. In this work, we present a detailed numerical investigation of a silicon MRR operation, in the presence of external optical feedback, in a time delay reservoir computing scheme. We demonstrate the versatility of this compact, passive device, by exploiting different operating regimes and solving computing tasks with diverse memory requirements. We show that when large memory is required, as it occurs in the Narma 10 task, the MRR nonlinearity does not play a significant role when the photodetection nonlinearity is involved, while the contribution of the external feedback is significant. On the contrary, for computing tasks such as the Mackey-Glass and the Santa Fe chaotic timeseries prediction, the MRR and the photodetection nonlinearities contribute both to efficient computation. The presence of optical feedback improves the prediction of the Mackey-Glass timeseries while it plays a minor role in the Santa Fe timeseries case.

Fast physical repetitive patterns generation for masking in time-delay reservoir computing

Albeit the conceptual simplicity of hardware reservoir computing, the various implementation schemes that have been proposed so far still face versatile challenges. The conceptually simplest implementation uses a time delay approach, where one replaces the ensemble of nonlinear nodes with a unique nonlinear node connected to a delayed feedback loop. This simplification comes at a price in other parts of the implementation; repetitive temporal masking sequences are required to map the input information onto the diverse states of the time delay reservoir. These sequences are commonly introduced by arbitrary waveform generators which is an expensive approach when exploring ultra-fast processing speeds. Here we propose the physical generation of clock-free, sub-nanosecond repetitive patterns, with increased intra-pattern diversity and their use as masking sequences. To that end, we investigate numerically a semiconductor laser with a short optical feedback cavity, a well-studied dynamical system that provides a wide diversity of emitted signals. We focus on those operating conditions that lead to a periodic signal generation, with multiple harmonic frequency tones and sub-nanosecond limit cycle dynamics. By tuning the strength of the different frequency tones in the microwave domain, we access a variety of repetitive patterns and sample them in order to obtain the desired masking sequences. Eventually, we apply them in a time delay reservoir computing approach and test them in a nonlinear time-series prediction task. In a performance comparison with masking sequences that originate from random values, we find that only minor compromises are made while significantly reducing the instrumentation requirements of the time delay reservoir computing system.

Optimizing input mask for maximum memory performance of time-delay reservoir subjected to state noise

Optimizing input mask for maximum memory performance of time-delay reservoir subjected to state noise

Deep time-delay reservoir computing: Dynamics and memory capacity.

The deep time-delay reservoir computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems' distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to an adjustable distribution between the linear and nonlinear MC. Furthermore, numerical simulations show resonances between the clock cycle and delays of the layers in all degrees of MC. Contrary to MC losses in single-layer reservoirs, these resonances can boost separate degrees of MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.

Accelerating photonic computing by bandwidth enhancement of a time-delay reservoir

AbstractSemiconductor lasers (SLs) that are subject to delayed optical feedback and external optical injection have been demonstrated to perform information processing using the photonic reservoir computing paradigm. Optical injection or optical feedback can under some conditions induce bandwidth-enhanced operation, expanding their modulation response up to several tens of GHz. However, these conditions may not always result in the best performance for computational tasks, since the dynamical and nonlinear properties of the reservoir might change as well. Here we show that by using strong optical injection we can obtain an increased frequency response and a significant acceleration in the information processing capability of this nonlinear system, without loss of performance. Specifically, we demonstrate numerically that the sampling time of the photonic reservoir can be as small as 12 ps while preserving the same computational performance when compared to a much slower sampling rate. We also show that strong optical injection expands the reservoir’s operating conditions for which we obtain improved task performance. The latter is validated experimentally for larger sampling times of 100 ps. The above attributes are demonstrated in a coherent optical communication decoding task.

Performance boost of time-delay reservoir computing by non-resonant clock cycle

The time-delay-based reservoir computing setup has seen tremendous success in both experiment and simulation. It allows for the construction of large neuromorphic computing systems with only few components. However, until now the interplay of the different timescales has not been investigated thoroughly. In this manuscript, we investigate the effects of a mismatch between the time-delay and the clock cycle for a general model. Typically, these two time scales are considered to be equal. Here we show that the case of equal or resonant time-delay and clock cycle could be actively detrimental and leads to an increase of the approximation error of the reservoir. In particular, we can show that non-resonant ratios of these time scales have maximal memory capacities. We achieve this by translating the periodically driven delay-dynamical system into an equivalent network. Networks that originate from a system with resonant delay-times and clock cycles fail to utilize all of their degrees of freedom, which causes the degradation of their performance.

Experimental reservoir computing using VCSEL polarization dynamics.

We realize an experimental setup of a time-delay reservoir using a VCSEL with optical feedback and optical injection. The VCSEL is operated in the injection-locking regime. This allows us to solve different information processing tasks, such as chaotic time-series prediction with a NMSE of 1.6×10-2 and nonlinear channel equalization with a SER of 1.5×10-2, improving state-of-the-art performance. We also demonstrate experimentally, through a careful statistical analysis, the impact of the VCSEL polarization dynamics on the performance of our architecture. More specifically, we confirm recent theoretical prediction stating that a polarization rotated feedback allows for the enhancement of the calculation performance compared to an isotropic feedback.

An FPGA Implementation of a Time Delay Reservoir Using Stochastic Logic

This article presents and demonstrates a stochastic logic time delay reservoir design in FPGA hardware. The reservoir network approach is analyzed using a number of metrics, such as kernel quality, generalization rank, and performance on simple benchmarks and is also compared to a deterministic design. A novel re-seeding method is introduced to reduce the adverse effects of stochastic noise, which may also be implemented in other stochastic logic reservoir computing designs, such as echo state networks. Benchmark results indicate that the proposed design performs well on noise-tolerant classification problems, but more work needs to be done to improve the stochastic logic time delay reservoir's robustness for regression problems. In addition, we show that the stochastic design can significantly reduce area cost if the conversion between binary and stochastic representations is implemented efficiently.

Nonlinear Memory Capacity of Parallel Time-Delay Reservoir Computers in the Processing of Multidimensional Signals.

This letter addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir architecture and its performance in the execution of a specific task. Second, the inference properties of the ridge regression estimator in the multivariate context are used to assess the impact of finite sample training on the decrease of the reservoir capacity. Finally, an empirical study is conducted that shows the adequacy of the theoretical results with the empirical performances exhibited by various reservoir architectures in the execution of several nonlinear tasks with multidimensional inputs. Our results confirm the robustness properties of the parallel reservoir architecture with respect to task misspecification and parameter choice already documented in the literature.

Stochastic nonlinear time series forecasting using time-delay reservoir computers: Performance and universality

Reservoir computing is a recently introduced machine learning paradigm that has already shown excellent performances in the processing of empirical data. We study a particular kind of reservoir computers called time-delay reservoirs that are constructed out of the sampling of the solution of a time-delay differential equation and show their good performance in the forecasting of the conditional covariances associated to multivariate discrete-time nonlinear stochastic processes of VEC-GARCH type as well as in the prediction of factual daily market realized volatilities computed with intraday quotes, using as training input daily log-return series of moderate size. We tackle some problems associated to the lack of task-universality for individually operating reservoirs and propose a solution based on the use of parallel arrays of time-delay reservoirs.

Stochastic Nonlinear Time Series Forecasting Using Time-Delay Reservoir Computers: Performance and Universality

Reservoir computing is a recently introduced machine learning paradigm that has already shown excellent performances in the processing of empirical data. We study a particular kind of reservoir computers called time-delay reservoirs that are constructed out of the sampling of the solution of a time-delay differential equation and show their good performance in the forecasting of the conditional covariances associated to multivariate discrete-time nonlinear stochastic processes of VEC-GARCH type as well as in the prediction of factual daily market realized volatilities computed with intraday quotes, using as training input daily log-return series of moderate size. We tackle some problems associated to the lack of task-universality for individually operating reservoirs and propose a solution based on the use of parallel arrays of time-delay reservoirs.