Abstract
Traditional machine-learning methods are inefficient in capturing chaos in nonlinear dynamical systems, especially when the time difference between consecutive steps is so large that the extracted time series looks apparently random. Here, we introduce a new long-short-term-memory (LSTM)-based recurrent architecture by tensorizing the cell-state-to-state propagation therein, maintaining the long-term memory feature of LSTM, while simultaneously enhancing the learning of short-term nonlinear complexity. We stress that the global minima of training can be most efficiently reached by our tensor structure where all nonlinear terms, up to some polynomial order, are treated explicitly and weighted equally. The efficiency and generality of our architecture are systematically investigated and tested through theoretical analysis and experimental examinations. In our design, we have explicitly used two different many-body entanglement structures—matrix product states (MPS) and the multiscale entanglement renormalization ansatz (MERA)—as physics-inspired tensor decomposition techniques, from which we find that MERA generally performs better than MPS, hence conjecturing that the learnability of chaos is determined not only by the number of free parameters but also the tensor complexity—recognized as how entanglement entropy scales with varying matricization of the tensor.
Highlights
IntroductionTime series forecasting [1], despite its undoubtedly tremendous potential in both theoretical issues (e.g., mechanical analysis, ergodicity) and real-world applications [2]
Time series forecasting [1], despite its undoubtedly tremendous potential in both theoretical issues and real-world applications [2](e.g., traffic, weather, and clinical records analysis), has long been known as an intricate field
multiscale entanglement renormalization ansatz (MERA) differs from matrix product states (MPS) in its hierarchical tree structure: within each level {I, II, · · · }, the structure contains a layer of 4-tensor disentanglers of dimensions { DI4, DII4, · · · }, and a layer of 3-tensor isometries of dimensions { DI2 × DII, DII2 × DIII, · · · }, of which details can be found in [36]
Summary
Time series forecasting [1], despite its undoubtedly tremendous potential in both theoretical issues (e.g., mechanical analysis, ergodicity) and real-world applications [2]. We introduce a new LSTM-based architecture by tensorizing the cellstate-to-state propagation therein, retaining the long-term memory features of LSTM while simultaneously enhancing the learning of short-term nonlinear complexity. Compared with traditional LSTM architectures, including stacked LSTM [33] and other aforementioned statistics/ML-based forecasting methods, our model is shown to be a general and outperforming approach for capturing chaos in almost every typical chaotic continuoustime dynamical system and discrete-time map with controlled comparable NN training conditions, justified by both our theoretical analysis and experimental results. Our experiments show that, regarding our entanglement-structured design of the new tensorized LSTM architecture, LSTM-MERA performs even better than LSTM-MPS in general without increasing the number of parameters This finding leads to another interesting result. Entropy 2021, 23, 1491 that should tensorization be introduced, but the tensor’s EE has to scale with the system size as well; MERA is more efficient than MPS in learning chaos
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
