2024-10-29 | | Total: 2
Reservoir computing (RC) harnesses the intrinsic dynamics of a chaotic system, called the reservoir, to perform various time-varying functions. An important use-case of RC is the generation of target temporal sequences via a trainable output-to-reservoir feedback loop. Despite the promise of RC in various domains, we lack a theory of performance bounds on RC systems. Here, we formulate an existence condition for a feedback loop that produces the target sequence. We next demonstrate that, given a sufficiently chaotic neural network reservoir, two separate factors are needed for successful training: global network stability of the target orbit, and the ability of the training algorithm to drive the system close enough to the target, which we term `reach'. By computing the training phase diagram over a range of target output amplitudes and periods, we verify that reach-limited failures depend on the training algorithm while stability-limited failures are invariant across different algorithms. Finally, we leverage dynamical mean field theory (DMFT) to provide an analytical amplitude-period bound on achievable outputs by RC networks and propose a way of enhancing algorithm reach via forgetting. The resulting mechanistic understanding of RC performance can guide the future design and deployment of reservoir networks.
In applications, an anticipated situation is where the system of interest has never been encountered before and sparse observations can be made only once. Can the dynamics be faithfully reconstructed from the limited observations without any training data? This problem defies any known traditional methods of nonlinear time-series analysis as well as existing machine-learning methods that typically require extensive data from the target system for training. We address this challenge by developing a hybrid transformer and reservoir-computing machine-learning scheme. The key idea is that, for a complex and nonlinear target system, the training of the transformer can be conducted not using any data from the target system, but with essentially unlimited synthetic data from known chaotic systems. The trained transformer is then tested with the sparse data from the target system. The output of the transformer is further fed into a reservoir computer for predicting the long-term dynamics or the attractor of the target system. The power of the proposed hybrid machine-learning framework is demonstrated using a large number of prototypical nonlinear dynamical systems, with high reconstruction accuracy even when the available data is only 20% of that required to faithfully represent the dynamical behavior of the underlying system. The framework provides a paradigm of reconstructing complex and nonlinear dynamics in the extreme situation where training data does not exist and the observations are random and sparse.