Dynamical time series embeddings in recurrent neural networks

Abstract

Time series forecasting has historically been a key research problem in science and engineering. In recent years, machine learning algorithms have proven to be a very successful data-driven approach in this area. In particular, Recurrent Neural Networks (RNNs) represent the state-of-the-art algorithms in many sequential tasks. In this paper we train Long Short Term Memory networks (LSTM), which are a type of RNNs, to make predictions in time series corresponding to the observation of a single variable of a chaotic system. We show that, under certain conditions, networks learn to generate an embedding of the data in their inner sate that is topologically equivalent to the original strange attractor. Remarkably, this resembles standard forecasting methods from nonlinear science in which the time series is embedded in a multi-valued space using Takens's delay embedding mechanism.