arXiv subjects

Lyudmila Grigoryeva

Publications and source records attributed to Lyudmila Grigoryeva.

28 records · Page 2Linked to original sources

arXiv2019

Risk bounds for reservoir computing

We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems when processing discrete-time inputs under various hypotheses on their dependence structure. Non-asymptotic bounds are explicitly written down in terms of the multivariate Rademacher complexities of the reservoir systems and the weak dependence structure of the signals that are being handled. This allows, in particular, to determine the minimal number of observations needed in order to guarantee a prescribed estimation accuracy with high probability for a given reservoir family. At the same time, the asymptotic behavior of the devised bounds guarantees the consistency of the empirical risk minimization procedure for various hypothesis classes of reservoir functionals.

cs.LG↗

arXiv2019

Differentiable reservoir computing

Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so-called echo state (ESP) and fading memory (FMP) properties. These important features amount, in mathematical terms, to the existence and continuity of global reservoir system solutions. That research is complemented in this paper with the characterization of the differentiability of reservoir filters for very general classes of discrete-time deterministic inputs. This constitutes a novel strong contribution to the long line of research on the ESP and the FMP and, in particular, links to existing research on the input-dependence of the ESP. Differentiability has been shown in the literature to be a key feature in the learning of attractors of chaotic dynamical systems. A Volterra-type series representation for reservoir filters with semi-infinite discrete-time inputs is constructed in the analytic case using Taylor's theorem and corresponding approximation bounds are provided. Finally, it is shown as a corollary of these results that any fading memory filter can be uniformly approximated by a finite Volterra series with finite memory.

cs.NE↗

arXiv2018

Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems

A new class of non-homogeneous state-affine systems is introduced for use in reservoir computing. Sufficient conditions are identified that guarantee first, that the associated reservoir computers with linear readouts are causal, time-invariant, and satisfy the fading memory property and second, that a subset of this class is universal in the category of fading memory filters with stochastic almost surely uniformly bounded inputs. This means that any discrete-time filter that satisfies the fading memory property with random inputs of that type can be uniformly approximated by elements in the non-homogeneous state-affine family.

cs.NE↗

arXiv2018

Echo state networks are universal

This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite-dimensional neural network-type state-space model with a static linear readout map. This approximation is valid for infinite time intervals. The proof of this statement is based on fundamental results, also presented in this work, about the topological nature of the fading memory property and about reservoir computing systems generated by continuous reservoir maps.

cs.NE↗

arXiv2016

Singular ridge regression with homoscedastic residuals: generalization error with estimated parameters

This paper characterizes the conditional distribution properties of the finite sample ridge regression estimator and uses that result to evaluate total regression and generalization errors that incorporate the inaccuracies committed at the time of parameter estimation. The paper provides explicit formulas for those errors. Unlike other classical references in this setup, our results take place in a fully singular setup that does not assume the existence of a solution for the non-regularized regression problem. In exchange, we invoke a conditional homoscedasticity hypothesis on the regularized regression residuals that is crucial in our developments.

stat.ML↗

arXiv2015

Nonlinear memory capacity of parallel time-delay reservoir computers in the processing of multidimensional signals

This paper addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir parameters and architecture and its performance in the execution of a specific task. Second, the inference properties of the ridge regression estimator in the multivariate context is used to assess the impact of finite sample training on the decrease of the reservoir capacity. Finally, an empirical study is conducted that shows the adequacy of the theoretical results with the empirical performances exhibited by various reservoir architectures in the execution of several nonlinear tasks with multidimensional inputs. Our results confirm the robustness properties of the parallel reservoir architecture with respect to task misspecification and parameter choice that had already been documented in the literature.

cs.NE↗

arXiv2015

Quantitative evaluation of the performance of discrete-time reservoir computers in the forecasting, filtering, and reconstruction of stochastic stationary signals

This paper extends the notion of information processing capacity for non-independent input signals in the context of reservoir computing (RC). The presence of input autocorrelation makes worthwhile the treatment of forecasting and filtering problems for which we explicitly compute this generalized capacity as a function of the reservoir parameter values using a streamlined model. The reservoir model leading to these developments is used to show that, whenever that approximation is valid, this computational paradigm satisfies the so called separation and fading memory properties that are usually associated with good information processing performances. We show that several standard memory, forecasting, and filtering problems that appear in the parametric stochastic time series context can be readily formulated and tackled via RC which, as we show, significantly outperforms standard techniques in some instances.

cs.ET↗

arXiv2014

Optimal nonlinear information processing capacity in delay-based reservoir computers

Reservoir computing is a recently introduced brain-inspired machine learning paradigm capable of excellent performances in the processing of empirical data. We focus in a particular kind of time-delay based reservoir computers that have been physically implemented using optical and electronic systems and have shown unprecedented data processing rates. Reservoir computing is well-known for the ease of the associated training scheme but also for the problematic sensitivity of its performance to architecture parameters. This article addresses the reservoir design problem, which remains the biggest challenge in the applicability of this information processing scheme. More specifically, we use the information available regarding the optimal reservoir working regimes to construct a functional link between the reservoir parameters and its performance. This function is used to explore various properties of the device and to choose the optimal reservoir architecture, thus replacing the tedious and time consuming parameter scannings used so far in the literature.

math.DS↗

arXiv2013

Stability of Hamiltonian relative equilibria in symmetric magnetically confined rigid bodies

This work studies the symmetries, the associated momentum map, and relative equilibria of a mechanical system consisting of a small axisymmetric magnetic body-dipole in an also axisymmetric external magnetic field that additionally exhibits a mirror symmetry; we call this system the ``orbitron". We study the nonlinear stability of a branch of equatorial quasiorbital relative equilibria using the energy-momentum method and we provide sufficient conditions for their $\mathbb{T}^2$--stability that complete partial stability relations already existing in the literature. These stability prescriptions are explicitly written down in terms of the some of the field parameters, which can be used in the design of stable solutions. We propose new linear methods to determine instability regions in the context of relative equilibria that we use to conclude the sharpness of some of the nonlinear stability conditions obtained.

math.SG↗

arXiv2012

Finite sample forecasting with estimated temporally aggregated linear processes

We propose a finite sample based predictor for estimated linear one dimensional time series models and compute the associated total forecasting error. The expression for the error that we present takes into account the estimation error. Unlike existing solutions in the literature, our formulas require neither assumptions on the second order stationarity of the sample nor Monte Carlo simulations for their evaluation. This result is used to prove the pertinence of a new hybrid scheme that we put forward for the forecast of linear temporal aggregates. This novel strategy consists of carrying out the parameter estimation based on disaggregated data and the prediction based on the corresponding aggregated model and data. We show that in some instances this scheme has a better performance than the "all-disaggregated" approach presented as optimal in the literature.

math.ST↗