arXiv subjects

Juan-Pablo Ortega

Publications and source records attributed to Juan-Pablo Ortega.

At least 55 records · Page 3Linked to original sources

arXiv2020

Memory and forecasting capacities of nonlinear recurrent networks

The notion of memory capacity, originally introduced for echo state and linear networks with independent inputs, is generalized to nonlinear recurrent networks with stationary but dependent inputs. The presence of dependence in the inputs makes natural the introduction of the network forecasting capacity, that measures the possibility of forecasting time series values using network states. Generic bounds for memory and forecasting capacities are formulated in terms of the number of neurons of the nonlinear recurrent network and the autocovariance function or the spectral density of the input. These bounds generalize well-known estimates in the literature to a dependent inputs setup. Finally, for the particular case of linear recurrent networks with independent inputs it is proved that the memory capacity is given by the rank of the associated controllability matrix, a fact that has been for a long time assumed to be true without proof by the community.

math.OC↗

arXiv2019

Risk bounds for reservoir computing

We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems when processing discrete-time inputs under various hypotheses on their dependence structure. Non-asymptotic bounds are explicitly written down in terms of the multivariate Rademacher complexities of the reservoir systems and the weak dependence structure of the signals that are being handled. This allows, in particular, to determine the minimal number of observations needed in order to guarantee a prescribed estimation accuracy with high probability for a given reservoir family. At the same time, the asymptotic behavior of the devised bounds guarantees the consistency of the empirical risk minimization procedure for various hypothesis classes of reservoir functionals.

cs.LG↗

arXiv2019

Differentiable reservoir computing

Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so-called echo state (ESP) and fading memory (FMP) properties. These important features amount, in mathematical terms, to the existence and continuity of global reservoir system solutions. That research is complemented in this paper with the characterization of the differentiability of reservoir filters for very general classes of discrete-time deterministic inputs. This constitutes a novel strong contribution to the long line of research on the ESP and the FMP and, in particular, links to existing research on the input-dependence of the ESP. Differentiability has been shown in the literature to be a key feature in the learning of attractors of chaotic dynamical systems. A Volterra-type series representation for reservoir filters with semi-infinite discrete-time inputs is constructed in the analytic case using Taylor's theorem and corresponding approximation bounds are provided. Finally, it is shown as a corollary of these results that any fading memory filter can be uniformly approximated by a finite Volterra series with finite memory.

cs.NE↗

arXiv2018

Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems

A new class of non-homogeneous state-affine systems is introduced for use in reservoir computing. Sufficient conditions are identified that guarantee first, that the associated reservoir computers with linear readouts are causal, time-invariant, and satisfy the fading memory property and second, that a subset of this class is universal in the category of fading memory filters with stochastic almost surely uniformly bounded inputs. This means that any discrete-time filter that satisfies the fading memory property with random inputs of that type can be uniformly approximated by elements in the non-homogeneous state-affine family.

cs.NE↗

arXiv2018

Echo state networks are universal

This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite-dimensional neural network-type state-space model with a static linear readout map. This approximation is valid for infinite time intervals. The proof of this statement is based on fundamental results, also presented in this work, about the topological nature of the fading memory property and about reservoir computing systems generated by continuous reservoir maps.

cs.NE↗

arXiv2018

Reservoir Computing Universality With Stochastic Inputs

The universal approximation properties with respect to $L ^p $-type criteria of three important families of reservoir computers with stochastic discrete-time semi-infinite inputs is shown. First, it is proved that linear reservoir systems with either polynomial or neural network readout maps are universal. More importantly, it is proved that the same property holds for two families with linear readouts, namely, trigonometric state-affine systems and echo state networks, which are the most widely used reservoir systems in applications. The linearity in the readouts is a key feature in supervised machine learning applications. It guarantees that these systems can be used in high-dimensional situations and in the presence of large datasets. The $L ^p $ criteria used in this paper allow the formulation of universality results that do not necessarily impose almost sure uniform boundedness in the inputs or the fading memory property in the filter that needs to be approximated.

cs.ET↗

arXiv2016

Singular ridge regression with homoscedastic residuals: generalization error with estimated parameters

This paper characterizes the conditional distribution properties of the finite sample ridge regression estimator and uses that result to evaluate total regression and generalization errors that incorporate the inaccuracies committed at the time of parameter estimation. The paper provides explicit formulas for those errors. Unlike other classical references in this setup, our results take place in a fully singular setup that does not assume the existence of a solution for the non-regularized regression problem. In exchange, we invoke a conditional homoscedasticity hypothesis on the regularized regression residuals that is crucial in our developments.

stat.ML↗

arXiv2015

Nonlinear memory capacity of parallel time-delay reservoir computers in the processing of multidimensional signals

This paper addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir parameters and architecture and its performance in the execution of a specific task. Second, the inference properties of the ridge regression estimator in the multivariate context is used to assess the impact of finite sample training on the decrease of the reservoir capacity. Finally, an empirical study is conducted that shows the adequacy of the theoretical results with the empirical performances exhibited by various reservoir architectures in the execution of several nonlinear tasks with multidimensional inputs. Our results confirm the robustness properties of the parallel reservoir architecture with respect to task misspecification and parameter choice that had already been documented in the literature.

cs.NE↗

arXiv2015

Quantitative evaluation of the performance of discrete-time reservoir computers in the forecasting, filtering, and reconstruction of stochastic stationary signals

This paper extends the notion of information processing capacity for non-independent input signals in the context of reservoir computing (RC). The presence of input autocorrelation makes worthwhile the treatment of forecasting and filtering problems for which we explicitly compute this generalized capacity as a function of the reservoir parameter values using a streamlined model. The reservoir model leading to these developments is used to show that, whenever that approximation is valid, this computational paradigm satisfies the so called separation and fading memory properties that are usually associated with good information processing performances. We show that several standard memory, forecasting, and filtering problems that appear in the parametric stochastic time series context can be readily formulated and tackled via RC which, as we show, significantly outperforms standard techniques in some instances.

cs.ET↗

arXiv2014

Optimal nonlinear information processing capacity in delay-based reservoir computers

Reservoir computing is a recently introduced brain-inspired machine learning paradigm capable of excellent performances in the processing of empirical data. We focus in a particular kind of time-delay based reservoir computers that have been physically implemented using optical and electronic systems and have shown unprecedented data processing rates. Reservoir computing is well-known for the ease of the associated training scheme but also for the problematic sensitivity of its performance to architecture parameters. This article addresses the reservoir design problem, which remains the biggest challenge in the applicability of this information processing scheme. More specifically, we use the information available regarding the optimal reservoir working regimes to construct a functional link between the reservoir parameters and its performance. This function is used to explore various properties of the device and to choose the optimal reservoir architecture, thus replacing the tedious and time consuming parameter scannings used so far in the literature.

math.DS↗

arXiv2013

Quadratic hedging schemes for non-Gaussian GARCH models

We propose different schemes for option hedging when asset returns are modeled using a general class of GARCH models. More specifically, we implement local risk minimization and a minimum variance hedge approximation based on an extended Girsanov principle that generalizes Duan's (1995) delta hedge. Since the minimal martingale measure fails to produce a probability measure in this setting, we construct local risk minimization hedging strategies with respect to a pricing kernel. These approaches are investigated in the context of non-Gaussian driven models. Furthermore, we analyze these methods for non-Gaussian GARCH diffusion limit processes and link them to the corresponding discrete time counterparts. A detailed numerical analysis based on S&P 500 European Call options is provided to assess the empirical performance of the proposed schemes. We also test the sensitivity of the hedging strategies with respect to the risk neutral measure used by recomputing some of our results with an exponential affine pricing kernel.

q-fin.PR↗

arXiv2013

Stability of Hamiltonian relative equilibria in symmetric magnetically confined rigid bodies

This work studies the symmetries, the associated momentum map, and relative equilibria of a mechanical system consisting of a small axisymmetric magnetic body-dipole in an also axisymmetric external magnetic field that additionally exhibits a mirror symmetry; we call this system the ``orbitron". We study the nonlinear stability of a branch of equatorial quasiorbital relative equilibria using the energy-momentum method and we provide sufficient conditions for their $\mathbb{T}^2$--stability that complete partial stability relations already existing in the literature. These stability prescriptions are explicitly written down in terms of the some of the field parameters, which can be used in the design of stable solutions. We propose new linear methods to determine instability regions in the context of relative equilibria that we use to conclude the sharpness of some of the nonlinear stability conditions obtained.

math.SG↗

arXiv2012

Finite sample forecasting with estimated temporally aggregated linear processes

We propose a finite sample based predictor for estimated linear one dimensional time series models and compute the associated total forecasting error. The expression for the error that we present takes into account the estimation error. Unlike existing solutions in the literature, our formulas require neither assumptions on the second order stationarity of the sample nor Monte Carlo simulations for their evaluation. This result is used to prove the pertinence of a new hybrid scheme that we put forward for the forecast of linear temporal aggregates. This novel strategy consists of carrying out the parameter estimation based on disaggregated data and the prediction based on the corresponding aggregated model and data. We show that in some instances this scheme has a better performance than the "all-disaggregated" approach presented as optimal in the literature.

math.ST↗

arXiv2011

Hedging of time discrete auto-regressive stochastic volatility options

Numerous empirical proofs indicate the adequacy of the time discrete auto-regressive stochastic volatility models introduced by Taylor in the description of the log-returns of financial assets. The pricing and hedging of contingent products that use these models for their underlying assets is a non-trivial exercise due to the incomplete nature of the corresponding market. In this paper we apply two volatility estimation techniques available in the literature for these models, namely Kalman filtering and the hierarchical-likelihood approach, in order to implement various pricing and dynamical hedging strategies. Our study shows that the local risk minimization scheme developed by Föllmer, Schweizer, and Sondermann is particularly appropriate in this setup, especially for at and in the money options or for low hedging frequencies.

q-fin.PR↗

arXiv2011

Multivariate GARCH estimation via a Bregman-proximal trust-region method

The estimation of multivariate GARCH time series models is a difficult task mainly due to the significant overparameterization exhibited by the problem and usually referred to as the "curse of dimensionality". For example, in the case of the VEC family, the number of parameters involved in the model grows as a polynomial of order four on the dimensionality of the problem. Moreover, these parameters are subjected to convoluted nonlinear constraints necessary to ensure, for instance, the existence of stationary solutions and the positive semidefinite character of the conditional covariance matrices used in the model design. So far, this problem has been addressed in the literature only in low dimensional cases with strong parsimony constraints. In this paper we propose a general formulation of the estimation problem in any dimension and develop a Bregman-proximal trust-region method for its solution. The Bregman-proximal approach allows us to handle the constraints in a very efficient and natural way by staying in the primal space and the Trust-Region mechanism stabilizes and speeds up the scheme. Preliminary computational experiments are presented and confirm the very good performances of the proposed approach.

q-fin.CP↗

arXiv2010

GARCH options via local risk minimization

We apply a quadratic hedging scheme developed by Foellmer, Schweizer, and Sondermann to European contingent products whose underlying asset is modeled using a GARCH process and show that local risk-minimizing strategies with respect to the physical measure do exist, even though an associated minimal martingale measure is only available in the presence of bounded innovations. More importantly, since those local risk-minimizing strategies are in general convoluted and difficult to evaluate, we introduce Girsanov-like risk-neutral measures for the log-prices that yield more tractable and useful results. Regarding this subject, we focus on GARCH time series models with Gaussian innovations and we provide specific sufficient conditions that have to do with the finiteness of the kurtosis, under which those martingale measures are appropriate in the context of quadratic hedging. When this equivalent martingale measure is adapted to the price representation we are able to recover out of it the classical pricing formulas of Duan and Heston-Nandi, as well as hedging schemes that improve the performance of those proposed in the literature.

q-fin.PR↗

arXiv2009

A new approach for scenario generation in Risk management

We provide a new dynamic approach to scenario generation for the purposes of risk management in the banking industry. We connect ideas from conventional techniques -- like historical and Monte Carlo simulation -- and we come up with a hybrid method that shares the advantages of standard procedures but eliminates several of their drawbacks. Instead of considering the static problem of constructing one or ten day ahead distributions for vectors of risk factors, we embed the problem into a dynamic framework, where any time horizon can be consistently simulated. Additionally, we use standard models from mathematical finance for each risk factor, whence bridging the worlds of trading and risk management. Our approach is based on stochastic differential equations (SDEs), like the HJM-equation or the Black-Scholes equation, governing the time evolution of risk factors, on an empirical calibration method to the market for the chosen SDEs, and on an Euler scheme (or high-order schemes) for the numerical evaluation of the respective SDEs. The empirical calibration procedure presented in this paper can be seen as the SDE-counterpart of the so called Filtered Historical Simulation method; the behavior of volatility stems in our case out of the assumptions on the underlying SDEs. Furthermore, we are able to easily incorporate "middle-size" and "large-size" events within our framework always making a precise distinction between the information obtained from the market and the one coming from the necessary a-priori intuition of the risk manager. Results of one concrete implementation are provided.

q-fin.RM↗

arXiv2009

The momentum map in Poisson geometry

Every action on a Poisson manifold by Poisson diffeomorphisms lifts to a Hamiltonian action on its symplectic groupoid which has a canonically defined momentum map. We study various properties of this momentum map as well as its use in reduction.

math.SG↗