arXiv subjects

A. M. G. Cox

Publications and source records attributed to A. M. G. Cox.

7 recordsLinked to original sources

arXiv2021

Using Echo State Networks to Approximate Value Functions for Control

An Echo State Network (ESN) is a type of single-layer recurrent neural network with randomly-chosen internal weights and a trainable output layer. We prove under mild conditions that a sufficiently large Echo State Network can approximate the value function of a broad class of stochastic and deterministic control problems. Such control problems are generally non-Markovian. We describe how the ESN can form the basis for novel and computationally efficient reinforcement learning algorithms in a non-Markovian framework. We demonstrate this theory with two examples. In the first, we use an ESN to solve a deterministic, partially observed, control problem which is a simple game we call `Bee World'. In the second example, we consider a stochastic control problem inspired by a market making problem in mathematical finance. In both cases we can compare the dynamics of the algorithms with analytic solutions to show that even after only a single reinforcement policy iteration the algorithms arrive at a good policy.

math.DS↗

arXiv2021

Switching Identities by Probabilistic Means

Switching identities have a long history in potential theory and stochastic analysis. In recent work of Cox and Wang, a switching identity was used to connect an optimal stopping problem and the Skorokhod embedding problem (SEP). Typically switching identies of this form are derived using deep analytic connections. In this paper, we prove the switching identities using a simple probabilistic argument, which furthermore highlights a previously unexplored symmetry between the Root and Rost solutions to the SEP.

math.PR↗

arXiv2019

Stochastic Methods for Neutron Transport Equation III: Generational many-to-one and $k_\texttt{eff}$

The Neutron Transport Equation (NTE) describes the flux of neutrons over time through an inhomogeneous fissile medium. In the recent articles [5, 10], a probabilistic solution of the NTE is considered in order to demonstrate a Perron-Frobenius type growth of the solution via its projection onto an associated leading eigenfunction. In [9, 4], further analysis is performed to understand the implications of this growth both in the stochastic sense, as well as from the perspective of Monte-Carlo simulation. Such Monte-Carlo simulations are prevalent in industrial applications, in particular where regulatory checks are needed in the process of reactor core design. In that setting, however, it turns out that a different notion of growth takes centre stage, which is otherwise characterised by another eigenvalue problem. In that setting, the eigenvalue, sometimes called k-effective (written $k_\texttt{eff}$), has the physical interpretation as being the ratio of neutrons produced (during fission events) to the number lost (due to absorption in the reactor or leakage at the boundary) per typical fission event. In this article, we aim to supplement [5, 10, 9, 4], by developing the stochastic analysis of the NTE further to the setting where a rigorous probabilistic interpretation of keff is given, both in terms of a Perron-Frobenius type analysis as well as via classical operator analysis. To our knowledge, despite the fact that an extensive engineering literature and industrial Monte-Carlo software is concentrated around the estimation of keff and its associated eigenfunction, we believe that our work is the first rigorous treatment in the probabilistic sense (which underpins some of the aforesaid Monte-Carlo simulations).

math.PR↗

arXiv2015

Embedding laws in diffusions by functions of time

We present a constructive probabilistic proof of the fact that if $B=(B_t)_{t\ge0}$ is standard Brownian motion started at $0$, and $μ$ is a given probability measure on $\mathbb{R}$ such that $μ(\{0\})=0$, then there exists a unique left-continuous increasing function $b:(0,\infty)\rightarrow\mathbb{R}\cup\{+\infty\}$ and a unique left-continuous decreasing function $c:(0,\infty)\rightarrow\mathbb{R}\cup\{-\infty\}$ such that $B$ stopped at $τ_{b,c}=\inf\{t>0\vert B_t\ge b(t)$ or $B_t\le c(t)\}$ has the law $μ$. The method of proof relies upon weak convergence arguments arising from Helly's selection theorem and makes use of the Lévy metric which appears to be novel in the context of embedding theorems. We show that $τ_{b,c}$ is minimal in the sense of Monroe so that the stopped process $B^{τ_{b,c}}=(B_{t\wedgeτ_{b,c}})_{t\ge0}$ satisfies natural uniform integrability conditions expressed in terms of $μ$. We also show that $τ_{b,c}$ has the smallest truncated expectation among all stopping times that embed $μ$ into $B$. The main results extend from standard Brownian motion to all recurrent diffusion processes on the real line.

math.PR↗

arXiv2008

Pathwise inequalities for local time: Applications to Skorokhod embeddings and optimal stopping

We develop a class of pathwise inequalities of the form $H(B_t)\ge M_t+F(L_t)$, where $B_t$ is Brownian motion, $L_t$ its local time at zero and $M_t$ a local martingale. The concrete nature of the representation makes the inequality useful for a variety of applications. In this work, we use the inequalities to derive constructions and optimality results of Vallois' Skorokhod embeddings. We discuss their financial interpretation in the context of robust pricing and hedging of options written on the local time. In the final part of the paper we use the inequalities to solve a class of optimal stopping problems of the form $\sup_τ\mathbb{E}[F(L_τ)-\int _0^τβ(B_s) ds]$. The solution is given via a minimal solution to a system of differential equations and thus resembles the maximality principle described by Peskir. Throughout, the emphasis is placed on the novelty and simplicity of the techniques.

math.PR↗

arXiv2005

A unifying class of Skorokhod embeddings: connecting the Azema-Yor and Vallois embeddings

In this paper we consider the Skorokhod embedding problem in Brownian motion. In particular, we give a solution based on the local time at zero of a variably skewed Brownian motion related to the underlying Brownian motion. Special cases of the construction include the Azema-Yor and Vallois embeddings. In turn, the construction has an interpretation in the Chacon-Walsh framework.

math.PR↗

arXiv2002

An Optimal Skorokhod Embedding for Diffusions

Given a Brownian motion $B_t$ and a general target law $μ$ (not necessarily centered or even integrable) we show how to construct an embedding of $μ$ in $B$. This embedding is an extension of an embedding due to Perkins, and is optimal in the sense that it simultaneously minimises the distribution of the maximum and maximises the distribution of the minimum among all embeddings of $μ$. The embedding is then applied to regular diffusions, and used to characterise the target laws for which a $H^p$-embedding may be found.

math.PR↗