arXiv subjects

Stefan Zohren

Publications and source records attributed to Stefan Zohren.

At least 91 records · Page 5Linked to original sources

arXiv2020

Investment sizing with deep learning prediction uncertainties for high-frequency Eurodollar futures trading

In this work we show that prediction uncertainty estimates gleaned from deep learning models can be useful inputs for influencing the relative allocation of risk capital across trades. In this way, consideration of uncertainty is important because it permits the scaling of investment size across trade opportunities in a principled and data-driven way. We showcase this insight with a prediction model and find clear outperformance based on a Sharpe ratio metric, relative to trading strategies that either do not take uncertainty into account, or that utilize an alternative market-based statistic as a proxy for uncertainty. Of added novelty is our modelling of high-frequency data at the top level of the Eurodollar Futures limit order book for each trading day of 2018, whereby we predict interest rate curve changes on small time horizons. We are motivated to study the market for these popularly-traded interest rate derivatives since it is deep and liquid, and contributes to the efficient functioning of global finance -- though there is relatively little by way of its modelling contained in the academic literature. Hence, we verify the utility of prediction models and uncertainty estimates for trading applications in this complex and multi-dimensional asset price space.

q-fin.ST↗

arXiv2020

Practical Bayesian Learning of Neural Networks via Adaptive Optimisation Methods

We introduce a novel framework for the estimation of the posterior distribution over the weights of a neural network, based on a new probabilistic interpretation of adaptive optimisation algorithms such as AdaGrad and Adam. We demonstrate the effectiveness of our Bayesian Adam method, Badam, by experimentally showing that the learnt uncertainties correctly relate to the weights' predictive capabilities by weight pruning. We also demonstrate the quality of the derived uncertainty measures by comparing the performance of Badam to standard methods in a Thompson sampling setting for multi-armed bandits, where good uncertainty measures are required for an agent to balance exploration and exploitation.

stat.ML↗

arXiv2020

DeepLOB: Deep Convolutional Neural Networks for Limit Order Books

We develop a large-scale deep learning model to predict price movements from limit order book (LOB) data of cash equities. The architecture utilises convolutional filters to capture the spatial structure of the limit order books as well as LSTM modules to capture longer time dependencies. The proposed network outperforms all existing state-of-the-art algorithms on the benchmark LOB dataset [1]. In a more realistic setting, we test our model by using one year market quotes from the London Stock Exchange and the model delivers a remarkably stable out-of-sample prediction accuracy for a variety of instruments. Importantly, our model translates well to instruments which were not part of the training set, indicating the model's ability to extract universal features. In order to better understand these features and to go beyond a "black box" model, we perform a sensitivity analysis to understand the rationale behind the model predictions and reveal the components of LOBs that are most relevant. The ability to extract robust features which translate well to other instruments is an important property of our model which has many other applications.

q-fin.CP↗

arXiv2019

A Maximum Entropy approach to Massive Graph Spectra

Graph spectral techniques for measuring graph similarity, or for learning the cluster number, require kernel smoothing. The choice of kernel function and bandwidth are typically chosen in an ad-hoc manner and heavily affect the resulting output. We prove that kernel smoothing biases the moments of the spectral density. We propose an information theoretically optimal approach to learn a smooth graph spectral density, which fully respects the moment information. Our method's computational cost is linear in the number of edges, and hence can be applied to large networks, with millions of nodes. We apply our method to the problems to graph similarity and cluster number learning, where we outperform comparable iterative spectral approaches on synthetic and real graphs.

stat.ML↗

arXiv2019

Deep Reinforcement Learning for Trading

We adopt Deep Reinforcement Learning algorithms to design trading strategies for continuous futures contracts. Both discrete and continuous action spaces are considered and volatility scaling is incorporated to create reward functions which scale trade positions based on market volatility. We test our algorithms on the 50 most liquid futures contracts from 2011 to 2019, and investigate how performance varies across different asset classes including commodities, equity indices, fixed income and FX markets. We compare our algorithms against classical time series momentum strategies, and show that our method outperforms such baseline models, delivering positive profits despite heavy transaction costs. The experiments show that the proposed algorithms can follow large market trends without changing positions and can also scale down, or hold, through consolidation periods.

q-fin.CP↗

arXiv2019

Quantum codes from classical graphical models

We introduce a new graphical framework for designing quantum error correction codes based on classical principles. A key feature of this graphical language, over previous approaches, is that it is closely related to that of factor graphs or graphical models in classical information theory and machine learning. It enables us to formulate the description of the recently-introduced `coherent parity check' quantum error correction codes entirely within the language of classical information theory. This makes our construction accessible without requiring background in quantum error correction or even quantum mechanics in general. More importantly, this allows for a collaborative interplay where one can design new quantum error correction codes derived from classical codes.

quant-ph↗

arXiv2019

Extending Deep Learning Models for Limit Order Books to Quantile Regression

We showcase how Quantile Regression (QR) can be applied to forecast financial returns using Limit Order Books (LOBs), the canonical data source of high-frequency financial time-series. We develop a deep learning architecture that simultaneously models the return quantiles for both buy and sell positions. We test our model over millions of LOB updates across multiple different instruments on the London Stock Exchange. Our results suggest that the proposed network not only delivers excellent performance but also provides improved prediction robustness by combining quantile estimates.

q-fin.TR↗

arXiv2019

MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning

Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation.

stat.ML↗

arXiv2019

Population-based Global Optimisation Methods for Learning Long-term Dependencies with RNNs

Despite recent innovations in network architectures and loss functions, training RNNs to learn long-term dependencies remains difficult due to challenges with gradient-based optimisation methods. Inspired by the success of Deep Neuroevolution in reinforcement learning (Such et al. 2017), we explore the use of gradient-free population-based global optimisation (PBO) techniques -- training RNNs to capture long-term dependencies in time-series data. Testing evolution strategies (ES) and particle swarm optimisation (PSO) on an application in volatility forecasting, we demonstrate that PBO methods lead to performance improvements in general, with ES exhibiting the most consistent results across a variety of architectures.

stat.ML↗

arXiv2019

Entropic Spectral Learning for Large-Scale Graphs

Graph spectra have been successfully used to classify network types, compute the similarity between graphs, and determine the number of communities in a network. For large graphs, where an eigen-decomposition is infeasible, iterative moment matched approximations to the spectra and kernel smoothing are typically used. We show that the underlying moment information is lost when using kernel smoothing. We further propose a spectral density approximation based on the method of Maximum Entropy, for which we develop a new algorithm. This method matches moments exactly and is everywhere positive. We demonstrate its effectiveness and superiority over existing approaches in learning graph spectra, via experiments on both synthetic networks, such as the Erdős-Rényi and Barabási-Albert random graphs, and real-world networks, such as the social networks for Orkut, YouTube, and Amazon from the SNAP dataset.

stat.ML↗

arXiv2019

Decoding quantum error correction with Ising model hardware

Fault tolerant quantum computers will require efficient co-processors for real-time decoding of their adopted quantum error correction protocols. In this work we examine the possibility of using specialised Ising model hardware to perform this decoding task. Examples of Ising model hardware include quantum annealers such as those produced by D-Wave Systems Inc., as well as classical devices such as those produced by Hitatchi and Fujitsu and optical devices known as coherent Ising machines. We use the coherent parity check (CPC) framework to derive an Ising model mapping of the quantum error correction decoding problem for an uncorrelated quantum error model. A specific advantage of our Ising model mapping is that it is compatible with maximum entropy inference techniques which can outperform maximum likelihood decoding in some circumstances. We use numerical calculations within our framework to demonstrate that maximum entropy decoding can not only lead to improved error suppression, but can also shift threshold values for simple codes. As a high value problem for which a small advantage can lead to major gains, we argue that decoding quantum codes is an ideal use case for quantum annealers. In addition, the structure of quantum error correction codes allows application specific integrated circuit (ASIC) annealing hardware to reduce embedding costs, a major bottleneck in quantum annealing. Finally, we also propose a way in which a quantum annealer could be optimally used as part of a hybrid quantum-classical decoding scheme.

quant-ph↗

arXiv2018

BDLOB: Bayesian Deep Convolutional Neural Networks for Limit Order Books

We showcase how dropout variational inference can be applied to a large-scale deep learning model that predicts price movements from limit order books (LOBs), the canonical data source representing trading and pricing movements. We demonstrate that uncertainty information derived from posterior predictive distributions can be utilised for position sizing, avoiding unnecessary trades and improving profits. Further, we test our models by using millions of observations across several instruments and markets from the London Stock Exchange. Our results suggest that those Bayesian techniques not only deliver uncertainty information that can be used for trading but also improve predictive performance as stochastic regularisers. To the best of our knowledge, we are the first to apply Bayesian networks to LOBs.

q-fin.CP↗

arXiv2018

Gradient descent in Gaussian random fields as a toy model for high-dimensional optimisation in deep learning

In this paper we model the loss function of high-dimensional optimization problems by a Gaussian random field, or equivalently a Gaussian process. Our aim is to study gradient descent in such loss functions or energy landscapes and compare it to results obtained from real high-dimensional optimization problems such as encountered in deep learning. In particular, we analyze the distribution of the improved loss function after a step of gradient descent, provide analytic expressions for the moments as well as prove asymptotic normality as the dimension of the parameter space becomes large. Moreover, we compare this with the expectation of the global minimum of the landscape obtained by means of the Euler characteristic of excursion sets. Besides complementing our analytical findings with numerical results from simulated Gaussian random fields, we also compare it to loss functions obtained from optimisation problems on synthetic and real data sets by proposing a "black box" random field toy-model for a deep neural network loss function.

stat.ML↗

arXiv2017

On the ratio probability of the smallest eigenvalues in the Laguerre Unitary Ensemble

We study the probability distribution of the ratio between the second smallest and smallest eigenvalue in the $n\times n$ Laguerre Unitary Ensemble. The probability that this ratio is greater than $r>1$ is expressed in terms of an $n \times n$ Hankel determinant with a perturbed Laguerre weight. The limiting probability distribution for the ratio as $n\to\infty$ is found as an integral over $(0,\infty)$ containing two functions $q_{1}(x)$ and $q_{2}(x)$. These functions satisfy a system of two coupled Painlevé V equations, which are derived from a Lax pair of a Riemann-Hilbert problem. We compute asymptotic behaviours of these functions as $rx \to 0_{+}$ and $(r-1)x \to \infty$, as well large $n$ asymptotics for the associated Hankel determinants in several regimes of $r$ and $x$.

math-ph↗

arXiv2017

Quantum-classical truncated Newton method for high-dimensional energy landscapes

We develop a quantum-classical hybrid algorithm for function optimization that is particularly useful in the training of neural networks since it makes use of particular aspects of high-dimensional energy landscapes. Due to a recent formulation of semi-supervised learning as an optimization problem, the algorithm can further be used to find the optimal model parameters for deep generative models. In particular, we present a truncated saddle-free Newton's method based on recent insight from optimization, analysis of deep neural networks and random matrix theory. By combining these with the specific quantum subroutines we are able to exhaust quantum computing in order to arrive at a new quantum-classical hybrid algorithm design. Our algorithm is expected to perform at least as well as existing classical algorithms while achieving a polynomial speedup. The speedup is limited by the required classical read-out. Omitting this requirement can in theory lead to an exponential speedup.

quant-ph↗

arXiv2017

Circuit design for multi-body interactions in superconducting quantum annealing system with applications to a scalable architecture

Quantum annealing provides a way of solving optimization problems by encoding them as Ising spin models which are implemented using physical qubits. The solution of the optimization problem then corresponds to the ground state of the system. Quantum tunnelling is harnessed to enable the system to move to the ground state in a potentially highly non-convex energy landscape. A major difficulty in encoding optimization problems in physical quantum annealing devices is the fact that many real world optimization problems require interactions of higher connectivity as well as multi-body terms beyond the limitations of the physical hardware. In this work we address the question of how to implement multi-body interactions using hardware which natively only provides two-body interactions. The main result is an efficient circuit design of such multi-body terms using superconducting flux qubits in which effective N-body interactions are implemented using N ancilla qubits and only two inductive couplers. It is then shown how this circuit can be used as the unit cell of a scalable architecture by applying it to a recently proposed embedding technique for constructing an architecture of logical qubits with arbitrary connectivity using physical qubits which have nearest-neighbor four-body interactions. It is further shown that this design is robust to non-linear effects in the coupling loops as well as mismatches in some of the circuit parameters.

quant-ph↗

arXiv2016

A Direct Mapping of Max k-SAT and High Order Parity Checks to a Chimera Graph

We demonstrate a direct mapping of max k-SAT problems (and weighted max k-SAT) to a Chimera graph, which is the non-planar hardware graph of the devices built by D-Wave Systems Inc. We further show that this mapping can be used to map a similar class of maximum satisfiability problems where the clauses are replaced by parity checks over potentially large numbers of bits. The latter is of specific interest for applications in decoding for communication. We discuss an example in which the decoding of a turbo code, which has been demonstrated to perform near the Shannon limit, can be mapped to a Chimera graph. The weighted max k-SAT problem is the most general class of satisfiability problems, so our result effectively demonstrates how any satisfiability problem may be directly mapped to a Chimera graph. Our methods faithfully reproduce the low energy spectrum of the target problems, so therefore may also be used for maximum entropy inference.

quant-ph↗

arXiv2016

Estimating a probability mass function with unknown labels

In the context of a species sampling problem we discuss a non-parametric maximum likelihood estimator for the underlying probability mass function. The estimator is known in the computer science literature as the high profile estimator. We prove strong consistency and derive the rates of convergence, for an extended model version of the estimator. We also study a sieved estimator for which similar consistency results are derived. Numerical computation of the sieved estimator is of great interest for practical problems, such as forensic DNA analysis, and we present a computational algorithm based on the stochastic approximation of the expectation maximisation algorithm. As an interesting byproduct of the numerical analyses we introduce an algorithm for bounded isotonic regression for which we also prove convergence.

math.ST↗