arXiv subjects

Matthew Willetts

Publications and source records attributed to Matthew Willetts.

At least 19 recordsLinked to original sources

arXiv2026

Riemannian Geometry of Optimal Rebalancing in Dynamic Weight Automated Market Makers

We show that when a dynamic-weight AMM rebalances by creating arbitrage opportunities, the per-step log loss is the KL divergence between successive weight vectors. The Fisher-Rao metric is therefore the natural Riemannian metric on the weight simplex. The loss-minimising interpolation under the leading-order expansion of this KL cost is SLERP (Spherical Linear Interpolation) in the Hellinger coordinates $\eta_i = \sqrt{w_i}$: a geodesic on the positive orthant of the unit sphere, traversed at constant speed. The SLERP midpoint equals the (AM+GM)/normalise heuristic of prior work (Willetts & Harrington, 2024), so the heuristic lies on the geodesic. This identity holds for any number of tokens and any magnitude of weight change; using this link, all dyadic points on the geodesic can be reached by recursive AM-GM bisection without trigonometric functions. SLERP's relative sub-optimality on the full KL cost is proportional to the squared magnitude of the overall weight change and to $1/f^2$, where $f$ is the number of interpolation steps. Under driftless GBM prices, the fractional value loss from each weight update is price-independent, and the cross term between weight and price changes telescopes, so the constant-price geometry carries over. LVR exposure introduces a finite optimal step count $f^*$, which lies in the perturbative regime where SLERP remains near-optimal.

q-fin.MF↗

arXiv2026

Pools as Portfolios: Observed arbitrage efficiency & LVR analysis of dynamic weight AMMs

Dynamic-weight AMMs (aka Temporal Function Market Makers, TFMMs) implement algorithmic asset allocation, analogous to index or smart beta funds, by continuously updating pools' weights. A strategy updates target weights over time, and arbitrageurs trade the pool back toward those weights. This creates a sequence of small, predictable mispricings that grow until taken, effectively executing rebalances as a series of Dutch reverse auctions. Prior theoretical and simulation work (Willetts & Harrington, 2024) predicted that this mechanism could outperform CEX-style rebalancing. We test that claim on two live pools on the QuantAMM protocol, one on Ethereum mainnet and one on Base, across two short rebalancing windows six months apart (July 2025 and January 2026). We perform block-level arbitrage analysis, and then measure long term outcomes using Loss-vs-Rebalancing (LVR) and Rebalancing-vs-Rebalancing (RVR) benchmarks. On mainnet, rebalancing becomes markedly more efficient over time (more frequent arbitrage trades with lower value extracted per trade), reaching performance comparable to or better than CEX-based models. On Base, rebalancing persists even when per-trade extraction is near (or below) zero, consistent with routing-driven execution, and achieves efficiencies that meet or exceed standard "perfect rebalancing" LVR baselines. These results demonstrate dynamic-weight AMMs as a competitive execution layer for tokenised funds, with superior performance on L2s where routing and lower data costs compress arbitrage spreads.

q-fin.TR↗

arXiv2024

Rebalancing-versus-Rebalancing: Improving the fidelity of Loss-versus-Rebalancing

Automated Market Makers (AMMs) hold assets and are constantly being rebalanced by external arbitrageurs to match external market prices. Loss-versus-rebalancing (LVR) is a pivotal metric for measuring how an AMM pool performs for its liquidity providers (LPs) relative to an idealised benchmark where rebalancing is done not via the action of arbitrageurs but instead by trading with a perfect centralised exchange with no fees, spread or slippage. This renders it an imperfect tool for judging rebalancing efficiency between execution platforms. We introduce Rebalancing-versus-rebalancing (RVR), a higher-fidelity model that better captures the frictions present in centralised rebalancing. We perform a battery of experiments comparing managing a portfolio on AMMs vs this new and more realistic centralised exchange benchmark-RVR. We are also particularly interested in dynamic AMMs that run strategies beyond fixed weight allocations-Temporal Function Market Makers. This is particularly important for asset managers evaluating execution management systems. In this paper we simulate more than 1000 different strategies settings as well as testing hundreds of different variations in centralised exchange (CEX) fees, AMM fees & gas costs. We find that, under this modeling approach, AMM pools (even with no retail/noise traders) often offer superior execution and rebalancing efficiency compared to centralised rebalancing, for all but the lowest CEX fee levels. We also take a simple approach to model noise traders & find that even a small amount of noise volume increases modeled AMM performance such that CEX rebalancing finds it hard to compete. This indicates that decentralised AMM-based asset management can offer superior performance and execution management for asset managers looking to rebalance portfolios, offering an alternative use case for dynamic AMMs beyond core liquidity providing.

q-fin.TR↗

arXiv2024

Multiblock MEV opportunities & protections in dynamic AMMs

Maximal Extractable Value (MEV) in Constant Function Market Making is fairly well understood. Does having dynamic weights, as found in liquidity boostrap pools (LBPs), Temporal-function market makers (TFMMs), and Replicating market makers (RMMs), introduce new attack vectors? In this paper we explore how inter-block weight changes can be analogous to trades, and can potentially lead to a multi-block MEV attack. New inter-block protections required to guard against this new attack vector are analysed. We also carry our a raft of numerical simulations, more than 450 million potential attack scenarios, showing both successful attacks and successful defense.

q-fin.TR↗

arXiv2024

Optimal Rebalancing in Dynamic AMMs

Dynamic AMM pools, as found in Temporal Function Market Making, rebalance their holdings to a new desired ratio (e.g. moving from being 50-50 between two assets to being 90-10 in favour of one of them) by introducing an arbitrage opportunity that disappears when their holdings are in line with their target. Structuring this arbitrage opportunity reduces to the problem of choosing the sequence of portfolio weights the pool exposes to the market via its trading function. Linear interpolation from start weights to end weights has been used to reduce the cost paid by pools to arbitrageurs to rebalance. Here we obtain the $\textit{optimal}$ interpolation in the limit of small weight changes (which has the downside of requiring a call to a transcendental function) and then obtain a cheap-to-compute approximation to that optimal approach that gives almost the same performance improvement. We then demonstrate this method on a range of market backtests, including simulating pool performance when trading fees are present, finding that the new approximately-optimal method of changing weights gives robust increases in pool performance. For a BTC-ETH-DAI pool from July 2022 to June 2023, the increases of pool P\&L from approximately-optimal weight changes is $\sim25\%$ for a range of different strategies and trading fees.

q-fin.TR↗

arXiv2024

Closed-form solutions for generic N-token AMM arbitrage

Convex optimisation has provided a mechanism to determine arbitrage trades on automated market markets (AMMs) since almost their inception. Here we outline generic closed-form solutions for $N$-token geometric mean market maker pool arbitrage, that in simulation (with synthetic and historic data) provide better arbitrage opportunities than convex optimisers and is able to capitalise on those opportunities sooner. Furthermore, the intrinsic parallelism of the proposed approach (unlike convex optimisation) offers the ability to scale on GPUs, opening up a new approach to AMM modelling by offering an alternative to numerical-solver-based methods. The lower computational cost of running this new mechanism can also enable on-chain arbitrage bots for multi-asset pools.

q-fin.TR↗

arXiv2023

A Multi-Resolution Framework for U-Nets with Applications to Hierarchical VAEs

U-Net architectures are ubiquitous in state-of-the-art deep learning, however their regularisation properties and relationship to wavelets are understudied. In this paper, we formulate a multi-resolution framework which identifies U-Nets as finite-dimensional truncations of models on an infinite-dimensional function space. We provide theoretical results which prove that average pooling corresponds to projection within the space of square-integrable functions and show that U-Nets with average pooling implicitly learn a Haar wavelet basis representation of the data. We then leverage our framework to identify state-of-the-art hierarchical VAEs (HVAEs), which have a U-Net architecture, as a type of two-step forward Euler discretisation of multi-resolution diffusion processes which flow from a point mass, introducing sampling instabilities. We also demonstrate that HVAEs learn a representation of time which allows for improved parameter efficiency through weight-sharing. We use this observation to achieve state-of-the-art HVAE performance with half the number of parameters of existing models, exploiting the properties of our continuous-time formulation.

stat.ML↗

arXiv2022

I Don't Need u: Identifiable Non-Linear ICA Without Side Information

In this paper, we investigate the algorithmic stability of unsupervised representation learning with deep generative models, as a function of repeated re-training on the same input data. Algorithms for learning low dimensional linear representations -- for example principal components analysis (PCA), or linear independent components analysis (ICA) -- come with guarantees that they will always reveal the same latent representations (perhaps up to an arbitrary rotation or permutation). Unfortunately, for non-linear representation learning, such as in a variational auto-encoder (VAE) model trained by stochastic gradient descent, we have no such guarantees. Recent work on identifiability in non-linear ICA have introduced a family of deep generative models that have identifiable latent representations, achieved by conditioning on side information (e.g. informative labels). We empirically evaluate the stability of these models under repeated re-estimation of parameters, and compare them to both standard VAEs and deep generative models which learn to cluster in their latent space. Surprisingly, we discover side information is not necessary for algorithmic stability: using standard quantitative measures of identifiability, we find deep generative models with latent clusterings are empirically identifiable to the same degree as models which rely on auxiliary labels. We relate these results to the possibility of identifiable non-linear ICA.

cs.LG↗

arXiv2022

Certifiably Robust Variational Autoencoders

We introduce an approach for training Variational Autoencoders (VAEs) that are certifiably robust to adversarial attack. Specifically, we first derive actionable bounds on the minimal size of an input perturbation required to change a VAE's reconstruction by more than an allowed amount, with these bounds depending on certain key parameters such as the Lipschitz constants of the encoder and decoder. We then show how these parameters can be controlled, thereby providing a mechanism to ensure \textit{a priori} that a VAE will attain a desired level of robustness. Moreover, we extend this to a complete practical approach for training such VAEs to ensure our criteria are met. Critically, our method allows one to specify a desired level of robustness \emph{upfront} and then train a VAE that is guaranteed to achieve this robustness. We further demonstrate that these Lipschitz--constrained VAEs are more robust to attack than standard VAEs in practice.

stat.ML↗

arXiv2022

Variational Autoencoders: A Harmonic Perspective

In this work we study Variational Autoencoders (VAEs) from the perspective of harmonic analysis. By viewing a VAE's latent space as a Gaussian Space, a variety of measure space, we derive a series of results that show that the encoder variance of a VAE controls the frequency content of the functions parameterised by the VAE encoder and decoder neural networks. In particular we demonstrate that larger encoder variances reduce the high frequency content of these functions. Our analysis allows us to show that increasing this variance effectively induces a soft Lipschitz constraint on the decoder network of a VAE, which is a core contributor to the adversarial robustness of VAEs. We further demonstrate that adding Gaussian noise to the input of a VAE allows us to more finely control the frequency content and the Lipschitz constant of the VAE encoder networks. To support our theoretical analysis we run experiments with VAEs with small fully-connected neural networks and with larger convolutional networks, demonstrating empirically that our theory holds for a variety of neural network architectures.

stat.ML↗

arXiv2021

Multi-Facet Clustering Variational Autoencoders

Work in deep clustering focuses on finding a single partition of data. However, high-dimensional data, such as images, typically feature multiple interesting characteristics one could cluster over. For example, images of objects against a background could be clustered over the shape of the object and separately by the colour of the background. In this paper, we introduce Multi-Facet Clustering Variational Autoencoders (MFCVAE), a novel class of variational autoencoders with a hierarchy of latent variables, each with a Mixture-of-Gaussians prior, that learns multiple clusterings simultaneously, and is trained fully unsupervised and end-to-end. MFCVAE uses a progressively-trained ladder architecture which leads to highly stable performance. We provide novel theoretical results for optimising the ELBO analytically with respect to the categorical variational posterior distribution, correcting earlier influential theoretical work. On image benchmarks, we demonstrate that our approach separates out and clusters over different aspects of the data in a disentangled manner. We also show other advantages of our model: the compositionality of its latent space and that it provides controlled generation of samples.

stat.ML↗

arXiv2021

Relaxed-Responsibility Hierarchical Discrete VAEs

Successfully training Variational Autoencoders (VAEs) with a hierarchy of discrete latent variables remains an area of active research. Vector-Quantised VAEs are a powerful approach to discrete VAEs, but naive hierarchical extensions can be unstable when training. Leveraging insights from classical methods of inference we introduce \textit{Relaxed-Responsibility Vector-Quantisation}, a novel way to parameterise discrete latent variables, a refinement of relaxed Vector-Quantisation that gives better performance and more stable training. This enables a novel approach to hierarchical discrete variational autoencoders with numerous layers of latent variables (here up to 32) that we train end-to-end. Within hierarchical probabilistic deep generative models with discrete latent variables trained end-to-end, we achieve state-of-the-art bits-per-dim results for various standard datasets. % Unlike discrete VAEs with a single layer of latent variables, we can produce samples by ancestral sampling: it is not essential to train a second autoregressive generative model over the learnt latent representations to then sample from and then decode. % Moreover, that latter approach in these deep hierarchical models would require thousands of forward passes to generate a single sample. Further, we observe different layers of our model become associated with different aspects of the data.

stat.ML↗

arXiv2021

Improving VAEs' Robustness to Adversarial Attack

Variational autoencoders (VAEs) have recently been shown to be vulnerable to adversarial attacks, wherein they are fooled into reconstructing a chosen target image. However, how to defend against such attacks remains an open problem. We make significant advances in addressing this issue by introducing methods for producing adversarially robust VAEs. Namely, we first demonstrate that methods proposed to obtain disentangled latent representations produce VAEs that are more robust to these attacks. However, this robustness comes at the cost of reducing the quality of the reconstructions. We ameliorate this by applying disentangling methods to hierarchical VAEs. The resulting models produce high-fidelity autoencoders that are also adversarially robust. We confirm their capabilities on several different datasets and with current state-of-the-art VAE adversarial attacks, and also show that they increase the robustness of downstream tasks to attack.

stat.ML↗

arXiv2021

Learning Bijective Feature Maps for Linear ICA

Separating high-dimensional data like images into independent latent factors, i.e independent component analysis (ICA), remains an open research problem. As we show, existing probabilistic deep generative models (DGMs), which are tailor-made for image data, underperform on non-linear ICA tasks. To address this, we propose a DGM which combines bijective feature maps with a linear ICA model to learn interpretable latent structures for high-dimensional data. Given the complexities of jointly training such a hybrid model, we introduce novel theory that constrains linear ICA to lie close to the manifold of orthogonal rectangular matrices, the Stiefel manifold. By doing so we create models that converge quickly, are easy to train, and achieve better unsupervised latent factor discovery than flow-based models, linear ICA, and Variational Autoencoders on images.

cs.LG↗

arXiv2021

Towards a Theoretical Understanding of the Robustness of Variational Autoencoders

We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations. While previous work has developed algorithmic approaches to attacking and defending VAEs, there remains a lack of formalization for what it means for a VAE to be robust. To address this, we develop a novel criterion for robustness in probabilistic models: $r$-robustness. We then use this to construct the first theoretical results for the robustness of VAEs, deriving margins in the input space for which we can provide guarantees about the resulting reconstruction. Informally, we are able to define a region within which any perturbation will produce a reconstruction that is similar to the original reconstruction. To support our analysis, we show that VAEs trained using disentangling methods not only score well under our robustness metrics, but that the reasons for this can be interpreted through our theoretical results.

stat.ML↗

arXiv2021

Explicit Regularisation in Gaussian Noise Injections

We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have been few studies on understanding the regularising effect they induce when applied to network activations. Here we derive the explicit regulariser of GNIs, obtained by marginalising out the injected noise, and show that it penalises functions with high-frequency components in the Fourier domain; particularly in layers closer to a neural network's output. We show analytically and empirically that such regularisation produces calibrated classifiers with large classification margins.

stat.ML↗

arXiv2021

Semi-Unsupervised Learning: Clustering and Classifying using Ultra-Sparse Labels

In semi-supervised learning for classification, it is assumed that every ground truth class of data is present in the small labelled dataset. Many real-world sparsely-labelled datasets are plausibly not of this type. It could easily be the case that some classes of data are found only in the unlabelled dataset -- perhaps the labelling process was biased -- so we do not have any labelled examples to train on for some classes. We call this learning regime $\textit{semi-unsupervised learning}$, an extreme case of semi-supervised learning, where some classes have no labelled exemplars in the training set. First, we outline the pitfalls associated with trying to apply deep generative model (DGM)-based semi-supervised learning algorithms to datasets of this type. We then show how a combination of clustering and semi-supervised learning, using DGMs, can be brought to bear on this problem. We study several different datasets, showing how one can still learn effectively when half of the ground truth classes are entirely unlabelled and the other half are sparsely labelled.

stat.ML↗

arXiv2020

Non-Determinism in TensorFlow ResNets

We show that the stochasticity in training ResNets for image classification on GPUs in TensorFlow is dominated by the non-determinism from GPUs, rather than by the initialisation of the weights and biases of the network or by the sequence of minibatches given. The standard deviation of test set accuracy is 0.02 with fixed seeds, compared to 0.027 with different seeds---nearly 74\% of the standard deviation of a ResNet model is non-deterministic. For test set loss the ratio of standard deviations is more than 80\%. These results call for more robust evaluation strategies of deep learning models, as a significant amount of the variation in results across runs can arise simply from GPU randomness.

cs.LG↗