arXiv subjects

Christoph Reisinger

Publications and source records attributed to Christoph Reisinger.

At least 19 recordsLinked to original sources

arXiv2026

A stochastic Fokker--Planck equation for the mean-field limit of a population of noisy integrate-and-fire neurons

We study a densely connected system of excitatory integrate-and-fire neurons which are subject to common noise. The system incorporates a gradual transmission of action potentials and captures the re- and hyperpolarization phases through a random refractory period followed by a reset to a randomized level below the rest potential. As the number of neurons tends to infinity, we show that there is weak convergence to a unique membrane potential density governed by a stochastic Fokker--Planck equation with a well-defined spike transmission rate. The latter is driven by the mean cumulative spike count, which is non-differentiable but shown to satisfy a generalized flux condition. We obtain the uniqueness of the Fokker--Planck equation from energy estimates in the dual of the first Sobolev space. Finally, we give a conditional McKean--Vlasov representation of the membrane potential density as the law of a representative neuron given the common noise.

math.AP↗

arXiv2026

Probabilistic estimates for a system of noisy integrate-and-fire neurons

In this note, we establish various probabilistic estimates for an interacting particle system that describes the evolution of the membrane potentials in a network of excitatory integrate-and-fire neurons, which are subject to both idiosyncratic and common noise. These estimates serve to support a separate work that studies the large population limit and the corresponding stochastic Fokker--Planck equation for the membrane potential density.

math.PR↗

arXiv2026

Numerical Approximation for Path-Dependent McKean-Vlasov Control with Non-Asymptotic Error Estimates

Path-dependent McKean--Vlasov (MKV) control models large interacting populations with history-dependent dynamics and costs. This paper develops a unified approximation-and-learning framework for continuous time path-dependent MKV problem under open-loop controls. First, an Euler discretization scheme with piecewise-constant controls is shown to achieve a non-asymptotic error of $O(h^{1/4})$. Second, we establish a discrete dynamic programming principle and prove value equivalence between open-loop and history-dependent feedback controls, enabling optimization on a reduced filtration. Third, an interacting particle system is introduced to approximate the continuous-time value, yielding an overall error bound of $O(h^{1/4}) + O(M^{-\gamma})$ for $M$ particles and an explicitly given $\gamma > 0$. Finally, we propose a fully implementable neural-network policy-gradient method using pathwise features. Numerical experiments, including a path-dependent linear-quadratic benchmark, demonstrate the effectiveness of the algorithm.

math.OC↗

arXiv2026

Uniform Scaling Limits in AdamW-Trained Transformers

We study the large-depth limit of transformers trained with AdamW, by modelling the hidden-state dynamics as an interacting particle system (IPS) coupled through the attention mechanism. Under appropriate scaling of the attention heads, we prove that the joint dynamics of the hidden states and backpropagated variables converge in $L^2$, uniformly over the initial condition, to the solution of a forward--backward system of ODEs at rate $\mathcal O(L^{-1}+L^{-1/3}H^{-1/2})$. Here, $L$ and $H$ denote the depth and number of heads of the transformer, respectively. The limiting system of ODEs can be identified with a McKean--Vlasov ODE (MVODE) when the attention heads do not incorporate causal masking. By using the flow maps associated with this MVODE and applying concentration of measure techniques, we obtain bounds on the difference between the discrete and continuous models that are uniform over compact sets of initial conditions. As this is achieved without resorting to a covering argument, the constants in our bounds are independent of the number of tokens. Furthermore, under a suitable adaptation to AdamW, the bounds become independent of the token embedding dimension.

stat.ML↗

arXiv2026

Model-free policy gradient for discrete-time mean-field control

We study model-free policy learning for discrete-time mean-field control (MFC) problems with finite state space and compact action space. In contrast to the extensive literature on value-based methods for MFC, policy-based approaches remain largely unexplored due to the intrinsic dependence of transition kernels and rewards on the evolving population state distribution, which prevents the direct use of likelihood-ratio estimators of policy gradients from classical single-agent reinforcement learning. We introduce a novel perturbation scheme on the state-distribution flow and prove that the gradient of the resulting perturbed value function converges to the true policy gradient as the perturbation magnitude vanishes. This construction yields a fully model-free estimator based solely on simulated trajectories and an auxiliary estimate of the sensitivity of the state distribution. Building on this framework, we develop MF-REINFORCE, a model-free policy gradient algorithm for MFC, and establish explicit quantitative bounds on its bias and mean-squared error. Numerical experiments on representative mean-field control tasks demonstrate the effectiveness of the proposed approach.

math.OC↗

arXiv2026

A higher order sparse grid combination technique

We show that a generalised sparse grid combination technique which combines multi-variate extrapolation of finite difference solutions with the standard combination formula lifts a second order accurate scheme on regular meshes to a fourth order combined sparse grid solution. In the analysis, working in a general dimension, we characterise all terms in a multivariate error expansion of the scheme as solutions of a sequence of semi-discrete problems. This is first carried out formally under suitable assumptions on the truncation error of the scheme, stability and regularity of solutions. We then verify the assumptions on the example of the Poisson problem with smooth data, and illustrate the practical convergence in up to seven dimensions.

math.NA↗

arXiv2025

Convergence Rates of Time Discretization in Extended Mean Field Control

Piecewise constant control approximation provides a practical framework for designing numerical schemes of continuous-time control problems. We analyze the accuracy of such approximations for extended mean field control (MFC) problems, where the dynamics and costs depend on the joint distribution of states and controls. For linear-convex extended MFC problems, we show that the optimal control is $1/2$-H\"older continuous in time. Using this regularity, we prove that the optimal cost of the continuous-time problem can be approximated by piecewise constant controls with order $1/2$, while the optimal control itself can be approximated with order $1/4$. For general extended MFC problems, we further show that, under sufficient regularity of the value functions, the value functions converge with an improved first-order rate, matching the best-known rate for classical control problems without mean field interaction, and consistent with the numerical observations for MFC of Cucker-Smale models.

math.OC↗

arXiv2025

Mean-Field Generalisation Bounds for Learning Controls in Stochastic Environments

We consider a data-driven formulation of the classical discrete-time stochastic control problem. Our approach exploits the natural structure of many such problems, in which significant portions of the system are uncontrolled. Employing the dynamic programming principle and the mean-field interpretation of single-hidden layer neural networks, we formulate the control problem as a series of infinite-dimensional minimisation problems. When regularised carefully, we provide practically verifiable assumptions for non-asymptotic bounds on the generalisation error achieved by the minimisers to this problem, thus ensuring stability in overparametrised settings, for controls learned using finitely many observations. We explore connections to the traditional noisy stochastic gradient descent algorithm, and subsequently show promising numerical results for some classic control problems.

math.OC↗

arXiv2025

Weighted Conditional Flow Matching

Conditional flow matching (CFM) has emerged as a powerful framework for training continuous normalizing flows due to its computational efficiency and effectiveness. However, standard CFM often produces paths that deviate significantly from straight-line interpolations between prior and target distributions, making generation slower and less accurate due to the need for fine discretization at inference. Recent methods enhance CFM performance by inducing shorter and straighter trajectories but typically rely on computationally expensive mini-batch optimal transport (OT). Drawing insights from entropic optimal transport (EOT), we propose Weighted Conditional Flow Matching (W-CFM), a novel approach that modifies the classical CFM loss by weighting each training pair $(x, y)$ with a Gibbs kernel. We show that this weighting recovers the entropic OT coupling up to some bias in the marginals, and we provide the conditions under which the marginals remain nearly unchanged. Moreover, we establish an equivalence between W-CFM and the minibatch OT method in the large-batch limit, showing how our method overcomes computational and performance bottlenecks linked to batch size. Empirically, we test our method on unconditional generation on various synthetic and real datasets, confirming that W-CFM achieves comparable or superior sample quality, fidelity, and diversity to other alternative baselines while maintaining the computational efficiency of vanilla CFM.

cs.LG↗

arXiv2025

Numerical analysis of a particle system for the calibrated Heston-type local stochastic volatility model

We analyse a Monte Carlo particle method for the simulation of the calibrated Heston-type local stochastic volatility (H-LSV) model. The common application of a kernel estimator for a conditional expectation in the calibration condition results in a McKean-Vlasov (MV) stochastic differential equation (SDE) with non-standard coefficients. The primary challenges lie in certain mean-field terms in the drift and diffusion coefficients and the $1/2$-H\"{o}lder regularity of the diffusion coefficient. We establish the well-posedness of this equation for a fixed but arbitrarily small bandwidth of the kernel estimator. Moreover, we prove a strong propagation of chaos result, ensuring convergence of the particle system under a condition on the Feller ratio and up to a critical time. For the numerical simulation, we employ an Euler-Maruyama scheme for the log-spot process and a full truncation Euler scheme for the CIR volatility process. Under certain conditions on the inputs and the Feller ratio, we prove strong convergence of the Euler-Maruyama scheme with rate $1/2$ in time, up to a logarithmic factor. Numerical experiments illustrate the convergence of the discretisation scheme and validate the propagation of chaos in practice.

q-fin.CP↗

arXiv2025

Efficient Learning for Entropy-Regularized Markov Decision Processes via Multilevel Monte Carlo

Designing efficient learning algorithms with complexity guarantees for Markov decision processes (MDPs) with large or continuous state and action spaces remains a fundamental challenge. We address this challenge for entropy-regularized MDPs with Polish state and action spaces, assuming access to a generative model of the environment. We propose a novel family of multilevel Monte Carlo (MLMC) algorithms that integrate fixed-point iteration with MLMC techniques and a generic stochastic approximation of the Bellman operator. We quantify the precise impact of the chosen approximate Bellman operator on the accuracy of the resulting MLMC estimator. Leveraging this error analysis, we show that using a biased plain MC estimate for the Bellman operator results in quasi-polynomial sample complexity, whereas an unbiased randomized multilevel approximation of the Bellman operator achieves polynomial sample complexity in expectation. Notably, these complexity bounds are independent of the dimensions or cardinalities of the state and action spaces, distinguishing our approach from existing algorithms whose complexities scale with the sizes of these spaces. We validate these theoretical performance guarantees through numerical experiments.

cs.LG↗

arXiv2024

Milstein-type schemes for McKean-Vlasov SDEs driven by Brownian motion and Poisson random measure (with super-linear coefficients)

In this work, we present a general Milstein-type scheme for McKean-Vlasov stochastic differential equations (SDEs) driven by Brownian motion and Poisson random measure and the associated system of interacting particles where drift, diffusion and jump coefficients may grow super-linearly in the state variable and linearly in the measure component. The strong rate of $\mathcal{L}^2$-convergence of the proposed scheme is shown to be arbitrarily close to one under appropriate regularity assumptions on the coefficients. For the derivation of the Milstein scheme and to show its strong rate of convergence, we provide an It\^o formula for the interacting particle system connected with the McKean-Vlasov SDE driven by Brownian motion and Poisson random measure. Moreover, we use the notion of Lions derivative to examine our results. The two-fold challenges arising due to the presence of the empirical measure and super-linearity of the jump coefficient are resolved by identifying and exploiting an appropriate coercivity-type condition.

math.PR↗

arXiv2024

Limit Order Book Simulation and Trade Evaluation with $K$-Nearest-Neighbor Resampling

In this paper, we show how $K$-nearest neighbor ($K$-NN) resampling, an off-policy evaluation method proposed in \cite{giegrich2023k}, can be applied to simulate limit order book (LOB) markets and how it can be used to evaluate and calibrate trading strategies. Using historical LOB data, we demonstrate that our simulation method is capable of recreating realistic LOB dynamics and that synthetic trading within the simulation leads to a market impact in line with the corresponding literature. Compared to other statistical LOB simulation methods, our algorithm has theoretical convergence guarantees under general conditions, does not require optimization, is easy to implement and computationally efficient. Furthermore, we show that in a benchmark comparison our method outperforms a deep learning-based algorithm for several key statistics. In the context of a LOB with pro-rata type matching, we demonstrate how our algorithm can calibrate the size of limit orders for a liquidation strategy. Finally, we describe how $K$-NN resampling can be modified for choices of higher dimensional state spaces.

q-fin.TR↗

arXiv2024

A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems

We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean-Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum parameters. At each step, the state dynamics is approximated via a particle system, and the required gradient is evaluated through a coupled system of nonlocal linear PDEs. The latter is solved by finite difference approximation or neural network-based residual approximation, depending on the state dimension. We present exhaustive numerical experiments for low and high-dimensional mean-field control problems, including sparse stabilization of stochastic Cucker-Smale models, which reveal that our algorithm captures important structures of the optimal feedback control and achieves a robust performance with respect to parameter perturbation.

math.OC↗

arXiv2024

Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problems

We study the global linear convergence of policy gradient (PG) methods for finite-horizon continuous-time exploratory linear-quadratic control (LQC) problems. The setting includes stochastic LQC problems with indefinite costs and allows additional entropy regularisers in the objective. We consider a continuous-time Gaussian policy whose mean is linear in the state variable and whose covariance is state-independent. Contrary to discrete-time problems, the cost is noncoercive in the policy and not all descent directions lead to bounded iterates. We propose geometry-aware gradient descents for the mean and covariance of the policy using the Fisher geometry and the Bures-Wasserstein geometry, respectively. The policy iterates are shown to satisfy an a-priori bound, and converge globally to the optimal policy with a linear rate. We further propose a novel PG method with discrete-time policies. The algorithm leverages the continuous-time analysis, and achieves a robust linear convergence across different action frequencies. A numerical experiment confirms the convergence and robustness of the proposed algorithm.

math.OC↗

arXiv2024

$K$-Nearest-Neighbor Resampling for Off-Policy Evaluation in Stochastic Control

In this paper, we propose a novel $K$-nearest neighbor resampling procedure for estimating the performance of a policy from historical data containing realized episodes of a decision process generated under a different policy. We provide statistical consistency results under weak conditions. In particular, we avoid the common assumption of identically and independently distributed transitions and rewards. Instead, our analysis allows for the sampling of entire episodes, as is common practice in most applications. To establish the consistency in this setting, we generalize Stone's Theorem, a well-known result in nonparametric statistics on local averaging, to include episodic data and the counterfactual estimation underlying off-policy evaluation (OPE). By focusing on feedback policies that depend deterministically on the current state in environments with continuous state-action spaces and system-inherent stochasticity effected by chosen actions, and relying on trajectory simulation similar to Monte Carlo methods, the proposed method is particularly well suited for stochastic control environments. Compared to other OPE methods, our algorithm does not require optimization, can be efficiently implemented via tree-based nearest neighbor search and parallelization, and does not explicitly assume a parametric model for the environment's dynamics. Numerical experiments demonstrate the effectiveness of the algorithm compared to existing baselines in a variety of stochastic control settings, including a linear quadratic regulator, trade execution in limit order books, and online stochastic bin packing.

stat.ML↗

arXiv2023

Randomness and early termination: what makes a game exciting?

In this paper we revisit an open problem posed by Aldous on the max-entropy win-probability martingale: given two players of equal strength, such that the win-probability is a martingale diffusion, which of these processes has maximum entropy and hence gives the most excitement for the spectators? We study a terminal-boundary value problem for the nonlinear parabolic PDE $2\partial_te(t,x)=\log(-\partial_{xx}e(t,x))$ derived by Aldous and prove its wellposedness and regularity of its solution by combining PDE analysis and probabilistic tools, in particular the reformulation as a stochastic control problem with restricted control set, which allows us to deduce strict ellipticity. We establish key qualitative properties of the solution including concavity, monotonicity, convergence to a steady state for long remaining time and the asymptotic behaviour shortly before the terminal time. Moreover, we construct convergent numerical approximations. The analytical and numerical results allow us to highlight the behaviour of the win-probability process in the present case where the match may end early, in contrast to recent work by Backhoff-Veraguas and Beiglböck where the match always runs the full length.

math.PR↗

arXiv2023

Mean-field games of speedy information access with observation costs

We investigate mean-field games (MFG) in which agents actively control their speed of access to information. Specifically, the agents can dynamically decide to obtain observations with reduced delay by accepting higher observation costs. Agents seek to exploit their active information acquisition by making further decisions to influence their state dynamics so as to maximise rewards. In a mean-field equilibrium, each generic agent solves individually a partially observed Markov decision problem in which the way partial observations are obtained is itself subject to dynamic control actions, while no agent can improve unilaterally given the actions of all others. We formulate the mean-field game with controlled costly information access as an equivalent standard mean-field game on an augmented space, by utilizing a parameterisation of the belief state by a finite number of variables. With sufficient entropy regularisation, a fixed point iteration converges to the unique MFG equilibrium. Moreover, we derive an approximate $\varepsilon$-Nash equilibrium for a large but finite population size and small regularisation parameter. We illustrate our (extended) MFG of information access and of controls by an example from epidemiology, where medical testing results can be procured at different speeds and costs.

math.OC↗