arXiv subjects

Mike Ludkovski

Publications and source records attributed to Mike Ludkovski.

At least 19 recordsLinked to original sources

arXiv2026

Groundwater Management: Combating the Sinking Feeling

This non-technical survey introduces the topic of groundwater management to the mathematics and statistics communities. We focus on the emerging marketplaces for groundwater pumping rights, outlining probabilistic and statistical approaches to groundwater dynamics and game equilibria that underpin stakeholder decisions. After discussing rationing schemes and regulatory policies we list a range of open problems that are well suited to be tackled by the mathematical sciences researchers.

econ.GN↗

arXiv2026

Battery Storage Co-Optimization in Day-Ahead and Real-Time Markets with Bayesian Optimization

We propose Adaptive Refinement Bayesian Optimization for Day-Ahead and Real-Time (ARBO-DART) markets, an algorithm for BESS intraday dispatch co-optimization in which day-ahead (DA) commitment profiles are optimized against value of real-time (RT) recourse computed by a black-box stochastic control solver. In our framework, the DA price curve is taken as exogenous and RT prices evolve as a mean-reverting process around it. The RT recourse layer performs dynamic closed-loop control while accounting for the piecewise-linear state-of-charge dynamics and the DA-driven feasible control set. By wrapping Bayesian Optimization (BO) around the RT solver, ARBO-DART jointly optimizes DA commitments and dynamic RT flexibility without requiring analytic gradients, closed-form value functions, or finite-scenario approximations. To overcome the curse of dimensionality in fixed-resolution DA profiles, ARBO-DART starts from a coarse partition of DA commitments and progressively refines charge and discharge blocks where additional temporal resolution is most needed, as judged by the corresponding RT policy. Numerical experiments across realistic DA price curves reveal the effectiveness of ARBO-DART in recovering economically meaningful DA bidding structures while being several times faster relative to fixed-resolution

q-fin.CP↗

arXiv2026

Modeling Stochastic Multi-Agent Interaction in Intraday Battery Energy Storage Dispatch with Market Power

We develop a stochastic game-theoretic model for intraday dispatch of grid-scale battery energy storage systems (BESSs). We assume that each BESS operator competitively manages her state-of-charge to maximize energy arbitrage revenues, driven by the endogenized electricity price that depends on the sum of the charging rates. We characterize the Nash equilibrium of the resulting finite-player linear-quadratic differential game with a shared stochastic driver, obtaining semi-explicit representations of equilibrium feedback controls and equilibrium prices both in the general heterogeneous and the simplified homogeneous BESS setting, via a system of Riccati equations. We then analyze competitive effects, including the marginal externality of additional BESS entering the market, the benefit of coordination and the corresponding market power of large operators, and supply effects from hybrid-type BESSs. We further study the asymptotic regime as the number of agents grows large. Our model provides a quantitative testbed to study the impact of decentralized BESS deployment on the grid and the resulting reduction in daily price spreads.

math.OC↗

arXiv2025

Functional Analysis of Loss-development Patterns in P&C Insurance

We analyze loss development in NAIC Schedule P loss triangles using functional data analysis methods. Adopting the functional viewpoint, our dataset comprises 3300+ curves of incremental loss ratios (ILR) of workers' compensation lines over 24 accident years. Relying on functional data depth, we first study similarities and differences in development patterns based on company-specific covariates, as well as identify anomalous ILR curves. The exploratory findings motivate the probabilistic forecasting framework developed in the second half of the paper. We propose a functional model to complete partially developed ILR curves based on partial least squares regression of PCA scores. Coupling the above with functional bootstrapping allows us to quantify future ILR uncertainty jointly across all future lags. We demonstrate that our method has much better probabilistic scores relative to Chain Ladder and in particular can provide accurate functional predictive intervals.

stat.AP↗

arXiv2025

Intraday Battery Dispatch for Hybrid Renewable Energy Assets

We develop a mathematical model for intraday dispatch of co-located wind-battery energy assets. Focusing on the primary objective of firming grid-side actual production vis-a-vis the preset day-ahead hourly generation targets, we conduct a comprehensive study of the resulting stochastic control problem across different firming formulations and wind generation dynamics. Among others, we provide a closed-form solution in the special case of a quadratic objective and linear dynamics, as well as design a novel adaptation of a Gaussian Process-based Regression Monte Carlo algorithm for our setting. Extensions studied include an asymmetric loss function for peak shaving, capturing the cost of battery cycling, and the role of battery duration. In the applied portion of our work, we calibrate our model to a collection of 140+ wind-battery assets in Texas, benchmarking the economic benefits of firming based on outputs of a realistic unit commitment and economic dispatch solver.

math.OC↗

arXiv2025

Selecting Critical Scenarios of DER Adoption in Distribution Grids Using Bayesian Optimization

We develop a new methodology to select scenarios of DER adoption most critical for distribution grids. Anticipating risks of future voltage and line flow violations due to additional PV adopters is central for utility investment planning but continues to rely on deterministic or ad hoc scenario selection. We propose a highly efficient search framework based on multi-objective Bayesian Optimization. We treat underlying grid stress metrics as computationally expensive black-box functions, approximated via Gaussian Process surrogates and design an acquisition function based on probability of scenarios being Pareto-critical across a collection of line- and bus-based violation objectives. Our approach provides a statistical guarantee and offers an order of magnitude speed-up relative to a conservative exhaustive search. Case studies on realistic feeders with 200-400 buses demonstrate the effectiveness and accuracy of our approach.

cs.LG↗

arXiv2024

Probabilistic Spatiotemporal Modeling of Day-Ahead Wind Power Generation with Input-Warped Gaussian Processes

We design a Gaussian Process (GP) spatiotemporal model to capture features of day-ahead wind power forecasts. We work with hourly-scale day-ahead forecasts across hundreds of wind farm locations, with the main aim of constructing a fully probabilistic joint model across space and hours of the day. To this end, we design a separable space-time kernel, implementing both temporal and spatial input warping to capture the non-stationarity in the covariance of wind power. We conduct synthetic experiments to validate our choice of the spatial kernel and to demonstrate the effectiveness of warping in addressing nonstationarity. The second half of the paper is devoted to a detailed case study using a realistic, fully calibrated dataset representing wind farms in the ERCOT region of Texas.

cs.LG↗

arXiv2023

Least-Cost Structuring of 24/7 Carbon-Free Electricity Procurements

We consider the construction of renewable portfolios targeting specified carbon-free (CFE) hourly performance scores. We work in a probabilistic framework that uses a collection of simulation scenarios and imposes probability constraints on achieving the desired CFE score. In our approach there is a fixed set of available CFE generators and a given load customer who seeks to minimize annual procurement costs. We illustrate results using a realistic dataset of jointly calibrated solar and wind assets, and compare different approaches to handling multiple loads.

q-fin.PM↗

arXiv2023

Analyzing State-Level Longevity Trends with the U.S. Mortality Database

We investigate state-level age-specific mortality trends based on the United States Mortality Database (USMDB) published by the Human Mortality Database. In tandem with looking at the longevity experience across the 51 states, we also consider a collection of socio-demographic, economic and educational covariates that correlate with mortality trends. To obtain smoothed mortality surfaces for each state, we implement the machine learning framework of Multi-Output Gaussian Process regression (Huynh & Ludkovski 2021) on targeted groupings of 3-6 states. Our detailed exploratory analysis shows that the mortality experience is highly inhomogeneous across states in terms of respective Age structures. We moreover document multiple divergent trends between best and worst states, between Females and Males, and between younger and older Ages. The comparisons across the 50+ fitted models offer opportunities for rich insights about drivers of mortality in the U.S. and are visualized through numerous figures and an online interactive dashboard.

stat.AP↗

arXiv2023

Expressive Mortality Models through Gaussian Process Kernels

We develop a flexible Gaussian Process (GP) framework for learning the covariance structure of Age- and Year-specific mortality surfaces. Utilizing the additive and multiplicative structure of GP kernels, we design a genetic programming algorithm to search for the most expressive kernel for a given population. Our compositional search builds off the Age-Period-Cohort (APC) paradigm to construct a covariance prior best matching the spatio-temporal dynamics of a mortality dataset. We apply the resulting genetic algorithm (GA) on synthetic case studies to validate the ability of the GA to recover APC structure, and on real-life national-level datasets from the Human Mortality Database. Our machine-learning based analysis provides novel insight into the presence/absence of Cohort effects in different populations, and into the relative smoothness of mortality surfaces along the Age and Year dimensions. Our modelling work is done with the PyTorch libraries in Python and provides an in-depth investigation of employing GA to aid in compositional kernel search for GP surrogates.

stat.ML↗

arXiv2022

mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms

We introduce mlOSP, a computational template for Machine Learning for Optimal Stopping Problems. The template is implemented in the R statistical environment and publicly available via a GitHub repository. mlOSP presents a unified numerical implementation of Regression Monte Carlo (RMC) approaches to optimal stopping, providing a state-of-the-art, open-source, reproducible and transparent platform. Highlighting its modular nature, we present multiple novel variants of RMC algorithms, especially in terms of constructing simulation designs for training the regressors, as well as in terms of machine learning regression modules. Furthermore, mlOSP nests most of the existing RMC schemes, allowing for a consistent and verifiable benchmarking of extant algorithms. The article contains extensive R code snippets and figures, and serves as a vignette to the underlying software package.

q-fin.CP↗

arXiv2022

Large-scale local surrogate modeling of stochastic simulation experiments

Gaussian process (GP) surrogate modeling for large computer experiments is limited by cubic runtimes, especially with data from stochastic simulations with input-dependent noise. A popular workaround to reduce computational complexity involves local approximation (e.g., LAGP). However, LAGP has only been vetted in deterministic settings. A recent variation utilizing inducing points (LIGP) for additional sparsity improves upon LAGP on the speed-vs-accuracy frontier. The authors show that another benefit of LIGP over LAGP is that (local) nugget estimation for stochastic responses is more natural, especially when designs contain substantial replication as is common when attempting to separate signal from noise. Woodbury identities, extended in LIGP from inducing points to replicates, afford efficient computation in terms of unique design locations only. This increases the amount of local data (i.e., the neighborhood size) that may be incorporated without additional flops, thereby enhancing statistical efficiency. Performance of the authors' LIGP upgrades is illustrated on benchmark data and real-world stochastic simulation experiments, including an options pricing control framework. Results indicatethat LIGP provides more accurate prediction and uncertainty quantification for varying data dimension and replication strategies versus modern alternatives.

stat.ME↗

arXiv2022

Large Scale Probabilistic Simulation of Renewables Production

We develop a probabilistic framework for joint simulation of short-term electricity generation from renewable assets. In this paper we describe a method for producing hourly day-ahead scenarios of generated power at grid-scale across hundreds of assets. These scenarios are conditional on specified forecasts and yield a full uncertainty quantification both at the marginal asset-level and across asset collections. Our simulation pipeline first applies asset calibration to normalize hourly, daily and seasonal generation profiles, and to Gaussianize the forecast--actuals distribution. We then develop a novel clustering approach to stably estimate the covariance matrix across assets; clustering is done hierarchically to achieve scalability. An extended case study using an ERCOT-like system with nearly 500 solar and wind farms is used for illustration.

q-fin.ST↗

arXiv2022

On Parametric Optimal Execution and Machine Learning Surrogates

We investigate optimal order execution problems in discrete time with instantaneous price impact and stochastic resilience. First, in the setting of linear transient price impact we derive a closed-form recursion for the optimal strategy, extending the deterministic results from Obizhaeva and Wang (J Financial Markets, 2013). Second, we develop a numerical algorithm based on dynamic programming and deep learning for the case of nonlinear transient price impact as proposed by Bouchaud et al. (Quant. Finance, 2004). Specifically, we utilize an actor-critic framework that constructs two neural-network (NN) surrogates for the value function and the feedback control. The flexible scalability of NN functional approximators enables parametric learning, i.e., incorporating several model or market parameters as part of the input space. Precise calibration of price impact, resilience, etc., is known to be extremely challenging and hence it is critical to understand sensitivity of the execution policy to these parameters. Our NN learner organically scales across multiple input dimensions and is shown to accurately approximate optimal strategies across a wide range of parameter configurations. We provide a fully reproducible Jupyter Notebook with our NN implementation, which is of independent pedagogical interest, demonstrating the ease of use of NN surrogates in (parametric) stochastic control problems.

q-fin.TR↗

arXiv2022

Regression Monte Carlo for Impulse Control

I develop a numerical algorithm for stochastic impulse control in the spirit of Regression Monte Carlo for optimal stopping. The approach consists in generating statistical surrogates (aka functional approximators) for the continuation function. The surrogates are recursively trained by empirical regression over simulated state trajectories. In parallel, the same surrogates are used to learn the intervention function characterizing the optimal impulse amounts. I discuss appropriate surrogate types for this task, as well as the choice of training sets. Case studies from forest rotation and irreversible investment illustrate the numerical scheme and highlight its flexibility and extensibility. Implementation in \texttt{R} is provided as a publicly available package posted on GitHub.

q-fin.CP↗

arXiv2022

KrigHedge: Gaussian Process Surrogates for Delta Hedging

We investigate a machine learning approach to option Greeks approximation based on Gaussian process (GP) surrogates. The method takes in noisily observed option prices, fits a nonparametric input-output map and then analytically differentiates the latter to obtain the various price sensitivities. Our motivation is to compute Greeks in cases where direct computation is expensive, such as in local volatility models, or can only ever be done approximately. We provide a detailed analysis of numerous aspects of GP surrogates, including choice of kernel family, simulation design, choice of trend function and impact of noise. We further discuss the application to Delta hedging, including a new Lemma that relates quality of the Delta approximation to discrete-time hedging loss. Results are illustrated with two extensive case studies that consider estimation of Delta, Theta and Gamma and benchmark approximation quality and uncertainty quantification using a variety of statistical metrics. Among our key take-aways are the recommendation to use Matern kernels, the benefit of including virtual training points to capture boundary conditions, and the significant loss of fidelity when training on stock-path-based datasets.

q-fin.CP↗

arXiv2021

Joint Models for Cause-of-Death Mortality in Multiple Populations

We investigate jointly modeling Age-specific rates of various causes of death in a multinational setting. We apply Multi-Output Gaussian Processes (MOGP), a spatial machine learning method, to smooth and extrapolate multiple cause-of-death mortality rates across several countries and both genders. To maintain flexibility and scalability, we investigate MOGPs with Kronecker-structured kernels and latent factors. In particular, we develop a custom multi-level MOGP that leverages the gridded structure of mortality tables to efficiently capture heterogeneity and dependence across different factor inputs. Results are illustrated with datasets from the Human Cause-of-Death Database (HCD). We discuss a case study involving cancer variations in three European nations, and a US-based study that considers eight top-level causes and includes comparison to all-cause analysis. Our models provide insights into the commonality of cause-specific mortality trends and demonstrate the opportunities for respective data fusion.

stat.AP↗

arXiv2021

Adaptive Batching for Gaussian Process Surrogates with Application in Noisy Level Set Estimation

We develop adaptive replicated designs for Gaussian process metamodels of stochastic experiments. Adaptive batching is a natural extension of sequential design heuristics with the benefit of replication growing as response features are learned, inputs concentrate, and the metamodeling overhead rises. Motivated by the problem of learning the level set of the mean simulator response we develop four novel schemes: Multi-Level Batching (MLB), Ratchet Batching (RB), Adaptive Batched Stepwise Uncertainty Reduction (ABSUR), Adaptive Design with Stepwise Allocation (ADSA) and Deterministic Design with Stepwise Allocation (DDSA). Our algorithms simultaneously (MLB, RB and ABSUR) or sequentially (ADSA and DDSA) determine the sequential design inputs and the respective number of replicates. Illustrations using synthetic examples and an application in quantitative finance (Bermudan option pricing via Regression Monte Carlo) show that adaptive batching brings significant computational speed-ups with minimal loss of modeling fidelity.

stat.ML↗