Searcharxiv⌕ Search

arXiv · 2611.14000

DynamAuction: A Reinforcement Learning Environment for Repeated Auctions with Dynamic Value

Abstract

A buyer who repeatedly participates in auctions for related items often faces a dynamic value problem: what they win today changes what tomorrow's items are worth. Despite the empirical importance of such effects, especially in display advertising, the algorithmic literature on repeated auctions has largely focused on the learning problem, by treating each opportunity as a contextual bandit, while sidestepping the planning problem induced by the dependence of current value on past wins. We introduce DynamAuction, a synthetic environment for single-buyer sequential bidding. The reward from each win depends on the buyer's history of previous wins, summarized through a carry-over state. The environment supports first- and second-price auctions, non-homogeneous Poisson arrivals, delayed feedback, saturation, and recency effects. We also introduce a new class of Model Predictive Control (MPC) algorithms. On our benchmark scenario, these MPC policies outperform off-the-shelf deep RL baselines (PPO, SAC, TD3) as well as a tuned shading baseline. We release DynamAuction to support RL research on repeated auctions.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Eugénie Patard, Corentin Pla, Benjamin Heymann, Patrick Loiseau. 2026-11-10. DynamAuction: A Reinforcement Learning Environment for Repeated Auctions with Dynamic Value. https://searcharxiv.com/abs/2611.14000

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Bayesian Framework for Latent Compliance Modeling in Cluster Randomized Trials with One-Sided Noncompliance

In pragmatic cluster randomized controlled trials (PCRCTs), healthcare providers are randomized, but intervention delivery and patient receipt may vary after assignment. In many PCRCTs, cluster-level implementation is measured using continuous implementation metrics, while individual compliance is recorded as a binary indicator. Standard complier average causal effect (CACE) estimands focus on individual-level compliance and do not account for heterogeneity in implementation across clusters. When intervention uptake is shaped by both cluster- and individual-level processes, it is of scientific interest to characterize how effects vary across these sources of compliance. We propose a Bayesian framework for PCRCTs with one-sided binary noncompliance at the individual level and one-sided partial compliance at the cluster level. The method uses a latent mixture model to summarize heterogeneity in cluster-level implementation based on baseline characteristics and treatment-arm implementation measures, and links these latent implementation strata to individual compliance and outcomes through a joint model. Because individual compliance and treatment-defined implementation measures are observed only under treatment assignment, the model imputes these quantities for control clusters and individuals assigned to control. The framework enables estimation of finite- and super-population intent-to-treat (ITT) and CACE estimands, both marginally and within latent implementation strata. We apply the method to the METRIcAL trial, a pragmatic cluster randomized study evaluating a personalized music intervention for nursing home residents with dementia. The analysis illustrates how accounting for implementation heterogeneity and individual compliance can provide insights beyond standard ITT analyses.

stat.AP↗

Copula-Based Synthetic Data Generation: A Use-Inspired Perspective on Trade-offs

The generation of synthetic data offers a promising way to address challenges related to data scarcity and privacy concerns. While many approaches have been developed, tabular synthetic data generation methods often require complex models and extensive training data, while lacking interpretability. In this work, we investigate versatile probabilistic approaches using copulas, and leverage in particular the Gaussian Mixture Copula Model, to model the joint distribution of multivariate data, with a main focus on moderate-dimensional data sets. Taking a use-inspired perspective, we examine key trade-offs inherent to synthetic data evaluation, spanning fidelity, utility, and privacy, and demonstrate that copula-based methods achieve a sound balance across all three dimensions while remaining interpretable and providing an explicit probabilistic model. We illustrate and highlight the potential benefits of the method on a test case focusing on systematic missing values in a cardiovascular risk calculator and develop specific evaluation metrics for this application.

stat.AP↗

JALE: A new Python toolbox for coordinate-based neuroimaging meta-analysis using activation likelihood estimation

Coordinate-based neuroimaging meta-analysis using activation likelihood estimation (ALE) is commonly used to synthesize reported results of functional or structural magnetic resonance imaging studies. We here present JALE, a new, open-source Python package for conducting ALE meta-analyses. Besides computing standard ALE main effects and contrast analyses, this toolbox implements several novel analytical approaches including new options for thresholding (threshold-free cluster enhancement statistics, significance threshold prediction) as well as sensitivity and cluster analyses that were only available as in-house scripts in the past. The purpose of this paper is to provide an integrated conceptual foundation of recent developments in the ALE meta-analysis framework, with a particular focus on novel methodological implementations that can be executed in a straightforward and standardized manner using the new JALE toolbox.

stat.AP↗