Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2611.01216

Adaptive Optimizers Can Get a Policy Ranking Wrong: Momentum, Dependent Data, and a Two-Run Remedy

Sunyoung An·Xiaoming Huo
View original source ↗Read source document ↗Export metadata ↓

Abstract

Operators choosing among decision rules, such as when a battery charges or where an idle taxi goes, rank them by estimated long-run value. A standard estimator, linear temporal-difference (TD) learning, fits values from one stream of correlated observations. Large fits often use an adaptive optimizer, such as adaptive moment estimation (Adam), with fixed averaging rates. We show that these rates can change what is estimated, not just how fast: the same observations drive the updates and the optimizer's step-size scaling, moving the fit away from the intended TD value. The shift differs across rules, so it need not cancel in a comparison, and more data only makes a wrong ranking look more certain. Our remedy refits the same data at a second averaging rate that halves the effective gain, the number to which the leading shift is proportional. Twice the new fit minus the original removes that shift. In New York taxi and battery models calibrated to real data, common optimizer settings leave the selected rule unchanged but order close rules wrongly and misstate its value. Our remedy corrects nearly every wrong ordering, with one extra fit when the shift is moderate and two when it is larger.

Explore related subjects

stat.MLcs.LGmath.OCmath.STstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Sunyoung An, Xiaoming Huo. 2026-10-31. Adaptive Optimizers Can Get a Policy Ranking Wrong: Momentum, Dependent Data, and a Two-Run Remedy. https://arxiv.org/abs/2611.01216

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Recovering Geodesic Distances from Noisy Observations

Let $(M,d,μ)$ be a nice geodesic probability space (such as a manifold) and $X=\{x_i\}_{i\in [N]} \subset M$ a finite $μ$-i.i.d. sample. Suppose we observe noisy distance random variables $d'(x_i,x_j)$. When can the true distances be recovered to small additive error? This question has attracted interest in two disparate fields: latent-space graph models and boundary-value inverse problems. We first show that, taking $N \asymp ε^{-2d-2}|\logε|$ and using runtime $O(N^{2.5+o_d (1)})$, we can recover all distances in $X$ to additive error $O(ε|\logε|)$ with high probability, assuming $\mathbb{E} d'(x_i, x_j) = F(d(x_i,x_j))$ for an unknown, increasing bilipschitz $F$ and with the $d'(x_i,x_j)$ allowed to be arbitrarily heterogeneous in their distribution as long as they are bounded in subgaussian norm. This generalizes existing work, which required $F$ to be known and made strict assumptions on the distribution of $d'(x_i, x_j)$. Achieving this generalization required a novel ``ruler-building'' procedure to learn $F$. We extend our results to a missing data case where we randomly lose access to noisy distances. Our second result uses ideas from the first to tackle a partial observability case: $X = Y \cup Z$ with $Z$ sampled from $M$ and $Y$ sampled from an observer subset satisfying visibility conditions that model the assumptions made on the boundary in boundary-value inverse problems. In this setting, we recover all pairwise distances in $Z$ to additive error $O(ε)$ with $|Z| \asymp ε^{-2d-2}|\logε|$, $|Y| \asymp ε^{-\max\{d_ν,4\}}$ with $d_ν$ determined by the volume-growth properties of the probability measure on the observer set, and runtime $O(|Z|^3)$. This is, to our knowledge, the first quantitative guarantee for this observer/observed split under noise, which complements existing stability results for boundary data.

stat.ML↗
arXiv2026

Emergence of Distortions in High-Dimensional Guided Diffusion Models

Classifier-free guidance (CFG) is the de facto standard for conditional sampling in diffusion models, yet it often reduces sample diversity. Using tools from statistical physics, we analyze the emergence of generative distortions induced by CFG, namely the mismatch between the CFG sampling distribution and the true conditional distribution. We study this phenomenon in analytically tractable settings with exact score functions, characterizing its dependence on data dimensionality and the number of classes. For high-dimensional Gaussian mixtures, we use dynamic mean-field theory to show that distortions persist when the number of classes scales exponentially with the data dimension, whereas they vanish in the sub-exponential regime due to a dynamical phase transition. Moreover we found that, in the infinite-class limit, distortions remain unavoidable regardless of dimensionality because of the increasing density of classes. Finally, we show that standard CFG schedules cannot prevent variance shrinkage, and we propose a theoretically grounded guidance schedule incorporating a negative-guidance window that improves both class separability and sample diversity in real-world latent diffusion models.

stat.ML↗
arXiv2026

Deep Learning of Compositional Targets with Hierarchical Spectral Methods

Why depth yields a computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional Gaussian setting, focusing on compositional target functions. We analyze their learnability using an explicit three-layer fitting model trained via layer-wise spectral estimators. The target combines polynomial latent features with a scalar readout, and its compositional structure allows learning to proceed in stages. Recovering an intermediate representation reduces the problem to lower-degree spectral estimation tasks, as in multi-index models. Our analysis establishes Gaussian comparison bounds for projected moments when $0<\varepsilon<1/3$, and motivates separations in sample complexity between two- and three-layer learning strategies, conditional on a conjectured spectral recovery step.

stat.ML↗