Dynamic Regret for Non-Stationary Linear Bandits via Misspecification Reductions
Many online decision-making problems involve both round-specific feasible actions and drifting reward models: eligible ad impressions, feasible prices, and available treatments can change over time, while user preferences, demand curves, and patient responses may evolve. Motivated by these applications, we study non-stationary linear bandits with round-specific feasible decision sets. Our key contribution is a blockwise reduction that connects parameter drift in non-stationary linear bandits to model misspecification in contextual linear bandits while preserving the round-specific decision benchmark. This connection allows us to leverage misspecification-adaptive learning to obtain dynamic-regret guarantees for general time-varying decision sets. For a known path-length budget $P_T$, we establish these guarantees for general compact, adaptive non-anticipating decision sets under an oblivious parameter path, and for $K$-armed contextual linear bandits under an oblivious environment. For general time-varying decision sets, our bounds improve the dependence on $T$ and $P_T$ from the $\widetilde O(T^{3/4}P_T^{1/4})$ guarantees available for existing LinUCB-based approaches to $\widetilde O(T^{2/3}P_T^{1/3})$, matching the non-stationary lower-bound dependence up to logarithmic factors.