Searcharxiv⌕ Search

arXiv subjects

Aarush Kulkarni

Publications and source records attributed to Aarush Kulkarni.

2 recordsLinked to original sources

Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal $\sqrt S$ and $\sqrt K$ Dependencies in Tabular and Linear AMDPs

We study regret minimization in episodic Adversarial Markov Decision Processes (AMDPs) with unknown transitions and bandit feedback. In tabular AMDPs with $S$ states, $A$ actions, and horizon $H$, the best known $K$-episode regret is $\tilde O(HS\sqrt{AK})$, whose $\sqrt S$ gap to the $Ω(H\sqrt{SAK})$ lower bound has long remained open. In linear AMDPs with $d$-dimensional features, after a five-year line of improvements starting from $K^{14/15}$, computationally efficient algorithms without a simulator or exploratory assumptions still only attain $\tilde O(K^{3/4})$ regret. A common challenge in both settings is to estimate the Q-function of a changing policy: past data comes from earlier policies that differ from (and are correlated with) the current one, while freezing the policy for future episodes worsens the dependency on $K$. We instead propose a general technique called introduced feedback delays: we estimate the Q-function of the policy using future episodes while still updating the policy, as if the feedback were delayed. While feedback delays are usually an obstacle imposed by the environment, here they are introduced voluntarily and, perhaps surprisingly, come at no extra cost. We develop Geometric Resampling with Rejection (GRR) to account for policy changes during the delay, and threshold the visitation probabilities to avoid long delays on rarely visited state-action pairs. With this unified technique, our policy optimization algorithms achieve, with high probability, $\tilde O(\text{poly}(H)\sqrt{SAK})$ regret in tabular AMDPs, resolving the open problem; and $\tilde O(\text{poly}(d,H)\sqrt K)$ regret in linear AMDPs, giving the first efficient algorithm optimal in $K$. They also naturally extend to delayed AMDPs with a total feedback delay $\mathfrak D$, yielding the first bounds optimal in $S$ and $\mathfrak D$ (tabular) and in $K$ and $\mathfrak D$ (linear).

cs.LG↗

New interpretations for Kromatic symmetric function expansions

The Kromatic symmetric function (KSF) $\overline{X}_{G}$, introduced by Crew, Pechenik, and Spirkl (2026), is a $K$-theoretic analogue of the chromatic symmetric function (CSF) $X_G$. We study expansion formulas for the KSF in two different bases. First, we explore recursive ways to compute the KSF's expansion in the $K$-theoretic monomial symmetric function basis $\overline{\widetilde{m}}_λ$, generalizing formulas that were used by the second author and Samanta to show that the KSF distinguishes certain families of graphs that are not distinguished by the CSF. Second, we give new interpretations for the KSF's expansion in the $K$-theoretic power sum basis $\overline{p}_λ$, using an inclusion-exclusion approach like the one from Stanley (1995) instead of an acyclic orientation approach. Finally, we give $K$-analogues of Schmitt's and of Humpert and Martin's antipode formulas for the Hopf algebra of graphs, along with a Hopf algebra interpretation for both the $\overline{p}$-expansion and $\overline{\widetilde{m}}$-expansion of the KSF.

math.CO↗