arXiv subjects

Kushagra Chandak

Publications and source records attributed to Kushagra Chandak.

5 recordsLinked to original sources

arXiv2026

Offline-to-Online Learning in Linear Bandits

We study online learning with an additional offline dataset in the stochastic linear bandit setting. Although this problem arises frequently in practice, the offline-to-online tradeoff remains poorly understood in structured environments. We propose a linear bandit algorithm that balances this tradeoff: it relies on offline data during early rounds, and increasingly favors exploration as the horizon grows. We establish regret bounds showing that our method is simultaneously competitive with both purely online and purely offline solutions. In particular, it achieves sublinear regret relative to the optimal action in the number of online interactions, while its regret relative to an offline reference decreases as the number of offline samples grows. Empirical results further demonstrate its effectiveness across various problem parameters.

cs.LG↗

arXiv2025

Context-Action Embedding Learning for Off-Policy Evaluation in Contextual Bandits

We consider off-policy evaluation (OPE) in contextual bandits with finite action space. Inverse Propensity Score (IPS) weighting is a widely used method for OPE due to its unbiased, but it suffers from significant variance when the action space is large or when some parts of the context-action space are underexplored. Recently introduced Marginalized IPS (MIPS) estimators mitigate this issue by leveraging action embeddings. However, these embeddings do not minimize the mean squared error (MSE) of the estimators and do not consider context information. To address these limitations, we introduce Context-Action Embedding Learning for MIPS, or CAEL-MIPS, which learns context-action embeddings from offline data to minimize the MSE of the MIPS estimator. Building on the theoretical analysis of bias and variance of MIPS, we present an MSE-minimizing objective for CAEL-MIPS. In the empirical studies on a synthetic dataset and a real-world dataset, we demonstrate that our estimator outperforms baselines in terms of MSE.

cs.LG↗

arXiv2024

Regret Minimization via Saddle Point Optimization

A long line of works characterizes the sample complexity of regret minimization in sequential decision-making by min-max programs. In the corresponding saddle-point game, the min-player optimizes the sampling distribution against an adversarial max-player that chooses confusing models leading to large regret. The most recent instantiation of this idea is the decision-estimation coefficient (DEC), which was shown to provide nearly tight lower and upper bounds on the worst-case expected regret in structured bandits and reinforcement learning. By re-parametrizing the offset DEC with the confidence radius and solving the corresponding min-max program, we derive an anytime variant of the Estimation-To-Decisions (E2D) algorithm. Importantly, the algorithm optimizes the exploration-exploitation trade-off online instead of via the analysis. Our formulation leads to a practical algorithm for finite model classes and linear feedback models. We further point out connections to the information ratio, decoupling coefficient and PAC-DEC, and numerically evaluate the performance of E2D on simple examples.

cs.LG↗

arXiv2020

Combining Propositional Logic Based Decision Diagrams with Decision Making in Urban Systems

Solving multiagent problems can be an uphill task due to uncertainty in the environment, partial observability, and scalability of the problem at hand. Especially in an urban setting, there are more challenges since we also need to maintain safety for all users while minimizing congestion of the agents as well as their travel times. To this end, we tackle the problem of multiagent pathfinding under uncertainty and partial observability where the agents are tasked to move from their starting points to ending points while also satisfying some constraints, e.g., low congestion, and model it as a multiagent reinforcement learning problem. We compile the domain constraints using propositional logic and integrate them with the RL algorithms to enable fast simulation for RL.

cs.AI↗

arXiv2019

Hunting scalar leptoquarks with boosted tops and light leptons

The LHC search strategies for leptoquarks that couple dominantly to a top quark are different than for the ones that couple mostly to the light quarks. We consider charge $1/3$ ($ϕ_1$) and $5/3$ ($ϕ_5$) scalar leptoquarks that can decay to a top quark and a charged lepton ($t\ell$) giving rise to a resonance system of a boosted top and a high-$p_{\rm T}$ lepton. We introduce simple phenomenological models suitable for bottom-up studies and explicitly map them to all possible scalar leptoquark models within the Buchmüller-Rückl-Wyler classifications that can have the desired decays. We study pair and single productions of these leptoquarks. Contrary to the common perception, we find that the single production of top-philic leptoquarks $ϕ= \{ϕ_1,ϕ_5\}$ in association with a lepton and jets could be significant for order one $ϕt\ell$ coupling in certain scenarios. We propose a strategy of selecting events with at least one hadronic-top and two high-$p_{\rm T}$ same flavour opposite sign leptons. This captures events from both pair and single productions. Our strategy can significantly enhance the LHC discovery potential especially in the high-mass region where single productions become more prominent. Our estimation shows that a scalar leptoquark as heavy as $\sim1.7$ TeV can be discovered at the $14$ TeV LHC with 3 ab$^{-1}$ of integrated luminosity in the $t\ell\ell+X$ channel for $100\%$ branching ratio in the $ϕ\to t\ell $ decay mode. However, in some scenarios, the discovery reach can increase beyond $2$ TeV even though the branching ratio comes down to about $50\%$.

hep-ph↗