arXiv subjects

Vineet Goyal

Publications and source records attributed to Vineet Goyal.

At least 19 recordsLinked to original sources

arXiv2026

A Consistency-Robustness Framework for Robust Optimization: Integrating Predictions into Robust Scheduling

Robust optimization protects against uncertainty by optimizing for the worst case over a prescribed uncertainty set. This protection can be overly conservative when forecasts, historical data, or learned predictions indicate a more likely scenario. We introduce a framework for robust optimization with predictions. The input consists of an uncertainty set together with a distinguished predicted scenario, and the goal is to compute a single solution that is both consistent, meaning near-optimal for the predicted scenario, and robust, meaning competitive with the classical min-max robust optimum. Unlike in standard learning-augmented algorithms, the prediction does not merely estimate the realized input; it creates a separate benchmark, the predicted optimum, which must be balanced against the min-max robust optimum. We study this framework for makespan scheduling with uncertain processing times and give a structural classification across standard uncertainty models and machine environments. For interval uncertainty, we obtain a smooth $(1+1/\lambda,1+\lambda)$ consistency-robustness tradeoff for restricted-assignment and related machines. Furthermore, we prove that unrelated machines admit no constant tradeoff. For budgeted uncertainty, we obtain a $(1+1/\lambda,2+\lambda)$ tradeoff for restricted assignment. Our analysis is based on a duality-based reduction to an interval-like upper envelope. We complement this with a lower bound showing that related machines admit no constant tradeoff even when only one job may deviate. For arbitrary uncertainty sets, we obtain constant tradeoffs for identical machines via a support-function block construction, and prove impossibility for restricted assignment. Our results show that the possibility of combining consistency and robustness in robust scheduling depends critically on the interaction between the uncertainty model and the machine environment.

cs.DS↗

arXiv2026

Which Pairs to Compare for LLM Post-Training?

Preference-based post-training has become a central paradigm for aligning language models. A common data-collection strategy is to generate a small set of completions for each prompt and label the resulting comparison pairs. However, human preference labels are often much more expensive than generating additional completions, suggesting a different use of the same labeling budget: generate a larger pool of completions, but label only the most informative comparison pairs. This paper studies which pairs should be compared in preference-based post-training. We formulate comparison curation as a sampling-design problem and evaluate designs by the quality of the final policy under the preference-based post-training objective. We instantiate this framework for Direct Preference Optimization (DPO), analyzing how the choice of labeled pairs propagates through DPO training to downstream policy performance. Our main results provide matching upper and lower bounds on the post-training optimality gap of the DPO-trained policy. The bounds show that comparison selection affects downstream performance through a single design-dependent information matrix, which links label allocation to parameter estimation error and policy suboptimality. This yields an explicit optimization criterion for budgeted comparison curation and motivates practical sampling designs for selecting informative pairs from large generated completion pools. Experiments on synthetic settings and language-model post-training benchmarks show that the proposed designs consistently improve sample efficiency over common comparison-selection heuristics.

cs.AI↗

arXiv2026

PREFER: Personalized Review Summarization with Online Preference Learning

Product reviews significantly influence purchasing decisions on e-commerce platforms. However, the sheer volume of reviews can overwhelm users, obscuring the information most relevant to their specific needs. Current e-commerce summarization systems typically produce generic, static summaries that fail to account for the fact that (i) different users care about different product characteristics, and (ii) these preferences may evolve with interactions. To address the challenge of unknown latent preferences, we propose an online learning framework that generates personalized summaries for each user. Our system iteratively refines its understanding of user preferences by incorporating feedback directly from the generated summaries over time. We provide a case study using the Amazon Reviews'23 dataset, showing in controlled simulations that online preference learning improves alignment with target user interests while maintaining summary quality.

cs.AI↗

arXiv2025

Collaborative Min-Max Regret in Grouped Multi-Armed Bandits

We study the impact of sharing exploration in multi-armed bandits in a grouped setting where a set of groups have overlapping feasible action sets [Baek and Farias '24]. In this grouped bandit setting, groups share reward observations, and the objective is to minimize the collaborative regret, defined as the maximum regret across groups. This naturally captures applications in which one aims to balance the exploration burden between groups or populations -- it is known that standard algorithms can lead to significantly imbalanced exploration cost between groups. We address this problem by introducing an algorithm Col-UCB that dynamically coordinates exploration across groups. We show that Col-UCB achieves both optimal minimax and instance-dependent collaborative regret up to logarithmic factors. These bounds are adaptive to the structure of shared action sets between groups, providing insights into when collaboration yields significant benefits over each group learning their best action independently.

cs.LG↗

arXiv2024

Distributionally Robust Newsvendor on a Metric

We consider a fundamental generalization of the classical newsvendor problem where the seller needs to decide on the inventory of a product jointly for multiple locations on a metric as well as a fulfillment policy to satisfy the uncertain demand that arises sequentially over time after the inventory decisions have been made. To address the distributional ambiguity, we consider a distributionally robust setting where the decision-maker only knows the mean and variance of the demand, and the goal is to make inventory and fulfillment decisions to minimize the worst-case expected inventory and fulfillment cost. We design a near-optimal policy for the problem with theoretical guarantees on its performance. Our policy generalizes the classical solution of Scarf (1957), maintaining its simplicity and interpretability: it identifies a hierarchical set of clusters, assigns a ``virtual" underage cost to each cluster, then makes sure that each cluster holds at least the inventory suggested by Scarf's solution if the cluster behaved as a single point with ``virtual" underage cost. As demand arrives sequentially, our policy fulfills orders from nearby clusters, minimizing fulfilment costs, while balancing inventory consumption across the clusters to avoid depleting any single one. We show that the policy achieves a poly-logarithmic approximation. To the best of our knowledge, this is the first algorithm with provable performance guarantees. Furthermore, our numerical experiments show that the policy performs well in practice.

math.OC↗

arXiv2024

Asymptotically Optimal Competitive Ratio for Online Allocation of Reusable Resources

We consider the problem of online allocation (matching, budgeted allocations, and assortments) of reusable resources where an adversarial sequence of resource requests is revealed over time and any allocated resource is used/rented for a stochastic duration drawn independently from a resource dependent usage distribution. Previously, it was known that a greedy algorithm is 0.5--competitive against the clairvoyant benchmark that knows the entire sequence of requests in advance (Gong et al. (2021)). We give a novel algorithm that is $(1-1/e)$--competitive for arbitrary usage distributions when the starting capacity of each resource is large and the usage distributions are known. This is the best achievable competitive ratio guarantee for the problem, i.e., no online algorithm can have better competitive ratio. We also give a distribution oblivious online algorithm and show that it is $(1-1/e)$--competitive in special cases. At the heart of our algorithms is a new quantity that factors in the potential of reusability for each resource by (computationally) creating an asymmetry between identical units of the resource. We establish the performance guarantee for our algorithms by constructing a feasible solution to a novel system of inequalities that allows direct comparison with the clairvoyant benchmark instead of a linear programming (LP) relaxation of the benchmark. Our technique generalizes the primal-dual analysis framework for online resource allocation and may be of broader interest.

cs.DS↗

arXiv2024

MNL-Bandit with Knapsacks: a near-optimal algorithm

We consider a dynamic assortment selection problem where a seller has a fixed inventory of $N$ substitutable products and faces an unknown demand that arrives sequentially over $T$ periods. In each period, the seller needs to decide on the assortment of products (satisfying certain constraints) to offer to the customers. The customer's response follows an unknown multinomial logit model (MNL) with parameter $\boldsymbol{v}$. If customer selects product $i \in [N]$, the seller receives revenue $r_i$. The goal of the seller is to maximize the total expected revenue from the $T$ customers given the fixed initial inventory of $N$ products. We present MNLwK-UCB, a UCB-based algorithm and characterize its regret under different regimes of inventory size. We show that when the inventory size grows quasi-linearly in time, MNLwK-UCB achieves a $\tilde{O}(N + \sqrt{NT})$ regret bound. We also show that for a smaller inventory (with growth $\sim T^α$, $α< 1$), MNLwK-UCB achieves a $\tilde{O}(N(1 + T^{\frac{1 - α}{2}}) + \sqrt{NT})$. In particular, over a long time horizon $T$, the rate $\tilde{O}(\sqrt{NT})$ is always achieved regardless of the constraints and the size of the inventory.

cs.LG↗

arXiv2023

MNL-Prophet: Sequential Assortment Selection under Uncertainty

Due to numerous applications in retail and (online) advertising the problem of assortment selection has been widely studied under many combinations of discrete choice models and feasibility constraints. In many situations, however, an assortment of products has to be constructed gradually and without accurate knowledge of all possible alternatives; in such cases, existing offline approaches become inapplicable. We consider a stochastic variant of the assortment selection problem, where the parameters that determine the revenue and (relative) demand of each item are jointly drawn from some known item-specific distribution. The items are observed sequentially in an arbitrary and unknown order; upon observing the realized parameters of each item, the decision-maker decides irrevocably whether to include it in the constructed assortment, or forfeit it forever. The objective is to maximize the expected total revenue of the constructed assortment, relative to that of an offline algorithm which foresees all the parameter realizations and computes the optimal assortment. We provide simple threshold-based online policies for the unconstrained and cardinality-constrained versions of the problem under a natural class of substitutable choice models; as we show, our policies are (worst-case) optimal under the celebrated Multinomial Logit choice model. We extend our results to the case of knapsack constraints and discuss interesting connections to the Prophet Inequality problem, which is already subsumed by our setting.

cs.DS↗

arXiv2023

MNL-Bandit in non-stationary environments

In this paper, we study the MNL-Bandit problem in a non-stationary environment and present an algorithm with a worst-case expected regret of $\tilde{O}\left( \min \left\{ \sqrt{NTL}\;,\; N^{\frac{1}{3}}(Δ_{\infty}^{K})^{\frac{1}{3}} T^{\frac{2}{3}} + \sqrt{NT}\right\}\right)$. Here $N$ is the number of arms, $L$ is the number of changes and $Δ_{\infty}^{K}$ is a variation measure of the unknown parameters. Furthermore, we show matching lower bounds on the expected regret (up to logarithmic factors), implying that our algorithm is optimal. Our approach builds upon the epoch-based algorithm for stationary MNL-Bandit in Agrawal et al. 2016. However, non-stationarity poses several challenges and we introduce new techniques and ideas to address these. In particular, we give a tight characterization for the bias introduced in the estimators due to non stationarity and derive new concentration bounds.

cs.LG↗

arXiv2023

LP-based Approximations for Disjoint Bilinear and Two-Stage Adjustable Robust Optimization

We consider the class of disjoint bilinear programs $ \max \, \{ \mathbf{x}^T\mathbf{y} \mid \mathbf{x} \in \mathcal{X}, \;\mathbf{y} \in \mathcal{Y}\}$ where $\mathcal{X}$ and $\mathcal{Y}$ are packing polytopes. We present an $O(\frac{\log \log m_1}{\log m_1} \frac{\log \log m_2}{\log m_2})$-approximation algorithm for this problem where $m_1$ and $m_2$ are the number of packing constraints in $\mathcal{X}$ and $\mathcal{Y}$ respectively. In particular, we show that there exists a near-optimal solution $(\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$ such that $\tilde{\mathbf{x}}$ and $\tilde{\mathbf{y}}$ are ``near-integral". We give an LP relaxation of the problem from which we obtain the near-optimal near-integral solution via randomized rounding. We show that our relaxation is tightly related to the widely used reformulation linearization technique (RLT). As an application of our techniques, we present a tight approximation for the two-stage adjustable robust optimization problem with covering constraints and right-hand side uncertainty where the separation problem is a bilinear optimization problem. In particular, based on the ideas above, we give an LP restriction of the two-stage problem that is an $O(\frac{\log n}{\log \log n} \frac{\log L}{\log \log L})$-approximation where $L$ is the number of constraints in the uncertainty set. This significantly improves over state-of-the-art approximation bounds known for this problem. Furthermore, we show that our LP restriction gives a feasible affine policy for the two-stage robust problem with the same (or better) objective value. As a consequence, affine policies give an $O(\frac{\log n}{\log \log n} \frac{\log L}{\log \log L})$-approximation of the two-stage problem, significantly generalizing the previously known bounds on their performance.

math.OC↗

arXiv2023

Last Switch Dependent Bandits with Monotone Payoff Functions

In a recent work, Laforgue et al. introduce the model of last switch dependent (LSD) bandits, in an attempt to capture nonstationary phenomena induced by the interaction between the player and the environment. Examples include satiation, where consecutive plays of the same action lead to decreased performance, or deprivation, where the payoff of an action increases after an interval of inactivity. In this work, we take a step towards understanding the approximability of planning LSD bandits, namely, the (NP-hard) problem of computing an optimal arm-pulling strategy under complete knowledge of the model. In particular, we design the first efficient constant approximation algorithm for the problem and show that, under a natural monotonicity assumption on the payoffs, its approximation guarantee (almost) matches the state-of-the-art for the special and well-studied class of recharging bandits (also known as delay-dependent). In this attempt, we develop new tools and insights for this class of problems, including a novel higher-dimensional relaxation and the technique of mirroring the evolution of virtual states. We believe that these novel elements could potentially be used for approaching richer classes of action-induced nonstationary bandits (e.g., special instances of restless bandits). In the case where the model parameters are initially unknown, we develop an online learning adaptation of our algorithm for which we provide sublinear regret guarantees against its full-information counterpart.

cs.LG↗

arXiv2022

Dynamic pricing and assortment under a contextual MNL demand

We consider dynamic multi-product pricing and assortment problems under an unknown demand over T periods, where in each period, the seller decides on the price for each product or the assortment of products to offer to a customer who chooses according to an unknown Multinomial Logit Model (MNL). Such problems arise in many applications, including online retail and advertising. We propose a randomized dynamic pricing policy based on a variant of the Online Newton Step algorithm (ONS) that achieves a $O(d\sqrt{T}\log(T))$ regret guarantee under an adversarial arrival model. We also present a new optimistic algorithm for the adversarial MNL contextual bandits problem, which achieves a better dependency than the state-of-the-art algorithms in a problem-dependent constant $κ_2$ (potentially exponentially small). Our regret upper bound scales as $\tilde{O}(d\sqrt{κ_2 T}+ \log(T)/κ_2)$, which gives a stronger bound than the existing $\tilde{O}(d\sqrt{T}/κ_2)$ guarantees.

cs.LG↗

arXiv2022

Online Matching with Stochastic Rewards: Optimal Competitive Ratio via Path Based Formulation

The problem of online matching with stochastic rewards is a generalization of the online bipartite matching problem where each edge has a probability of success. When a match is made it succeeds with the probability of the corresponding edge. Introducing this model, Mehta and Panigrahi (FOCS 2012) focused on the special case of identical edge probabilities. Comparing against a deterministic offline LP, they showed that the Ranking algorithm of Karp et al. (STOC 1990) is 0.534 competitive and proposed a new online algorithm with an improved guarantee of $0.567$ for vanishingly small probabilities. For the case of vanishingly small but heterogeneous probabilities Mehta et al. (SODA 2015), gave a 0.534 competitive algorithm against the same LP benchmark. For the more general vertex-weighted version of the problem, to the best of our knowledge, no results being $1/2$ were previously known even for identical probabilities. We focus on the vertex-weighted version and give two improvements. First, we show that a natural generalization of the Perturbed-Greedy algorithm of Aggarwal et al. (SODA 2011), is $(1-1/e)$ competitive when probabilities decompose as a product of two factors, one corresponding to each vertex of the edge. This is the best achievable guarantee as it includes the case of identical probabilities and in particular, the classical online bipartite matching problem. Second, we give a deterministic $0.596$ competitive algorithm for the previously well studied case of fully heterogeneous but vanishingly small edge probabilities. A key contribution of our approach is the use of novel path-based analysis. This allows us to compare against the natural benchmarks of adaptive offline algorithms that know the sequence of arrivals and the edge probabilities in advance, but not the outcomes of potential matches.

cs.DS↗

arXiv2021

Interpretable Machine Learning for Resource Allocation with Application to Ventilator Triage

Rationing of healthcare resources is a challenging decision that policy makers and providers may be forced to make during a pandemic, natural disaster, or mass casualty event. Well-defined guidelines to triage scarce life-saving resources must be designed to promote transparency, trust, and consistency. To facilitate buy-in and use during high-stress situations, these guidelines need to be interpretable and operational. We propose a novel data-driven model to compute interpretable triage guidelines based on policies for Markov Decision Process that can be represented as simple sequences of decision trees ("tree policies"). In particular, we characterize the properties of optimal tree policies and present an algorithm based on dynamic programming recursions to compute good tree policies. We utilize this methodology to obtain simple, novel triage guidelines for ventilator allocations for COVID-19 patients, based on real patient data from Montefiore hospitals. We also compare the performance of our guidelines to the official New York State guidelines that were developed in 2015 (well before the COVID-19 pandemic). Our empirical study shows that the number of excess deaths associated with ventilator shortages could be reduced significantly using our policy. Our work highlights the limitations of the existing official triage guidelines, which need to be adapted specifically to COVID-19 before being successfully deployed.

cs.LG↗

arXiv2021

Robust Markov Decision Process: Beyond Rectangularity

We consider a robust approach to address uncertainty in model parameters in Markov Decision Processes (MDPs), which are widely used to model dynamic optimization in many applications. Most prior works consider the case where the uncertainty on transitions related to different states is uncoupled and the adversary is allowed to select the worst possible realization for each state unrelated to others, potentially leading to highly conservative solutions. On the other hand, the case of general uncertainty sets is known to be intractable. We consider a factor model for probability transitions where the transition probability is a linear function of a factor matrix that is uncertain and belongs to a factor matrix uncertainty set. This is a fairly general model of uncertainty in probability transitions, allowing the decision maker to model dependence between probability transitions across different states and it is significantly less conservative than prior approaches. We show that under an underlying rectangularity assumption, we can efficiently compute an optimal robust policy under the factor matrix uncertainty model. Furthermore, we show that there is an optimal robust policy that is deterministic, which is of interest from an interpretability standpoint. We also introduce the robust counterpart of important structural results of classical MDPs, including the maximum principle and Blackwell optimality, and we provide a computational study to demonstrate the effectiveness of our approach in mitigating the conservativeness of robust policies.

math.OC↗

arXiv2021

A First-Order Approach To Accelerated Value Iteration

Markov decision processes (MDPs) are used to model stochastic systems in many applications. Several efficient algorithms to compute optimal policies have been studied in the literature, including value iteration (VI) and policy iteration. However, these do not scale well especially when the discount factor for the infinite horizon discounted reward, $λ$, gets close to $1$. In particular, the running time scales as $O \left( 1/(1-λ) \right)$ for these algorithms. In this paper, our goal is to design new algorithms that scale better than previous approaches when $λ$ approaches $1$. Our main contribution is to present a connection between VI and gradient descent and adapt the ideas of acceleration and momentum in convex optimization to design faster algorithms for MDPs. We prove theoretical guarantees of a faster convergence of our algorithms for the computation of the value function of a policy, where the running times of our algorithms scale as $O \left( 1/\sqrt{1-λ} \right)$ for reversible MDP instances. The improvement is quite analogous to Nesterov's acceleration and momentum in convex optimization. We also provide a lower bound on the convergence properties of any first-order algorithm for solving MDPs, presenting a family of MDPs instances for which no algorithm can converge faster than VI when the number of iterations is smaller than the number of states. We introduce a Safe Accelerated Value Iteration (S-AVI), which alternates between accelerated updates and value iteration updates. Our algorithm S-AVI is worst-case optimal and retains the theoretical convergence properties of VI while exhibiting strong empirical performances, providing significant speedups compared to classical approaches (up to one order of magnitude in many cases) for a large test bed of MDP instances.

math.OC↗

arXiv2021

Robust Policies For Proactive ICU Transfers

Patients whose transfer to the Intensive Care Unit (ICU) is unplanned are prone to higher mortality rates than those who were admitted directly to the ICU. Recent advances in machine learning to predict patient deterioration have introduced the possibility of \emph{proactive transfer} from the ward to the ICU. In this work, we study the problem of finding \emph{robust} patient transfer policies which account for uncertainty in statistical estimates due to data limitations when optimizing to improve overall patient care. We propose a Markov Decision Process model to capture the evolution of patient health, where the states represent a measure of patient severity. Under fairly general assumptions, we show that an optimal transfer policy has a threshold structure, i.e., that it transfers all patients above a certain severity level to the ICU (subject to available capacity). As model parameters are typically determined based on statistical estimations from real-world data, they are inherently subject to misspecification and estimation errors. We account for this parameter uncertainty by deriving a robust policy that optimizes the worst-case reward across all plausible values of the model parameters. We show that the robust policy also has a threshold structure under fairly general assumptions. Moreover, it is more aggressive in transferring patients than the optimal nominal policy, which does not take into account parameter uncertainty. We present computational experiments using a dataset of hospitalizations at 21 KNPC hospitals, and present empirical evidence of the sensitivity of various hospital metrics (mortality, length-of-stay, average ICU occupancy) to small changes in the parameters. Our work provides useful insights into the impact of parameter uncertainty on deriving simple policies for proactive ICU transfer that have strong empirical performance and theoretical guarantees.

cs.LG↗

arXiv2020

A Generalized Markov Chain Model to Capture Dynamic Preferences and Choice Overload

Assortment optimization is an important problem that arises in many industries such as retailing and online advertising where the goal is to find a subset of products from a universe of substitutable products which maximize seller's expected revenue. One of the key challenges in this problem is to model the customer substitution behavior. Many parametric random utility maximization (RUM) based choice models have been considered in the literature. However, in all these models, probability of purchase increases as we include more products to an assortment. This is not true in general and in many settings more choices hurt sales. This is commonly referred to as the choice overload. In this paper we attempt to address this limitation in RUM through a generalization of the Markov chain based choice model considered in Blanchet et al. (2016). As a special case, we show that our model reduces to a generalization of MNL with no-purchase attractions dependent on the assortment S and strictly increasing with the size of assortment S. While we show that the assortment optimization under this model is NP-hard, we present fully polynomial-time approximation scheme (FPTAS) under reasonable assumptions.

econ.TH↗