arXiv subjects

Patrick Jaillet

Publications and source records attributed to Patrick Jaillet.

At least 91 records · Page 5Linked to original sources

arXiv2020

R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games

This paper presents a recursive reasoning formalism of Bayesian optimization (BO) to model the reasoning process in the interactions between boundedly rational, self-interested agents with unknown, complex, and costly-to-evaluate payoff functions in repeated games, which we call Recursive Reasoning-Based BO (R2-B2). Our R2-B2 algorithm is general in that it does not constrain the relationship among the payoff functions of different agents and can thus be applied to various types of games such as constant-sum, general-sum, and common-payoff games. We prove that by reasoning at level 2 or more and at one level higher than the other agents, our R2-B2 agent can achieve faster asymptotic convergence to no regret than that without utilizing recursive reasoning. We also propose a computationally cheaper variant of R2-B2 called R2-B2-Lite at the expense of a weaker convergence guarantee. The performance and generality of our R2-B2 algorithm are empirically demonstrated using synthetic games, adversarial machine learning, and multi-agent reinforcement learning.

cs.LG↗

arXiv2019

Stochastic Dynamic Pricing for Same-Day Delivery Routing

Same-day delivery for e-commerce has become a popular service. Companies usually offer several time delivery options with the earliest one being next hour delivery. Due to tight delivery deadlines and thin margins, companies often find it challenging to provide efficient same-day delivery services. In this work, we propose a holistic scheme that combines the optimization of routing and pricing for same-day delivery. The proposed approach is able to take into account uncertainty in travel times, a crucial factor for delivery applications in urban environments. We model this problem as a Markov decision process. We apply a value function approximation technique to compute opportunity costs. Based on these opportunity costs, as well as the customer choice model and travel time distribution, we optimize the prices for various delivery deadlines. We perform extensive computational experiments to compare the proposed model with baseline policies. We also investigate how the (potentially wrong) choice of travel time distributions affect the performance of the proposed optimization scheme. Through numerical simulations of realistic scenarios, we observe that compared to the deterministic model, the proposed approach can reduce the number of missed deliveries up to 40%; at the same time, it can increase revenue by more than 5% compared to the baseline policies. We explore new issues that arise due to the stochastic nature of the problem such as the effect of penalties for missed deliveries on pricing structure and overall revenue.

math.OC↗

arXiv2019

Inverse Reinforcement Learning with Missing Data

We consider the problem of recovering an expert's reward function with inverse reinforcement learning (IRL) when there are missing/incomplete state-action pairs or observations in the demonstrated trajectories. This issue of missing trajectory data or information occurs in many situations, e.g., GPS signals from vehicles moving on a road network are intermittent. In this paper, we propose a tractable approach to directly compute the log-likelihood of demonstrated trajectories with incomplete/missing data. Our algorithm is efficient in handling a large number of missing segments in the demonstrated trajectories, as it performs the training with incomplete data by solving a sequence of systems of linear equations, and the number of such systems to be solved does not depend on the number of missing segments. Empirical evaluation on a real-world dataset shows that our training algorithm outperforms other conventional techniques.

cs.LG↗

arXiv2019

Implicit Posterior Variational Inference for Deep Gaussian Processes

A multi-layer deep Gaussian process (DGP) model is a hierarchical composition of GP models with a greater expressive power. Exact DGP inference is intractable, which has motivated the recent development of deterministic and stochastic approximation methods. Unfortunately, the deterministic approximation methods yield a biased posterior belief while the stochastic one is computationally costly. This paper presents an implicit posterior variational inference (IPVI) framework for DGPs that can ideally recover an unbiased posterior belief and still preserve time efficiency. Inspired by generative adversarial networks, our IPVI framework achieves this by casting the DGP inference problem as a two-player game in which a Nash equilibrium, interestingly, coincides with an unbiased posterior belief. This consequently inspires us to devise a best-response dynamics algorithm to search for a Nash equilibrium (i.e., an unbiased posterior belief). Empirical evaluation shows that IPVI outperforms the state-of-the-art approximation methods for DGPs.

cs.LG↗

arXiv2019

The Price of Interpretability

When quantitative models are used to support decision-making on complex and important topics, understanding a model's ``reasoning'' can increase trust in its predictions, expose hidden biases, or reduce vulnerability to adversarial attacks. However, the concept of interpretability remains loosely defined and application-specific. In this paper, we introduce a mathematical framework in which machine learning models are constructed in a sequence of interpretable steps. We show that for a variety of models, a natural choice of interpretable steps recovers standard interpretability proxies (e.g., sparsity in linear models). We then generalize these proxies to yield a parametrized family of consistent measures of model interpretability. This formal definition allows us to quantify the ``price'' of interpretability, i.e., the tradeoff with predictive accuracy. We demonstrate practical algorithms to apply our framework on real and synthetic datasets.

cs.LG↗

arXiv2019

Optimal Explanations of Linear Models

When predictive models are used to support complex and important decisions, the ability to explain a model's reasoning can increase trust, expose hidden biases, and reduce vulnerability to adversarial attacks. However, attempts at interpreting models are often ad hoc and application-specific, and the concept of interpretability itself is not well-defined. We propose a general optimization framework to create explanations for linear models. Our methodology decomposes a linear model into a sequence of models of increasing complexity using coordinate updates on the coefficients. Computing this decomposition optimally is a difficult optimization problem for which we propose exact algorithms and scalable heuristics. By solving this problem, we can derive a parametrized family of interpretability metrics for linear models that generalizes typical proxies, and study the tradeoff between interpretability and predictive accuracy.

cs.LG↗

arXiv2019

An Improved Lower Bound for the Traveling Salesman Constant

Let $X_1, X_2, \dots, X_n$ be independent uniform random variables on $[0,1]^2$. Let $L(X_1, \dots, X_n)$ be the length of the shortest Traveling Salesman tour through these points. It is known that there exists a constant $β$ such that $$\lim_{n \to \infty} \frac{L(X_1, \dots, X_n)}{\sqrt{n}} = β$$ almost surely (Beardwood 1959). The original analysis in (Beardwood 1959) showed that $β\geq 0.625$. Building upon an approach proposed in (Steinerberger 2015), we improve the lower bound to $β\geq 0.6277$.

math.PR↗

arXiv2019

Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression

This paper presents a novel variational inference framework for deriving a family of Bayesian sparse Gaussian process regression (SGPR) models whose approximations are variationally optimal with respect to the full-rank GPR model enriched with various corresponding correlation structures of the observation noises. Our variational Bayesian SGPR (VBSGPR) models jointly treat both the distributions of the inducing variables and hyperparameters as variational parameters, which enables the decomposability of the variational lower bound that in turn can be exploited for stochastic optimization. Such a stochastic optimization involves iteratively following the stochastic gradient of the variational lower bound to improve its estimates of the optimal variational distributions of the inducing variables and hyperparameters (and hence the predictive distribution) of our VBSGPR models and is guaranteed to achieve asymptotic convergence to them. We show that the stochastic gradient is an unbiased estimator of the exact gradient and can be computed in constant time per iteration, hence achieving scalability to big data. We empirically evaluate the performance of our proposed framework on two real-world, massive datasets.

cs.LG↗

arXiv2018

Exponential Convergence Rates for Stochastically Ordered Markov Processes with Random Initial Conditions

In this brief paper we find computable exponential convergence rates for a large class of stochastically ordered Markov processes. We extend the result of Lund, Meyn, and Tweedie (1996), who found exponential convergence rates for stochastically ordered Markov processes starting from a fixed initial state, by allowing for a random initial condition that is also stochastically ordered. Our bounds are formulated in terms of moment-generating functions of hitting times. To illustrate our result, we find an explicit exponential convergence rate for an M/M/1 queue beginning in equilibrium and then experiencing a change in its arrival or departure rates, a setting which has not been studied to our knowledge.

math.PR↗

arXiv2018

Online Resource Allocation under Partially Predictable Demand

For online resource allocation problems, we propose a new demand arrival model where the sequence of arrivals contains both an adversarial component and a stochastic one. Our model requires no demand forecasting; however, due to the presence of the stochastic component, we can partially predict future demand as the sequence of arrivals unfolds. Under the proposed model, we study the problem of the online allocation of a single resource to two types of customers, and design online algorithms that outperform existing ones. Our algorithms are adjustable to the relative size of the stochastic component, and our analysis reveals that as the portion of the stochastic component grows, the loss due to making online decisions decreases. This highlights the value of (even partial) predictability in online resource allocation. We impose no conditions on how the resource capacity scales with the maximum number of customers. However, we show that using an adaptive algorithm---which makes online decisions based on observed data---is particularly beneficial when capacity scales linearly with the number of customers. Our work serves as a first step in bridging the long-standing gap between the two well-studied approaches to the design and analysis of online algorithms based on (1) adversarial models and (2) stochastic ones. Using novel algorithm design, we demonstrate that even if the arrival sequence contains an adversarial component, we can take advantage of the limited information that the data reveals to improve allocation decisions. We also study the classical secretary problem under our proposed arrival model, and we show that randomizing over multiple stopping rules may increase the probability of success.

cs.DS↗

arXiv2018

Separable Convex Optimization with Nested Lower and Upper Constraints

We study a convex resource allocation problem in which lower and upper bounds are imposed on partial sums of allocations. This model is linked to a large range of applications, including production planning, speed optimization, stratified sampling, support vector machines, portfolio management, and telecommunications. We propose an efficient gradient-free divide-and-conquer algorithm, which uses monotonicity arguments to generate valid bounds from the recursive calls, and eliminate linking constraints based on the information from sub-problems. This algorithm does not need strict convexity or differentiability. It produces an $ε$-approximate solution for the continuous problem in $\mathcal{O}(n \log m \log \frac{n B}ε)$ time and an integer solution in $\mathcal{O}(n \log m \log B)$ time, where $n$ is the number of decision variables, $m$ is the number of constraints, and $B$ is the resource bound. A complexity of $\mathcal{O}(n \log m)$ is also achieved for the linear and quadratic cases. These are the best complexities known to date for this important problem class. Our experimental analyses confirm the good performance of the method, which produces optimal solutions for problems with up to 1,000,000 variables in a few seconds. Promising applications to the support vector ordinal regression problem are also investigated.

math.OC↗

arXiv2018

Signaling Game-based Misbehavior Inspection in V2I-enabled Highway Operations

Vehicle-to-Infrastructure (V2I) communications are increasingly supporting highway operations such as electronic toll collection, carpooling, and vehicle platooning. In this paper we study the incentives of strategic misbehavior by individual vehicles who can exploit the security vulnerabilities in V2I communications and negatively impact the highway operations. We consider a V2I-enabled highway segment facing two classes of vehicles (agent populations), each with an authorized access to one server (subset of lanes). Vehicles are strategic in that they can misreport their class (type) to the system operator and get an unauthorized access to the server dedicated to the other class. This misbehavior causes additional congestion externality on the compliant vehicles, and thus, needs to be deterred. We focus on an environment where the operator is able to inspect the vehicles for misbehavior. The inspection is costly and successful detection incurs a fine on the misbehaving vehicle. We formulate a signaling game to study the strategic interaction between the vehicle classes and the operator. Our equilibrium analysis provides conditions on the cost parameters that govern the vehicles' incentive to misbehave or not. We also determine the operator's equilibrium inspection strategy.

cs.GT↗

arXiv2018

Maximum Weight Online Matching with Deadlines

cs.DS↗

arXiv2018

Maximizing Efficiency in Dynamic Matching Markets

We study the problem of matching agents who arrive at a marketplace over time and leave after d time periods. Agents can only be matched while they are present in the marketplace. Each pair of agents can yield a different match value, and the planner's goal is to maximize the total value over a finite time horizon. We study matching algorithms that perform well over any sequence of arrivals when there is no a priori information about the match values or arrival times. Our main contribution is a 1/4-competitive algorithm. The algorithm randomly selects a subset of agents who will wait until right before their departure to get matched, and maintains a maximum-weight matching with respect to the other agents. The primal-dual analysis of the algorithm hinges on a careful comparison between the initial dual value associated with an agent when it first arrives, and the final value after d time steps. It is also shown that no algorithm is 1/2-competitive. We extend the model to the case in which departure times are drawn i.i.d from a distribution with non-decreasing hazard rate, and establish a 1/8-competitive algorithm in this setting. Finally we show on real-world data that a modified version of our algorithm performs well in practice.

cs.DS↗

arXiv2017

On Matching and Thickness in Heterogeneous Dynamic Markets

We study dynamic matching in an infinite-horizon stochastic market. While all agents are potentially compatible with each other, some are hard-to-match and others are easy-to-match. Agents prefer to be matched as soon as possible and matches are formed either bilaterally or indirectly through chains. We adopt an asymptotic approach and compute tight bounds on the limit of waiting time of agents under myopic policies that differ in matching technology and prioritization. We find that the market composition is a key factor in the desired matching technology and prioritization level. When hard-to-match agents arrive less frequently than easy-to-match ones (i) bilateral matching is almost as efficient as chains (waiting times scale similarly under both, though chains always outperform bilateral matching by a constant factor), and (ii) assigning priorities to hard-to-match agents improves their waiting times. When hard-to-match agents arrive more frequently, chains are much more efficient than bilateral matching and prioritization has no impact. We further conduct comparative statics on arrival rates. Somewhat surprisingly, we find that in a heterogeneous market and under bilateral matching, increasing arrival rate has a non-monotone effect on waiting times, due to the fact that, under some market compositions, there is an adverse effect of competition. Our comparative statics shed light on the impact of merging markets and attracting altruistic agents (that initiate chains) or easy-to-match agents. This work uncovers fundamental differences between heterogeneous and homogeneous dynamic markets, and potentially helps policy makers to generate insights on the operations of matching markets such as kidney exchange programs.

cs.DS↗

arXiv2017

The Stochastic Container Relocation Problem

The Container Relocation Problem (CRP) is concerned with finding a sequence of moves of containers that minimizes the number of relocations needed to retrieve all containers, while respecting a given order of retrieval. However, the assumption of knowing the full retrieval order of containers is particularly unrealistic in real operations. This paper studies the stochastic CRP (SCRP), which relaxes this assumption. A new multi-stage stochastic model, called the batch model, is introduced, motivated, and compared with an existing model (the online model). The two main contributions are an optimal algorithm called Pruning-Best-First-Search (PBFS) and a randomized approximate algorithm called PBFS-Approximate with a bounded average error. Both algorithms, applicable in the batch and online models, are based on a new family of lower bounds for which we show some theoretical properties. Moreover, we introduce two new heuristics outperforming the best existing heuristics. Algorithms, bounds and heuristics are tested in an extensive computational section. Finally, based on strong computational evidence, we conjecture the optimality of the "Leveling" heuristic in a special "no information" case, where at any retrieval stage, any of the remaining containers is equally likely to be retrieved next.

cs.DS↗

arXiv2017

Logarithmic regret bounds for Bandits with Knapsacks

Optimal regret bounds for Multi-Armed Bandit problems are now well documented. They can be classified into two categories based on the growth rate with respect to the time horizon $T$: (i) small, distribution-dependent, bounds of order of magnitude $\ln(T)$ and (ii) robust, distribution-free, bounds of order of magnitude $\sqrt{T}$. The Bandits with Knapsacks model, an extension to the framework allowing to model resource consumption, lacks this clear-cut distinction. While several algorithms have been shown to achieve asymptotically optimal distribution-free bounds on regret, there has been little progress toward the development of small distribution-dependent regret bounds. We partially bridge the gap by designing a general-purpose algorithm with distribution-dependent regret bounds that are logarithmic in the initial endowments of resources in several important cases that cover many practical applications, including dynamic pricing with limited supply, bid optimization in online advertisement auctions, and dynamic procurement.

cs.DS↗

arXiv2017

Structured Prediction by Conditional Risk Minimization

We propose a general approach for supervised learning with structured output spaces, such as combinatorial and polyhedral sets, that is based on minimizing estimated conditional risk functions. Given a loss function defined over pairs of output labels, we first estimate the conditional risk function by solving a (possibly infinite) collection of regularized least squares problems. A prediction is made by solving an inference problem that minimizes the estimated conditional risk function over the output space. We show that this approach enables, in some cases, efficient training and inference without explicitly introducing a convex surrogate for the original loss function, even when it is discontinuous. Empirical evaluations on real-world and synthetic data sets demonstrate the effectiveness of our method in adapting to a variety of loss functions.

stat.ML↗