Searcharxiv⌕ Search

arXiv subjects

Yishay Mansour

Publications and source records attributed to Yishay Mansour.

At least 145 records · Page 8Linked to original sources
arXiv2019

Online Convex Optimization in Adversarial Markov Decision Processes

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner. We show $\tilde{O}(L|X|\sqrt{|A|T})$ regret bound, where $T$ is the number of episodes, $X$ is the state space, $A$ is the action space, and $L$ is the length of each episode. Our online algorithm is implemented using entropic regularization methodology, which allows to extend the original adversarial MDP model to handle convex performance criteria (different ways to aggregate the losses of a single episode) , as well as improve previous regret bounds.

cs.LG↗
arXiv2019

Optimal Algorithm for Bayesian Incentive-Compatible Exploration

We consider a social planner faced with a stream of myopic selfish agents. The goal of the social planner is to maximize the social welfare, however, it is limited to using only information asymmetry (regarding previous outcomes) and cannot use any monetary incentives. The planner recommends actions to agents, but her recommendations need to be Bayesian Incentive Compatible to be followed by the agents. Our main result is an optimal algorithm for the planner, in the case that the actions realizations are deterministic and have limited support, making significant important progress on this open problem. Our optimal protocol has two interesting features. First, it always completes the exploration of a priori more beneficial actions before exploring a priori less beneficial actions. Second, the randomization in the protocol is correlated across agents and actions (and not independent at each decision time).

cs.GT↗
arXiv2019

Bayesian Incentive-Compatible Bandit Exploration

Individual decision-makers consume information revealed by the previous decision makers, and produce information that may help in future decisions. This phenomenon is common in a wide range of scenarios in the Internet economy, as well as in other domains such as medical decisions. Each decision-maker would individually prefer to "exploit": select an action with the highest expected reward given her current information. At the same time, each decision-maker would prefer previous decision-makers to "explore", producing information about the rewards of various actions. A social planner, by means of carefully designed information disclosure, can incentivize the agents to balance the exploration and exploitation so as to maximize social welfare. We formulate this problem as a multi-armed bandit problem (and various generalizations thereof) under incentive-compatibility constraints induced by the agents' Bayesian priors. We design an incentive-compatible bandit algorithm for the social planner whose regret is asymptotically optimal among all bandit algorithms (incentive-compatible or not). Further, we provide a black-box reduction from an arbitrary multi-arm bandit algorithm to an incentive-compatible one, with only a constant multiplicative increase in regret. This reduction works for very general bandit setting that incorporate contexts and arbitrary auxiliary feedback.

cs.GT↗
arXiv2019

Competitive ratio versus regret minimization: achieving the best of both worlds

We consider online algorithms under both the competitive ratio criteria and the regret minimization one. Our main goal is to build a unified methodology that would be able to guarantee both criteria simultaneously. For a general class of online algorithms, namely any Metrical Task System (MTS), we show that one can simultaneously guarantee the best known competitive ratio and a natural regret bound. For the paging problem we further show an efficient online algorithm (polynomial in the number of pages) with this guarantee. To this end, we extend an existing regret minimization algorithm (specifically, Kapralov and Panigrahy) to handle movement cost (the cost of switching between states of the online system). We then show how to use the extended regret minimization algorithm to combine multiple online algorithms. Our end result is an online algorithm that can combine a "base" online algorithm, having a guaranteed competitive ratio, with a range of online algorithms that guarantee a small regret over any interval of time. The combined algorithm guarantees both that the competitive ratio matches that of the base algorithm and a low regret over any time interval. As a by product, we obtain an expert algorithm with close to optimal regret bound on every time interval, even in the presence of switching costs. This result is of independent interest.

cs.LG↗
arXiv2019

Differentially Private Learning of Geometric Concepts

We present differentially private efficient algorithms for learning union of polygons in the plane (which are not necessarily convex). Our algorithms achieve $(α,β)$-PAC learning and $(ε,δ)$-differential privacy using a sample of size $\tilde{O}\left(\frac{1}{αε}k\log d\right)$, where the domain is $[d]\times[d]$ and $k$ is the number of edges in the union of polygons.

cs.LG↗
arXiv2018

Adversarial Online Learning with noise

We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback. Specifically, we consider binary losses xored with the noise, which is a Bernoulli random variable. We consider both a constant noise rate and a variable noise rate. Our main results are tight regret bounds for learning with noise in the adversarial online learning model.

cs.LG↗
arXiv2018

Flow Equilibria via Online Surge Pricing

We explore issues of dynamic supply and demand in ride sharing services such as Lyft and Uber, where demand fluctuates over time and geographic location. We seek to maximize social welfare which depends on taxicab and passenger locations, passenger valuations for service, and the distances between taxicabs and passengers. Our only means of control is to set surge prices, then taxicabs and passengers maximize their utilities subject to these prices. We study two related models: a continuous passenger-taxicab setting, similar to the Wardrop model, and a discrete passenger-taxicab setting. In the continuous setting, every location is occupied by a set of infinitesimal strategic taxicabs and a set of infinitesimal non-strategic passengers. In the discrete setting every location is occupied by a set of strategic agents, taxicabs and passengers, passengers have differing values for service. We expand the continuous model to a time-dependent setting and study the corresponding online environment. Surge prices are in passenger-taxicab equilibrium if there exists a min cost flow that moves taxicabs about such that (a) every taxicab follows a best response, (b) all strategic passengers at $v$ with value above the surge price $r_v$ for $v$, are served and (c) no strategic passengers with value below $r_v$ are served (non-strategic infinitesimal passengers are always served). This paper computes surge prices such that resulting passenger-taxicab equilibrium maximizes social welfare, and the computation of such surge prices is in poly time. Moreover, it is a dominant strategy for passengers to reveal their true values. We seek to maximize social welfare in the online environment, and derive tight competitive ratio bounds to this end. Our online algorithms make use of the surge prices computed over time and geographic location, inducing successive passenger-taxicab equilibria.

cs.GT↗
arXiv2018

Are Two (Samples) Really Better Than One? On the Non-Asymptotic Performance of Empirical Revenue Maximization

The literature on "mechanism design from samples," which has flourished in recent years at the interface of economics and computer science, offers a bridge between the classic computer-science approach of worst-case analysis (corresponding to "no samples") and the classic economic approach of average-case analysis for a given Bayesian prior (conceptually corresponding to the number of samples tending to infinity). Nonetheless, the two directions studied so far are two extreme and almost diametrically opposed directions: that of asymptotic results where the number of samples grows large, and that where only a single sample is available. In this paper, we take a first step toward understanding the middle ground that bridges these two approaches: that of a fixed number of samples greater than one. In a variety of contexts, we ask what is possibly the most fundamental question in this direction: "are two samples really better than one sample?". We present a few surprising negative results, and complement them with our main result: showing that the worst-case, over all regular distributions, expected-revenue guarantee of the Empirical Revenue Maximization algorithm given two samples is greater than that of this algorithm given one sample. The proof is technically challenging, and provides the first result that shows that some deterministic mechanism constructed using two samples can guarantee more than one half of the optimal revenue.

cs.GT↗
arXiv2018

Online Linear Quadratic Control

We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee $O(\sqrt{T})$ regret under mild assumptions, where $T$ is the time horizon. Our algorithms rely on a novel SDP relaxation for the steady-state distribution of the system. Crucially, and in contrast to previously proposed relaxations, the feasible solutions of our SDP all correspond to "strongly stable" policies that mix exponentially fast to a steady state.

cs.LG↗
arXiv2018

Fair Leader Election for Rational Agents in Asynchronous Rings and Networks

We study a game theoretic model where a coalition of processors might collude to bias the outcome of the protocol, where we assume that the processors always prefer any legitimate outcome over a non-legitimate one. We show that the problems of Fair Leader Election and Fair Coin Toss are equivalent, and focus on Fair Leader Election. Our main focus is on a directed asynchronous ring of $n$ processors, where we investigate the protocol proposed by Abraham et al. \cite{abraham2013distributed} and studied in Afek et al. \cite{afek2014distributed}. We show that in general the protocol is resilient only to sub-linear size coalitions. Specifically, we show that $Ω(\sqrt{n\log n})$ randomly located processors or $Ω(\sqrt[3]{n})$ adversarially located processors can force any outcome. We complement this by showing that the protocol is resilient to any adversarial coalition of size $O(\sqrt[4]{n})$. We propose a modification to the protocol, and show that it is resilient to every coalition of size $Θ(\sqrt{n})$, by exhibiting both an attack and a resilience result. For every $k \geq 1$, we define a family of graphs ${\mathcal{G}}_{k}$ that can be simulated by trees where each node in the tree simulates at most $k$ processors. We show that for every graph in ${\mathcal{G}}_{k}$, there is no fair leader election protocol that is resilient to coalitions of size $k$. Our result generalizes a previous result of Abraham et al. \cite{abraham2013distributed} that states that for every graph, there is no fair leader election protocol which is resilient to coalitions of size $\lceil \frac{n}{2} \rceil$.

cs.DC↗
arXiv2018

Are All Experts Equally Good? A Study of Analyst Earnings Estimates

We investigate whether experts possess differential expertise when making predictions. We note that this would make it possible to aggregate multiple predictions into a result that is more accurate than their consensus average, and that the improvement prospects grow with the amount of differentiation. Turning this argument on its head, we show how differentiation can be measured by how much weighted aggregation improves on simple averaging. Taking stock-market analysts as experts in their domain, we do a retrospective study using historical quarterly earnings forecasts and actual results for large publicly traded companies. We use it to shed new light on the Sinha et al. (1997) result, showing that analysts indeed possess individual expertise, but that their differentiation is modest. On the other hand, they have significant individual bias. Together, these enable a 20%-30% accuracy improvement over consensus average.

cs.CY↗
arXiv2018

Hierarchical Reinforcement Learning: Approximating Optimal Discounted TSP Using Local Policies

In this work, we provide theoretical guarantees for reward decomposition in deterministic MDPs. Reward decomposition is a special case of Hierarchical Reinforcement Learning, that allows one to learn many policies in parallel and combine them into a composite solution. Our approach builds on mapping this problem into a Reward Discounted Traveling Salesman Problem, and then deriving approximate solutions for it. In particular, we focus on approximate solutions that are local, i.e., solutions that only observe information about the current state. Local policies are easy to implement and do not require substantial computational resources as they do not perform planning. While local deterministic policies, like Nearest Neighbor, are being used in practice for hierarchical reinforcement learning, we propose three stochastic policies that guarantee better performance than any deterministic policy.

cs.LG↗
arXiv2018

Discriminative Learning of Prediction Intervals

In this work we consider the task of constructing prediction intervals in an inductive batch setting. We present a discriminative learning framework which optimizes the expected error rate under a budget constraint on the interval sizes. Most current methods for constructing prediction intervals offer guarantees for a single new test point. Applying these methods to multiple test points can result in a high computational overhead and degraded statistical guarantees. By focusing on expected errors, our method allows for variability in the per-example conditional error rates. As we demonstrate both analytically and empirically, this flexibility can increase the overall accuracy, or alternatively, reduce the average interval size. While the problem we consider is of a regressive flavor, the loss we use is combinatorial. This allows us to provide PAC-style, finite-sample guarantees. Computationally, we show that our original objective is NP-hard, and suggest a tractable convex surrogate. We conclude with a series of experimental evaluations.

cs.LG↗
arXiv2017

Competing Bandits: Learning under Competition

Most modern systems strive to learn from interactions with users, and many engage in exploration: making potentially suboptimal choices for the sake of acquiring new information. We initiate a study of the interplay between exploration and competition--how such systems balance the exploration for learning and the competition for users. Here the users play three distinct roles: they are customers that generate revenue, they are sources of data for learning, and they are self-interested agents which choose among the competing systems. In our model, we consider competition between two multi-armed bandit algorithms faced with the same bandit instance. Users arrive one by one and choose among the two algorithms, so that each algorithm makes progress if and only if it is chosen. We ask whether and to what extent competition incentivizes the adoption of better bandit algorithms. We investigate this issue for several models of user response, as we vary the degree of rationality and competitiveness in the model. Our findings are closely related to the "competition vs. innovation" relationship, a well-studied theme in economics.

cs.GT↗
arXiv2017

Submultiplicative Glivenko-Cantelli and Uniform Convergence of Revenues

In this work we derive a variant of the classic Glivenko-Cantelli Theorem, which asserts uniform convergence of the empirical Cumulative Distribution Function (CDF) to the CDF of the underlying distribution. Our variant allows for tighter convergence bounds for extreme values of the CDF. We apply our bound in the context of revenue learning, which is a well-studied problem in economics and algorithmic game theory. We derive sample-complexity bounds on the uniform convergence rate of the empirical revenues to the true revenues, assuming a bound on the $k$th moment of the valuations, for any (possibly fractional) $k>1$. For uniform convergence in the limit, we give a complete characterization and a zero-one law: if the first moment of the valuations is finite, then uniform convergence almost surely occurs; conversely, if the first moment is infinite, then uniform convergence almost never occurs.

cs.LG↗
arXiv2017

Multi-Armed Bandits with Metric Movement Costs

We consider the non-stochastic Multi-Armed Bandit problem in a setting where there is a fixed and known metric on the action space that determines a cost for switching between any pair of actions. The loss of the online learner has two components: the first is the usual loss of the selected actions, and the second is an additional loss due to switching between actions. Our main contribution gives a tight characterization of the expected minimax regret in this setting, in terms of a complexity measure $\mathcal{C}$ of the underlying metric which depends on its covering numbers. In finite metric spaces with $k$ actions, we give an efficient algorithm that achieves regret of the form $\widetilde{O}(\max\{\mathcal{C}^{1/3}T^{2/3},\sqrt{kT}\})$, and show that this is the best possible. Our regret bound generalizes previous known regret bounds for some special cases: (i) the unit-switching cost regret $\widetildeΘ(\max\{k^{1/3}T^{2/3},\sqrt{kT}\})$ where $\mathcal{C}=Θ(k)$, and (ii) the interval metric with regret $\widetildeΘ(\max\{T^{2/3},\sqrt{kT}\})$ where $\mathcal{C}=Θ(1)$. For infinite metrics spaces with Lipschitz loss functions, we derive a tight regret bound of $\widetildeΘ(T^{\frac{d+1}{d+2}})$ where $d \ge 1$ is the Minkowski dimension of the space, which is known to be tight even when there are no switching costs.

cs.LG↗
arXiv2017

The Strategy of Experts for Repeated Predictions

We investigate the behavior of experts who seek to make predictions with maximum impact on an audience. At a known future time, a certain continuous random variable will be realized. A public prediction gradually converges to the outcome, and an expert has access to a more accurate prediction. We study when the expert should reveal his information, when his reward is based on a proper scoring rule (e.g., is proportional to the change in log-likelihood of the outcome). In Azar et. al. (2016), we analyzed the case where the expert may make a single prediction. In this paper, we analyze the case where the expert is allowed to revise previous predictions. This leads to a rather different set of dilemmas for the strategic expert. We find that it is optimal for the expert to always tell the truth, and to make a new prediction whenever he has a new signal. We characterize the expert's expectation for his total reward, and show asymptotic limits

cs.GT↗