arXiv subjects

Bo Waggoner

Publications and source records attributed to Bo Waggoner.

43 records · Page 3Linked to original sources

arXiv2017

Multi-Observation Elicitation

We study loss functions that measure the accuracy of a prediction based on multiple data points simultaneously. To our knowledge, such loss functions have not been studied before in the area of property elicitation or in machine learning more broadly. As compared to traditional loss functions that take only a single data point, these multi-observation loss functions can in some cases drastically reduce the dimensionality of the hypothesis required. In elicitation, this corresponds to requiring many fewer reports; in empirical risk minimization, it corresponds to algorithms on a hypothesis space of much smaller dimension. We explore some examples of the tradeoff between dimensionality and number of observations, give some geometric characterizations and intuition for relating loss functions and the properties that they elicit, and discuss some implications for both elicitation and machine-learning contexts.

cs.LG↗

arXiv2017

Accuracy First: Selecting a Differential Privacy Level for Accuracy-Constrained ERM

Traditional approaches to differential privacy assume a fixed privacy requirement $ε$ for a computation, and attempt to maximize the accuracy of the computation subject to the privacy constraint. As differential privacy is increasingly deployed in practical settings, it may often be that there is instead a fixed accuracy requirement for a given computation and the data analyst would like to maximize the privacy of the computation subject to the accuracy constraint. This raises the question of how to find and run a maximally private empirical risk minimizer subject to a given accuracy requirement. We propose a general "noise reduction" framework that can apply to a variety of private empirical risk minimization (ERM) algorithms, using them to "search" the space of privacy levels to find the empirically strongest one that meets the accuracy constraint, incurring only logarithmic overhead in the number of privacy levels searched. The privacy analysis of our algorithm leads naturally to a version of differential privacy where the privacy parameters are dependent on the data, which we term ex-post privacy, and which is related to the recently introduced notion of privacy odometers. We also give an ex-post privacy analysis of the classical AboveThreshold privacy tool, modifying it to allow for queries chosen depending on the database. Finally, we apply our approach to two common objectives, regularized linear and logistic regression, and empirically compare our noise reduction methods to (i) inverting the theoretical utility guarantees of standard private ERM algorithms and (ii) a stronger, empirical baseline based on binary search.

cs.LG↗

arXiv2017

Informational Substitutes

We propose definitions of substitutes and complements for pieces of information ("signals") in the context of a decision or optimization problem, with game-theoretic and algorithmic applications. In a game-theoretic context, substitutes capture diminishing marginal value of information to a rational decision maker. We use the definitions to address the question of how and when information is aggregated in prediction markets. Substitutes characterize "best-possible" equilibria with immediate information aggregation, while complements characterize "worst-possible", delayed aggregation. Game-theoretic applications also include settings such as crowdsourcing contests and Q\&A forums. In an algorithmic context, where substitutes capture diminishing marginal improvement of information to an optimization problem, substitutes imply efficient approximation algorithms for a very general class of (adaptive) information acquisition problems. In tandem with these broad applications, we examine the structure and design of informational substitutes and complements. They have equivalent, intuitive definitions from disparate perspectives: submodularity, geometry, and information theory. We also consider the design of scoring rules or optimization problems so as to encourage substitutability or complementarity, with positive and negative results. Taken as a whole, the results give some evidence that, in parallel with substitutable items, informational substitutes play a natural conceptual and formal role in game theory and algorithms.

cs.GT↗

arXiv2016

Descending Price Optimally Coordinates Search

Investigating potential purchases is often a substantial investment under uncertainty. Standard market designs, such as simultaneous or English auctions, compound this with uncertainty about the price a bidder will have to pay in order to win. As a result they tend to confuse the process of search both by leading to wasteful information acquisition on goods that have already found a good purchaser and by discouraging needed investigations of objects, potentially eliminating all gains from trade. In contrast, we show that the Dutch auction preserves all of its properties from a standard setting without information costs because it guarantees, at the time of information acquisition, a price at which the good can be purchased. Calibrations to start-up acquisition and timber auctions suggest that in practice the social losses through poor search coordination in standard formats are an order of magnitude or two larger than the (negligible) inefficiencies arising from ex-ante bidder asymmetries.

cs.GT↗

arXiv2015

Low-Cost Learning via Active Data Procurement

We design mechanisms for online procurement of data held by strategic agents for machine learning tasks. The challenge is to use past data to actively price future data and give learning guarantees even when an agent's cost for revealing her data may depend arbitrarily on the data itself. We achieve this goal by showing how to convert a large class of no-regret algorithms into online posted-price and learning mechanisms. Our results in a sense parallel classic sample complexity guarantees, but with the key resource being money rather than quantity of data: With a budget constraint $B$, we give robust risk (predictive error) bounds on the order of $1/\sqrt{B}$. Because we use an active approach, we can often guarantee to do significantly better by leveraging correlations between costs and data. Our algorithms and analysis go through a model of no-regret learning with $T$ arriving pairs (cost, data) and a budget constraint of $B$. Our regret bounds for this model are on the order of $T/\sqrt{B}$ and we give lower bounds on the same order.

cs.GT↗

arXiv2015

$\ell_p$ Testing and Learning of Discrete Distributions

The classic problems of testing uniformity of and learning a discrete distribution, given access to independent samples from it, are examined under general $\ell_p$ metrics. The intuitions and results often contrast with the classic $\ell_1$ case. For $p > 1$, we can learn and test with a number of samples that is independent of the support size of the distribution: With an $\ell_p$ tolerance $ε$, $O(\max\{ \sqrt{1/ε^q}, 1/ε^2 \})$ samples suffice for testing uniformity and $O(\max\{ 1/ε^q, 1/ε^2\})$ samples suffice for learning, where $q=p/(p-1)$ is the conjugate of $p$. As this parallels the intuition that $O(\sqrt{n})$ and $O(n)$ samples suffice for the $\ell_1$ case, it seems that $1/ε^q$ acts as an upper bound on the "apparent" support size. For some $\ell_p$ metrics, uniformity testing becomes easier over larger supports: a 6-sided die requires fewer trials to test for fairness than a 2-sided coin, and a card-shuffler requires fewer trials than the die. In fact, this inverse dependence on support size holds if and only if $p > \frac{4}{3}$. The uniformity testing algorithm simply thresholds the number of "collisions" or "coincidences" and has an optimal sample complexity up to constant factors for all $1 \leq p \leq 2$. Another algorithm gives order-optimal sample complexity for $\ell_{\infty}$ uniformity testing. Meanwhile, the most natural learning algorithm is shown to have order-optimal sample complexity for all $\ell_p$ metrics. The author thanks Clément Canonne for discussions and contributions to this work.

cs.DS↗

arXiv2013

Designing Markets for Daily Deals

Daily deals platforms such as Amazon Local, Google Offers, GroupOn, and LivingSocial have provided a new channel for merchants to directly market to consumers. In order to maximize consumer acquisition and retention, these platforms would like to offer deals that give good value to users. Currently, selecting such deals is done manually; however, the large number of submarkets and localities necessitates an automatic approach to selecting good deals and determining merchant payments. We approach this challenge as a market design problem. We postulate that merchants already have a good idea of the attractiveness of their deal to consumers as well as the amount they are willing to pay to offer their deal. The goal is to design an auction that maximizes a combination of the revenue of the auctioneer (platform), welfare of the bidders (merchants), and the positive externality on a third party (the consumer), despite the asymmetry of information about this consumer benefit. We design auctions that truthfully elicit this information from the merchants and maximize the social welfare objective, and we characterize the consumer welfare functions for which this objective is truthfully implementable. We generalize this characterization to a very broad mechanism-design setting and give examples of other applications.

cs.GT↗