arXiv subjects

Tie-Yan Liu

Publications and source records attributed to Tie-Yan Liu.

244 records · Page 14Linked to original sources

arXiv2014

WordRep: A Benchmark for Research on Learning Word Representations

WordRep is a benchmark collection for the research on learning distributed word representations (or word embeddings), released by Microsoft Research. In this paper, we describe the details of the WordRep collection and show how to use it in different types of machine learning research related to word embedding. Specifically, we describe how the evaluation tasks in WordRep are selected, how the data are sampled, and how the evaluation tool is built. We then compare several state-of-the-art word representations on WordRep, report their evaluation performance, and make discussions on the results. After that, we discuss new potential research topics that can be supported by WordRep, in addition to algorithm comparison. We hope that this paper can help people gain deeper understanding of WordRep, and enable more interesting research on learning distributed word representations and related topics.

cs.CL↗

arXiv2014

A Game-theoretic Machine Learning Approach for Revenue Maximization in Sponsored Search

Sponsored search is an important monetization channel for search engines, in which an auction mechanism is used to select the ads shown to users and determine the prices charged from advertisers. There have been several pieces of work in the literature that investigate how to design an auction mechanism in order to optimize the revenue of the search engine. However, due to some unrealistic assumptions used, the practical values of these studies are not very clear. In this paper, we propose a novel \emph{game-theoretic machine learning} approach, which naturally combines machine learning and game theory, and learns the auction mechanism using a bilevel optimization framework. In particular, we first learn a Markov model from historical data to describe how advertisers change their bids in response to an auction mechanism, and then for any given auction mechanism, we use the learnt model to predict its corresponding future bid sequences. Next we learn the auction mechanism through empirical revenue maximization on the predicted bid sequences. We show that the empirical revenue will converge when the prediction period approaches infinity, and a Genetic Programming algorithm can effectively optimize this empirical revenue. Our experiments indicate that the proposed approach is able to produce a much more effective auction mechanism than several baselines.

cs.GT↗

arXiv2014

Incentivizing High-quality Content from Heterogeneous Users: On the Existence of Nash Equilibrium

We study the existence of pure Nash equilibrium (PNE) for the mechanisms used in Internet services (e.g., online reviews and question-answer websites) to incentivize users to generate high-quality content. Most existing work assumes that users are homogeneous and have the same ability. However, real-world users are heterogeneous and their abilities can be very different from each other due to their diverse background, culture, and profession. In this work, we consider heterogeneous users with the following framework: (1) the users are heterogeneous and each of them has a private type indicating the best quality of the content she can generate; (2) there is a fixed amount of reward to allocate to the participated users. Under this framework, we study the existence of pure Nash equilibrium of several mechanisms composed by different allocation rules, action spaces, and information settings. We prove the existence of PNE for some mechanisms and the non-existence of PNE for some mechanisms. We also discuss how to find a PNE for those mechanisms with PNE either through a constructive way or a search algorithm.

cs.GT↗

arXiv2014

Generalized Second Price Auction with Probabilistic Broad Match

Generalized Second Price (GSP) auctions are widely used by search engines today to sell their ad slots. Most search engines have supported broad match between queries and bid keywords when executing GSP auctions, however, it has been revealed that GSP auction with the standard broad-match mechanism they are currently using (denoted as SBM-GSP) has several theoretical drawbacks (e.g., its theoretical properties are known only for the single-slot case and full-information setting, and even in this simple setting, the corresponding worst-case social welfare can be rather bad). To address this issue, we propose a novel broad-match mechanism, which we call the Probabilistic Broad-Match (PBM) mechanism. Different from SBM that puts together the ads bidding on all the keywords matched to a given query for the GSP auction, the GSP with PBM (denoted as PBM-GSP) randomly samples a keyword according to a predefined probability distribution and only runs the GSP auction for the ads bidding on this sampled keyword. We perform a comprehensive study on the theoretical properties of the PBM-GSP. Specifically, we study its social welfare in the worst equilibrium, in both full-information and Bayesian settings. The results show that PBM-GSP can generate larger welfare than SBM-GSP under mild conditions. Furthermore, we also study the revenue guarantee for PBM-GSP in Bayesian setting. To the best of our knowledge, this is the first work on broad-match mechanisms for GSP that goes beyond the single-slot case and the full-information setting.

cs.GT↗

arXiv2014

Online Mechanism Design for Cloud Computing

In this work, we study the problem of online mechanism design for resources allocation and pricing in cloud computing (RAPCC). We show that in general the allocation problems in RAPCC are NP-hard, and therefore we focus on designing dominant-strategy incentive compatible (DSIC) mechanisms with good competitive ratios compared to the offline optimal allocation (with the prior knowledge about the future jobs). We propose two kinds of DSIC online mechanisms. The first mechanism, which is based on a greedy allocation rule and leverages a priority function for allocation, is very fast and has a tight competitive bound. We discuss several priority functions including exponential and linear priority functions, and show that the former one has a better competitive ratio. The second mechanism, which is based on a dynamic program for allocation, also has a tight competitive ratio and performs better than the first one when the maximum demand of cloud customers is close to the capacity of the cloud provider.

cs.GT↗

arXiv2013

Optimal Groupon Allocations

Group-buying websites represented by Groupon.com are very popular in electronic commerce and online shopping nowadays. They have multiple slots to provide deals with significant discounts to their visitors every day. The current user traffic allocation mostly relies on human decisions. We study the problem of automatically allocating the user traffic of a group-buying website to different deals to maximize the total revenue and refer to it as the Group-buying Allocation Problem (\GAP). The key challenge of \GAP\ is how to handle the tipping point (lower bound) and the purchase limit (upper bound) of each deal. We formulate \GAP\ as a knapsack-like problem with variable-sized items and majorization constraints. Our main results for \GAP\ can be summarized as follows. (1) We first show that for a special case of \GAP, in which the lower bound equals the upper bound for each deal, there is a simple dynamic programming-based algorithm that can find an optimal allocation in pseudo-polynomial time. (2) The general case of \GAP\ is much more difficult than the special case. To solve the problem, we first discover several structural properties of the optimal allocation, and then design a two-layer dynamic programming-based algorithm leveraging those properties. This algorithm can find an optimal allocation in pseudo-polynomial time. (3) We convert the two-layer dynamic programming based algorithm to a fully polynomial time approximation scheme (FPTAS), using the technique developed in \cite{ibarra1975fast}, combined with some careful modifications of the dynamic programs. Besides these results, we further investigate some natural generalizations of \GAP, and propose effective algorithms.

cs.GT↗

arXiv2013

$K$-anonymous Signaling Scheme

We incorporate signaling scheme into Ad Auction setting, to achieve better welfare and revenue while protect users' privacy. We propose a new \emph{$K$-anonymous signaling scheme setting}, prove the hardness of the corresponding welfare/revenue maximization problem, and finally propose the algorithms to approximate the optimal revenue or welfare.

cs.GT↗

arXiv2013

Introducing LETOR 4.0 Datasets

LETOR is a package of benchmark data sets for research on LEarning TO Rank, which contains standard features, relevance judgments, data partitioning, evaluation tools, and several baselines. Version 1.0 was released in April 2007. Version 2.0 was released in Dec. 2007. Version 3.0 was released in Dec. 2008. This version, 4.0, was released in July 2009. Very different from previous versions (V3.0 is an update based on V2.0 and V2.0 is an update based on V1.0), LETOR4.0 is a totally new release. It uses the Gov2 web page collection (~25M pages) and two query sets from Million Query track of TREC 2007 and TREC 2008. We call the two query sets MQ2007 and MQ2008 for short. There are about 1700 queries in MQ2007 with labeled documents and about 800 queries in MQ2008 with labeled documents. If you have any questions or suggestions about the datasets, please kindly email us (letor@microsoft.com). Our goal is to make the dataset reliable and useful for the community.

cs.IR↗

arXiv2013

Pure Price of Anarchy for Generalized Second Price Auction

The Generalized Second Price auction (GSP) has been widely used by search engines to sell ad slots. Previous studies have shown that the pure Price Of Anarchy (POA) of GSP is 1.25 when there are two ad slots and 1.259 when three ad slots. For the cases with more than three ad slots, however, only some untight upper bounds of the pure POA were obtained. In this work, we improve previous results in two aspects: (1) We prove that the pure POA for GSP is 1.259 when there are four ad slots, and (2) We show that the pure POA for GSP with more than four ad slots is also 1.259 given the bidders are ranked according to a particular permutation.

cs.GT↗

arXiv2013

A Theoretical Analysis of NDCG Type Ranking Measures

A central problem in ranking is to design a ranking measure for evaluation of ranking functions. In this paper we study, from a theoretical perspective, the widely used Normalized Discounted Cumulative Gain (NDCG)-type ranking measures. Although there are extensive empirical studies of NDCG, little is known about its theoretical properties. We first show that, whatever the ranking function is, the standard NDCG which adopts a logarithmic discount, converges to 1 as the number of items to rank goes to infinity. On the first sight, this result is very surprising. It seems to imply that NDCG cannot differentiate good and bad ranking functions, contradicting to the empirical success of NDCG in many applications. In order to have a deeper understanding of ranking measures in general, we propose a notion referred to as consistent distinguishability. This notion captures the intuition that a ranking measure should have such a property: For every pair of substantially different ranking functions, the ranking measure can decide which one is better in a consistent manner on almost all datasets. We show that NDCG with logarithmic discount has consistent distinguishability although it converges to the same limit for all ranking functions. We next characterize the set of all feasible discount functions for NDCG according to the concept of consistent distinguishability. Specifically we show that whether NDCG has consistent distinguishability depends on how fast the discount decays, and 1/r is a critical point. We then turn to the cut-off version of NDCG, i.e., NDCG@k. We analyze the distinguishability of NDCG@k for various choices of k and the discount functions. Experimental results on real Web search datasets agree well with the theory.

cs.LG↗