arXiv subjects

Yanyan Lan

Publications and source records attributed to Yanyan Lan.

93 records · Page 6Linked to original sources

arXiv2015

Structural Learning of Diverse Ranking

Relevance and diversity are both crucial criteria for an effective search system. In this paper, we propose a unified learning framework for simultaneously optimizing both relevance and diversity. Specifically, the problem is formalized as a structural learning framework optimizing Diversity-Correlated Evaluation Measures (DCEM), such as ERR-IA, a-NDCG and NRBP. Within this framework, the discriminant function is defined to be a bi-criteria objective maximizing the sum of the relevance scores and dissimilarities (or diversity) among the documents. Relevance and diversity features are utilized to define the relevance scores and dissimilarities, respectively. Compared with traditional methods, the advantages of our approach lie in that: (1) Directly optimizing DCEM as the loss function is more fundamental for the task; (2) Our framework does not rely on explicit diversity information such as subtopics, thus is more adaptive to real application; (3) The representation of diversity as the feature-based scoring function is more flexible to incorporate rich diversity-based features into the learning framework. Extensive experiments on the public TREC datasets show that our approach significantly outperforms state-of-the-art diversification approaches, which validate the above advantages.

cs.IR↗

arXiv2015

Topic-focused Dynamic Information Filtering in Social Media

With the quick development of online social media such as twitter or sina weibo in china, many users usually track hot topics to satisfy their desired information need. For a hot topic, new opinions or ideas will be continuously produced in the form of online data stream. In this scenario, how to effectively filter and display information for a certain topic dynamically, will be a critical problem. We call the problem as Topic-focused Dynamic Information Filtering (denoted as TDIF for short) in social media. In this paper, we start open discussions on such application problems. We first analyze the properties of the TDIF problem, which usually contains several typical requirements: relevance, diversity, recency and confidence. Recency means that users want to follow the recent opinions or news. Additionally, the confidence of information must be taken into consideration. How to balance these factors properly in online data stream is very important and challenging. We propose a dynamic preservation strategy on the basis of an existing feature-based utility function, to solve the TDIF problem. Additionally, we propose new dynamic diversity measures, to get a more reasonable evaluation for such application problems. Extensive exploratory experiments have been conducted on TREC public twitter dataset, and the experimental results validate the effectiveness of our approach.

cs.IR↗

arXiv2013

Stochastic Rank Aggregation

This paper addresses the problem of rank aggregation, which aims to find a consensus ranking among multiple ranking inputs. Traditional rank aggregation methods are deterministic, and can be categorized into explicit and implicit methods depending on whether rank information is explicitly or implicitly utilized. Surprisingly, experimental results on real data sets show that explicit rank aggregation methods would not work as well as implicit methods, although rank information is critical for the task. Our analysis indicates that the major reason might be the unreliable rank information from incomplete ranking inputs. To solve this problem, we propose to incorporate uncertainty into rank aggregation and tackle the problem in both unsupervised and supervised scenario. We call this novel framework {stochastic rank aggregation} (St.Agg for short). Specifically, we introduce a prior distribution on ranks, and transform the ranking functions or objectives in traditional explicit methods to their expectations over this distribution. Our experiments on benchmark data sets show that the proposed St.Agg outperforms the baselines in both unsupervised and supervised scenarios.

cs.LG↗