arXiv subjects

Ruijie Xi

Publications and source records attributed to Ruijie Xi.

6 recordsLinked to original sources

arXiv2025

Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation

Embedding-Based Retrieval (EBR) is an important technique in modern search engines, enabling semantic match between search queries and relevant results. However, search logging data on platforms like Facebook Marketplace lacks the diversity and details needed for effective EBR model training, limiting the models' ability to capture nuanced search patterns. To address this challenge, we propose Aug2Search, an EBR-based framework leveraging synthetic data generated by Generative AI (GenAI) models, in a multimodal and multitask approach to optimize query-product relevance. This paper investigates the capabilities of GenAI, particularly Large Language Models (LLMs), in generating high-quality synthetic data, and analyzing its impact on enhancing EBR models. We conducted experiments using eight Llama models and 100 million data points from Facebook Marketplace logs. Our synthetic data generation follows three strategies: (1) generate queries, (2) enhance product listings, and (3) generate queries from enhanced listings. We train EBR models on three different datasets: sampled engagement data or original data ((e.g., "Click" and "Listing Interactions")), synthetic data, and a mixture of both engagement and synthetic data to assess their performance across various training sets. Our findings underscore the robustness of Llama models in producing synthetic queries and listings with high coherence, relevance, and diversity, while maintaining low levels of hallucination. Aug2Search achieves an improvement of up to 4% in ROC_AUC with 100 million synthetic data samples, demonstrating the effectiveness of our approach. Moreover, our experiments reveal that with the same volume of training data, models trained exclusively on synthetic data often outperform those trained on original data only or a mixture of original and synthetic data.

cs.IR↗

arXiv2024

Reasoner Outperforms: Generative Stance Detection with Rationalization for Social Media

Stance detection is crucial for fostering a human-centric Web by analyzing user-generated content to identify biases and harmful narratives that undermine trust. With the development of Large Language Models (LLMs), existing approaches treat stance detection as a classification problem, providing robust methodologies for modeling complex group interactions and advancing capabilities in natural language tasks. However, these methods often lack interpretability, limiting their ability to offer transparent and understandable justifications for predictions. This study adopts a generative approach, where stance predictions include explicit, interpretable rationales, and integrates them into smaller language models through single-task and multitask learning. We find that incorporating reasoning into stance detection enables the smaller model (FlanT5) to outperform GPT-3.5's zero-shot performance, achieving an improvement of up to 9.57%. Moreover, our results show that reasoning capabilities enhance multitask learning performance but may reduce effectiveness in single-task settings. Crucially, we demonstrate that faithful rationales improve rationale distillation into SLMs, advancing efforts to build interpretable, trustworthy systems for addressing discrimination, fostering trust, and promoting equitable engagement on social media.

cs.CL↗

arXiv2024

A Benchmark for Cross-Domain Argumentative Stance Classification on Social Media

Argumentative stance classification plays a key role in identifying authors' viewpoints on specific topics. However, generating diverse pairs of argumentative sentences across various domains is challenging. Existing benchmarks often come from a single domain or focus on a limited set of topics. Additionally, manual annotation for accurate labeling is time-consuming and labor-intensive. To address these challenges, we propose leveraging platform rules, readily available expert-curated content, and large language models to bypass the need for human annotation. Our approach produces a multidomain benchmark comprising 4,498 topical claims and 30,961 arguments from three sources, spanning 21 domains. We benchmark the dataset in fully supervised, zero-shot, and few-shot settings, shedding light on the strengths and limitations of different methodologies. We release the dataset and code in this study at hidden for anonymity.

cs.CL↗

arXiv2023

Moral Sparks in Social Media Narratives

There is increasing interest in building computational models of moral reasoning by people to enable effective interaction by Artificial Intelligence (AI) agents. We examine interactions on social media to understand human moral judgments in real-life ethical scenarios. Specifically, we examine posts from a popular Reddit subreddit (i.e., a subcommunity) called r/AmITheAsshole, where authors and commenters share their moral judgments on who (i.e., which participant of the described scenario) is blameworthy. To investigate the underlying reasoning influencing moral judgments, we focus on excerpts-which we term moral sparks-from original posts that some commenters include to indicate what motivates their judgments. To this end, we examine how (1) events activating social commonsense and (2) linguistic signals affect the identified moral sparks and their subsequent judgments. By examining over 24672 posts and 175988 comments, we find that event-related negative character traits (e.g., immature and rude) attract attention and stimulate blame, implying a dependent relationship between character traits and moral values. Specifically, we focus on causal graphs involving events (c-events) that activate social commonsense. We observe that c-events are perceived with varying levels of informativeness, influencing moral spark and judgment assignment in distinct ways. This observation is reinforced by examining linguistic features describing semantically similar c-events. Moreover, language influencing commenters' cognitive processes enhances the probability of an excerpt becoming a moral spark, while factual and concrete descriptions tend to inhibit this effect.

cs.SI↗

arXiv2023

The blame game: Understanding blame assignment in social media

Cognitive and psychological studies on morality have proposed underlying linguistic and semantic factors. However, laboratory experiments in the philosophical literature often lack the nuances and complexity of real life. This paper examines how well the findings of these cognitive studies generalize to a corpus of over 30,000 narratives of tense social situations submitted to a popular social media forum. These narratives describe interpersonal moral situations or misgivings; other users judge from the post whether the author (protagonist) or the opposing side (antagonist) is morally culpable. Whereas previous work focuses on predicting the polarity of normative behaviors, we extend and apply natural language processing (NLP) techniques to understand the effects of descriptions of the people involved in these posts. We conduct extensive experiments to investigate the effect sizes of features to understand how they affect the assignment of blame on social media. Our findings show that aggregating psychology theories enables understanding real-life moral situations. Moreover, our results suggest that there exist biases in blame assignment on social media, such as males are more likely to receive blame no matter whether they are protagonists or antagonists.

cs.SI↗

arXiv2023

Morality in the mundane: Categorizing moral reasoning in real-life social situations

Moral reasoning reflects how people acquire and apply moral rules in particular situations. With increasingly social interactions happening online, social media data provides an unprecedented opportunity to assess in-the-wild moral reasoning. We investigate the commonsense aspects of morality in ordinary matters empirically. To this end, we examine data from a Reddit subcommunity (i.e., a subreddit) where an author may describe their behavior in a situation to seek comments about whether that behavior was appropriate. Other users comment to provide judgments and reasoning. We focus on the novel problem of understanding the moral reasoning implicit in user comments about the propriety of an author's behavior. Especially, we explore associations between the common elements of the indicated reasoning and the extractable social factors. Our results suggest the reasoning depends on the author's gender and the topic of a post, such as when expressing anger emotion and using sensible words (e.g., f-ck, hell, and damn) in work-related situations. Moreover, we find that the commonly expressed semantics also depends on commenters' interests.

cs.SI↗