Searcharxiv⌕ Search

arXiv subjects

Qingfeng Sun

Publications and source records attributed to Qingfeng Sun.

At least 37 records · Page 2Linked to original sources
arXiv2024

Towards a Unified Paradigm: Integrating Recommendation Systems as a New Language in Large Models

This paper explores the use of Large Language Models (LLMs) for sequential recommendation, which predicts users' future interactions based on their past behavior. We introduce a new concept, "Integrating Recommendation Systems as a New Language in Large Models" (RSLLM), which combines the strengths of traditional recommenders and LLMs. RSLLM uses a unique prompting method that combines ID-based item embeddings from conventional recommendation models with textual item features. It treats users' sequential behaviors as a distinct language and aligns the ID embeddings with the LLM's input space using a projector. We also propose a two-stage LLM fine-tuning framework that refines a pretrained LLM using a combination of two contrastive losses and a language modeling loss. The LLM is first fine-tuned using text-only prompts, followed by target domain fine-tuning with unified prompts. This trains the model to incorporate behavioral knowledge from the traditional sequential recommender into the LLM. Our empirical results validate the effectiveness of our proposed framework.

cs.IR↗
arXiv2024

On an unconditional spectral analog of Selberg's result on $S(t)$

Let $S_j(t)=\frac{1}π\arg L(1/2+it, u_j)$, where $u_j$ is an even Hecke--Maass cusp form for $\rm SL_2(\mathbb{Z})$ with Laplacian eigenvalue $λ_j=\frac{1}{4}+t_j^2$. Without assuming the GRH, we establish an asymptotic formula for the moments of $S_j(t)$.

math.NT↗
arXiv2024

Higher order divisor functions over values of mixed powers

Let $τ_k(n)$ be the $k$-th divisor function. In this paper, we derive an asymptotic formula for the sum $$ \sum_{1\leq n_1,n_2, \dots, n_{\ell}\leq X^{\frac{1}{r}} \atop 1\leq n_{\ell+1}\le X^{\frac{1}{s}}}τ_k(n_1^r+n_2^r+\dots +n_{\ell}^r+n_{\ell+1}^s), $$ where $k\geq 4$, $r\geq 2$, $s\geq 2$ and $\ell\geq 2$ are integers. Previously only special cases are studied.

math.NT↗
arXiv2024

Mass Distribution for holomorphic cusp forms on the vertical geodesic

We compute the quantum variance of holomorphic cusp forms on the vertical geodesic for smooth compactly supported test functions. As an application we show that almost all holomorphic Hecke cusp forms, whose weights are in a short interval, satisfy QUE conjecture on the vertical geodesic.

math.NT↗
arXiv2024

Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena

Assessing the effectiveness of large language models (LLMs) presents substantial challenges. The method of conducting human-annotated battles in an online Chatbot Arena is a highly effective evaluative technique. However, this approach is limited by the costs and time required for human annotation. In this paper, we introduce Arena Learning, an innovative offline strategy designed to simulate these arena battles using AI-driven annotations to evaluate battle outcomes, thus facilitating the continuous improvement of the target model through both supervised fine-tuning and reinforcement learning. Arena Learning comprises two key elements. First, it ensures precise evaluations and maintains consistency between offline simulations and online competitions via WizardArena, a pipeline developed to accurately predict the Elo rankings of various models using a meticulously designed offline test set. Our results demonstrate that WizardArena's predictions closely align with those from the online Arena. Second, it involves the continuous improvement of training data based on the battle results and the refined model. We establish a data flywheel to iteratively update the training data by highlighting the weaknesses of the target model based on its battle results, enabling it to learn from the strengths of multiple different models. We apply Arena Learning to train our target model, WizardLM-$β$, and demonstrate significant performance enhancements across various metrics. This fully automated training and evaluation pipeline sets the stage for continuous advancements in various LLMs via post-training. Notably, Arena Learning plays a pivotal role in the success of WizardLM-2, and this paper serves both as an exploration of its efficacy and a foundational study for future discussions related to WizardLM-2 and its derivatives.

cs.CL↗
arXiv2023

Self-Supervised Multi-Modal Sequential Recommendation

With the increasing development of e-commerce and online services, personalized recommendation systems have become crucial for enhancing user satisfaction and driving business revenue. Traditional sequential recommendation methods that rely on explicit item IDs encounter challenges in handling item cold start and domain transfer problems. Recent approaches have attempted to use modal features associated with items as a replacement for item IDs, enabling the transfer of learned knowledge across different datasets. However, these methods typically calculate the correlation between the model's output and item embeddings, which may suffer from inconsistencies between high-level feature vectors and low-level feature embeddings, thereby hindering further model learning. To address this issue, we propose a dual-tower retrieval architecture for sequence recommendation. In this architecture, the predicted embedding from the user encoder is used to retrieve the generated embedding from the item encoder, thereby alleviating the issue of inconsistent feature levels. Moreover, in order to further improve the retrieval performance of the model, we also propose a self-supervised multi-modal pretraining method inspired by the consistency property of contrastive learning. This pretraining method enables the model to align various feature combinations of items, thereby effectively generalizing to diverse datasets with different item features. We evaluate the proposed method on five publicly available datasets and conduct extensive experiments. The results demonstrate significant performance improvement of our method.

cs.IR↗
arXiv2022

MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation

Responding with multi-modal content has been recognized as an essential capability for an intelligent conversational agent. In this paper, we introduce the MMDialog dataset to better facilitate multi-modal conversation. MMDialog is composed of a curated set of 1.08 million real-world dialogues with 1.53 million unique images across 4,184 topics. MMDialog has two main and unique advantages. First, it is the largest multi-modal conversation dataset by the number of dialogues by 88x. Second, it contains massive topics to generalize the open-domain. To build engaging dialogue system with this dataset, we propose and normalize two response producing tasks based on retrieval and generative scenarios. In addition, we build two baselines for above tasks with state-of-the-art techniques and report their experimental performance. We also propose a novel evaluation metric MM-Relevance to measure the multi-modal responses. Our dataset and scripts are available in https://github.com/victorsungo/MMDialog.

cs.CL↗
arXiv2022

A bound for twists of $\rm GL_3\times GL_2$ $L$-functions with composite modulus

Let $π$ be a Hecke-Maass cusp form for $\rm SL_3(\mathbf{Z})$ and let $g$ be a holomorphic or Maass cusp form for $\rm SL_2(\mathbf{Z})$. Let $χ$ be a primitive Dirichlet character of modulus $M=M_1M_2$ with $M_i$ prime, $i=1,2$. Suppose that $M^{1/2+2η}<M_1<M^{1-2η}$ with $0<η<1/8$. Then we have $$ L\left(\frac{1}{2},π\otimes g \otimes χ\right)\ll_{π,g,\varepsilon} M^{3/2-η+\varepsilon}. $$

math.NT↗
arXiv2022

Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting

Current Knowledge-Grounded Dialogue Generation (KDG) models specialize in producing rational and factual responses. However, to establish long-term relationships with users, the KDG model needs the capability to generate responses in a desired style or attribute. Thus, we study a new problem: Stylized Knowledge-Grounded Dialogue Generation (SKDG). It presents two challenges: (1) How to train a SKDG model where no triples are available. (2) How to cohere with context and preserve the knowledge when generating a stylized response. In this paper, we propose a novel disentangled template rewriting (DTR) method which generates responses via combing disentangled style templates (from monolingual stylized corpus) and content templates (from KDG corpus). The entire framework is end-to-end differentiable and learned without supervision. Extensive experiments on two benchmarks indicate that DTR achieves a significant improvement on all evaluation metrics compared with previous state-of-the-art stylized dialogue generation methods. Besides, DTR achieves comparable performance with the state-of-the-art KDG methods in standard KDG evaluation setting.

cs.CL↗
arXiv2022

Multimodal Dialogue Response Generation

Responsing with image has been recognized as an important capability for an intelligent conversational agent. Yet existing works only focus on exploring the multimodal dialogue models which depend on retrieval-based methods, but neglecting generation methods. To fill in the gaps, we first present a multimodal dialogue generation model, which takes the dialogue history as input, then generates a textual sequence or an image as response. Learning such a model often requires multimodal dialogues containing both texts and images which are difficult to obtain. Motivated by the challenge in practice, we consider multimodal dialogue generation under a natural assumption that only limited training examples are available. In such a low-resource setting, we devise a novel conversational agent, Divter, in order to isolate parameters that depend on multimodal dialogues from the entire generation model. By this means, the major part of the model can be learned from a large number of text-only dialogues and text-image pairs respectively, then the whole parameters can be well fitted using the limited training examples. Extensive experiments demonstrate our method achieves state-of-the-art results in both automatic and human evaluation, and can generate informative text and high-resolution image responses.

cs.CL↗
arXiv2022

PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks

This paper focuses on the Data Augmentation for low-resource Natural Language Understanding (NLU) tasks. We propose Prompt-based D}ata Augmentation model (PromDA) which only trains small-scale Soft Prompt (i.e., a set of trainable vectors) in the frozen Pre-trained Language Models (PLMs). This avoids human effort in collecting unlabeled in-domain data and maintains the quality of generated synthetic data. In addition, PromDA generates synthetic data via two different views and filters out the low-quality data using NLU models. Experiments on four benchmarks show that synthetic data produced by PromDA successfully boost up the performance of NLU models which consistently outperform several competitive baseline models, including a state-of-the-art semi-supervised model using unlabeled in-domain data. The synthetic data from PromDA are also complementary with unlabeled in-domain data. The NLU models can be further improved when they are combined for training.

cs.CL↗
arXiv2022

On some estimates involving Fourier coefficients of Maass cusp forms

Let $f$ be a Hecke-Maass cusp form for $\rm SL_2(\mathbb{Z})$ with Laplace eigenvalue $λ_f(Δ)=1/4+μ^2$ and let $λ_f(n)$ be its $n$-th normalized Fourier coefficient. It is proved that, uniformly in $α, β\in \mathbb{R}$, $$ \sum_{n \leq X}λ_f(n)e\left(αn^2+βn\right) \ll X^{7/8+\varepsilon}λ_f(Δ)^{1/2+\varepsilon}, $$ where the implied constant depends only on $\varepsilon$. We also consider the summation function of $λ_f(n)$ and under the Ramanujan conjecture we are able to prove $$ \sum_{n \leq X}λ_f(n)\ll X^{1/3+\varepsilon}λ_f(Δ)^{4/9+\varepsilon} $$ with the implied constant depending only on $\varepsilon$.

math.NT↗
arXiv2021

Uniform subconvex bounds for Rankin-Selberg $L$-functions

Let $f$ be a Maass cusp form for $\rm SL_2(\mathbb{Z})$ with Laplace eigenvalue $1/4+μ_f^2$, $μ_f>0$. Let $g$ be an arbitrary but fixed holomorphic or Maass cusp form for $\rm SL_2(\mathbb{Z})$. In this paper, we establish the following uniform subconvexity bound for the Rankin-Selberg $L$-function $L(s,f\otimes g)$ $$ L\left(1/2+it,f\otimes g\right)\ll (μ_f+|t|)^{9/10+\varepsilon}, $$ where the implied constant depends only on $\varepsilon$ and $g$.

math.NT↗
arXiv2021

Analytic twists of $\rm GL_2\times\rm GL_2$ automorphic forms

Let $f$ and $g$ be holomorphic or Maass cusp forms for $\rm SL_2(\mathbb{Z})$ with normalized Fourier coefficients $λ_f(n)$ and $λ_g(n)$, respectively. In this paper, we prove nontrivial estimates for the sum $$ \sum_{n=1}^{\infty}λ_f(n) λ_g(n)e\left(t φ\left(\frac{n}{X}\right)\right)V\left(\frac{n}{X}\right), $$ where $e(x)=e^{2πix}$, $V(x)\in \mathcal{C}_c^{\infty}(1,2)$, $t\geq 1$ is a large parameter and $φ(x)$ is some nonlinear real valued smooth function. Applications of these estimates include a subconvex bound for the Rankin-Selberg $L$-function $L(s,f\otimes g)$ in the $t$-aspect, an improved estimate for a nonlinear exponential twisted sum and the following asymptotic formula for the sum of the Fourier coefficients of certain $\rm{GL}_5$ Eisenstein series $$ \sum_{n \leq X}λ_{1\boxplus(f\times g)}(n) =L(1,f\times g)X + O(X^{\frac{2}{3}-\frac{1}{356}+\varepsilon}) $$ for any $\varepsilon>0$.

math.NT↗
arXiv2021

Analytic twists of $\rm GL_3\times \rm GL_2$ automorphic forms

Let $π$ be a Hecke--Maass cusp form for $\rm SL_3(\mathbb{Z})$ with normalized Hecke eigenvalues $λ_π(n,r)$. Let $f$ be a holomorphic or Maass cusp form for $\rm SL_2(\mathbb{Z})$ with normalized Hecke eigenvalues $λ_f(n)$. In this paper, we are concerned with obtaining nontrivial estimates for the sum \begin{equation*} \sum_{r,n\geq 1}λ_π(n,r)λ_f(n)e\left(t\,φ(r^2n/N)\right)V\left(r^2n/N\right), \end{equation*} where $e(x)=e^{2πix}$, $V(x)\in \mathcal{C}_c^{\infty}(0,\infty)$, $t\geq 1$ is a large parameter and $φ(x)$ is some real-valued smooth function. As applications, we give an improved subconvexity bound for $\rm GL_3\times \rm GL_2$ $L$-functions in the $t$-aspect, and under the Ramanujan--Petersson conjecture we derive the following bound for sums of $\rm GL_3\times \rm GL_2$ Fourier coefficients \begin{equation*} \sum_{r^2n\leq x}λ_π(r,n)λ_f(n)\ll_{π, f, \varepsilon} x^{5/7-1/364+\varepsilon} \end{equation*} for any $\varepsilon>0$, which breaks for the first time the barrier $O(x^{5/7+\varepsilon})$ in a work by Friedlander--Iwaniec.

math.NT↗
arXiv2020

Bounds for $\rm GL_2\times GL_2$ $L$-functions in depth aspect

Let $f$ and $g$ be holomorphic or Maass cusp forms for $\rm SL_2(\mathbb{Z})$ and let $χ$ be a primitive Dirichlet character of prime power conductor $\mathfrak{q}=p^κ$ with $p$ prime and $κ>12$. A subconvex bound for the central values of the Rankin-Selberg $L$-functions $L(s,f\otimes g \otimes χ)$ is proved in the depth-aspect $$ L\left(\frac{1}{2},f\otimes g \otimes χ\right)\ll_{f,g,\varepsilon} p^{3/4}\mathfrak{q}^{15/16+\varepsilon}. $$

math.NT↗
arXiv2020

A Bessel $δ$-method and hybrid bounds for $\mathrm{GL}_2$

Let $g$ be a primitive holomorphic or Maass newform for $Γ_0(D)$. In this paper, by studying the Bessel integrals associated to $g$, we prove an asymptotic Bessel $δ$-identity associated to $g$. Among other applications, we prove the following hybrid subconvexity bound $$ L\left(1/2+it,g\otimes χ\right)\ll_{g,\varepsilon} (q(1+|t|))^{\varepsilon}q^{3/8}(1+|t|)^{1/3} $$ for any $\varepsilon>0$, where $χ\bmod q$ is a primitive Dirichlet character with $(q, D)=1$. This improves the previous known result.

math.NT↗