arXiv subjects

Shicheng Xu

Publications and source records attributed to Shicheng Xu.

At least 37 records · Page 2Linked to original sources

arXiv2024

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating additional information from retrieval. However, studies have shown that LLMs still face challenges in effectively using the retrieved information, even ignoring it or being misled by it. The key reason is that the training of LLMs does not clearly make LLMs learn how to utilize input retrieved texts with varied quality. In this paper, we propose a novel perspective that considers the role of LLMs in RAG as ``Information Refiner'', which means that regardless of correctness, completeness, or usefulness of retrieved texts, LLMs can consistently integrate knowledge within the retrieved texts and model parameters to generate the texts that are more concise, accurate, and complete than the retrieved texts. To this end, we propose an information refinement training method named InFO-RAG that optimizes LLMs for RAG in an unsupervised manner. InFO-RAG is low-cost and general across various tasks. Extensive experiments on zero-shot prediction of 11 datasets in diverse tasks including Question Answering, Slot-Filling, Language Modeling, Dialogue, and Code Generation show that InFO-RAG improves the performance of LLaMA2 by an average of 9.39\% relative points. InFO-RAG also shows advantages in in-context learning and robustness of RAG.

cs.CL↗

arXiv2024

Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Images

With the advancement of generation models, AI-generated content (AIGC) is becoming more realistic, flooding the Internet. A recent study suggests that this phenomenon causes source bias in text retrieval for web search. Specifically, neural retrieval models tend to rank generated texts higher than human-written texts. In this paper, we extend the study of this bias to cross-modal retrieval. Firstly, we successfully construct a suitable benchmark to explore the existence of the bias. Subsequent extensive experiments on this benchmark reveal that AI-generated images introduce an invisible relevance bias to text-image retrieval models. Specifically, our experiments show that text-image retrieval models tend to rank the AI-generated images higher than the real images, even though the AI-generated images do not exhibit more visually relevant features to the query than real images. This invisible relevance bias is prevalent across retrieval models with varying training data and architectures. Furthermore, our subsequent exploration reveals that the inclusion of AI-generated images in the training data of the retrieval models exacerbates the invisible relevance bias. The above phenomenon triggers a vicious cycle, which makes the invisible relevance bias become more and more serious. To elucidate the potential causes of invisible relevance and address the aforementioned issues, we introduce an effective training method aimed at alleviating the invisible relevance bias. Subsequently, we apply our proposed debiasing method to retroactively identify the causes of invisible relevance, revealing that the AI-generated images induce the image encoder to embed additional information into their representation. This information exhibits a certain consistency across generated images with different semantics and can make the retriever estimate a higher relevance score.

cs.IR↗

arXiv2024

Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks

Making the content generated by Large Language Model (LLM), accurate, credible and traceable is crucial, especially in complex knowledge-intensive tasks that require multi-step reasoning and each step needs knowledge to solve. Retrieval-augmented generation is good potential to solve this problem. However, where and how to introduce Information Retrieval (IR) to LLM is a big challenge. Previous work has the problems that wrong knowledge retrieved by IR misleads the LLM and interaction between IR and LLM breaks the reasoning chain of LLM. This paper proposes a novel framework named \textbf{Search-in-the-Chain} (SearChain) for the interaction between LLM and IR to solve the challenges. First, LLM generates the reasoning chain named Chain-of-Query (CoQ) where each node consists of an IR-oriented query-answer pair. Second, IR verifies the answer of each node of CoQ. It corrects the answer that is not consistent with the retrieved information when IR gives high confidence, which improves the credibility. Third, LLM can indicate its missing knowledge in CoQ and rely on IR to provide this knowledge to LLM. These operations improve the accuracy in terms of reasoning and knowledge. Finally, SearChain generates the reasoning process and marks references to supporting documents for each reasoning step, which improves traceability. Interaction with IR in SearChain forms a novel reasoning path based on a tree, which enables LLM to dynamically modify the direction of reasoning. Experiments show that SearChain outperforms state-of-the-art baselines on complex knowledge-intensive tasks including multi-hop Q\&A, slot filling, fact checking, and long-form Q\&A.

cs.CL↗

arXiv2024

List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation

The results of information retrieval (IR) are usually presented in the form of a ranked list of candidate documents, such as web search for humans and retrieval-augmented generation for large language models (LLMs). List-aware retrieval aims to capture the list-level contextual features to return a better list, mainly including reranking and truncation. Reranking finely re-scores the documents in the list. Truncation dynamically determines the cut-off point of the ranked list to achieve the trade-off between overall relevance and avoiding misinformation from irrelevant documents. Previous studies treat them as two separate tasks and model them separately. However, the separation is not optimal. First, it is hard to share the contextual information of the ranking list between the two tasks. Second, the separate pipeline usually meets the error accumulation problem, where the small error from the reranking stage can largely affect the truncation stage. To solve these problems, we propose a Reranking-Truncation joint model (GenRT) that can perform the two tasks concurrently. GenRT integrates reranking and truncation via generative paradigm based on encoder-decoder architecture. We also design the novel loss functions for joint optimization to make the model learn both tasks. Sharing parameters by the joint model is conducive to making full use of the common modeling information of the two tasks. Besides, the two tasks are performed concurrently and co-optimized to solve the error accumulation problem between separate stages. Experiments on public learning-to-rank benchmarks and open-domain Q\&A tasks show that our method achieves SOTA performance on both reranking and truncation tasks for web search and retrieval-augmented LLMs.

cs.IR↗

arXiv2023

NIR-Prompt: A Multi-task Generalized Neural Information Retrieval Training Framework

Information retrieval aims to find information that meets users' needs from the corpus. Different needs correspond to different IR tasks such as document retrieval, open-domain question answering, retrieval-based dialogue, etc., while they share the same schema to estimate the relationship between texts. It indicates that a good IR model can generalize to different tasks and domains. However, previous studies indicate that state-of-the-art neural information retrieval (NIR) models, e.g, pre-trained language models (PLMs) are hard to generalize. Mainly because the end-to-end fine-tuning paradigm makes the model overemphasize task-specific signals and domain biases but loses the ability to capture generalized essential signals. To address this problem, we propose a novel NIR training framework named NIR-Prompt for retrieval and reranking stages based on the idea of decoupling signal capturing and combination. NIR-Prompt exploits Essential Matching Module (EMM) to capture the essential matching signals and gets the description of tasks by Matching Description Module (MDM). The description is used as task-adaptation information to combine the essential matching signals to adapt to different tasks. Experiments under in-domain multi-task, out-of-domain multi-task, and new task adaptation settings show that NIR-Prompt can improve the generalization of PLMs in NIR for both retrieval and reranking stages compared with baselines.

cs.IR↗

arXiv2023

Plot Retrieval as an Assessment of Abstract Semantic Association

Retrieving relevant plots from the book for a query is a critical task, which can improve the reading experience and efficiency of readers. Readers usually only give an abstract and vague description as the query based on their own understanding, summaries, or speculations of the plot, which requires the retrieval model to have a strong ability to estimate the abstract semantic associations between the query and candidate plots. However, existing information retrieval (IR) datasets cannot reflect this ability well. In this paper, we propose Plot Retrieval, a labeled dataset to train and evaluate the performance of IR models on the novel task Plot Retrieval. Text pairs in Plot Retrieval have less word overlap and more abstract semantic association, which can reflect the ability of the IR models to estimate the abstract semantic association, rather than just traditional lexical or semantic matching. Extensive experiments across various lexical retrieval, sparse retrieval, dense retrieval, and cross-encoder methods compared with human studies on Plot Retrieval show current IR models still struggle in capturing abstract semantic association between texts. Plot Retrieval can be the benchmark for further research on the semantic association modeling ability of IR models.

cs.IR↗

arXiv2023

Quantitative Volume Space Form Rigidity Under Lower Ricci Curvature Bound

Let $M$ be a compact $n$-manifold of $\operatorname{Ric}_M\ge (n-1)H$ ($H$ is a constant). We are concerned with the following space form rigidity: $M$ is isometric to a space form of constant curvature $H$ under either of the following conditions: (i) There is $ρ>0$ such that for any $x\in M$, the open $ρ$-ball at $x^*$ in the (local) Riemannian universal covering space, $(U^*_ρ,x^*)\to (B_ρ(x),x)$, has the maximal volume i.e., the volume of a $ρ$-ball in the simply connected $n$-space form of curvature $H$. (ii) For $H=-1$, the volume entropy of $M$ is maximal i.e. $n-1$ ([LW1]). The main results of this paper are quantitative space form rigidity i.e., statements that $M$ is diffeomorphic and close in the Gromov-Hausdorff topology to a space form of constant curvature $H$, if $M$ almost satisfies, under some additional condition, the above maximal volume condition. For $H=1$, the quantitative spherical space form rigidity improves and generalizes the diffeomorphic sphere theorem in [CC2].

math.DG↗

arXiv2023

Multi-task 3D building understanding with multi-modal pretraining

This paper explores various learning strategies for 3D building type classification and part segmentation on the BuildingNet dataset. ULIP with PointNeXt and PointNeXt segmentation are extended for the classification and segmentation task on BuildingNet dataset. The best multi-task PointNeXt-s model with multi-modal pretraining achieves 59.36 overall accuracy for 3D building type classification, and 31.68 PartIoU for 3D building part segmentation on validation split. The final PointNeXt XL model achieves 31.33 PartIoU and 22.78 ShapeIoU on test split for BuildingNet-Points segmentation, which significantly improved over PointNet++ model reported from BuildingNet paper, and it won the 1st place in the BuildingNet challenge at CVPR23 StruCo3D workshop.

cs.CV↗

arXiv2023

BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval

Dense retrieval has shown promise in the first-stage retrieval process when trained on in-domain labeled datasets. However, previous studies have found that dense retrieval is hard to generalize to unseen domains due to its weak modeling of domain-invariant and interpretable feature (i.e., matching signal between two texts, which is the essence of information retrieval). In this paper, we propose a novel method to improve the generalization of dense retrieval via capturing matching signal called BERM. Fully fine-grained expression and query-oriented saliency are two properties of the matching signal. Thus, in BERM, a single passage is segmented into multiple units and two unit-level requirements are proposed for representation as the constraint in training to obtain the effective matching signal. One is semantic unit balance and the other is essential matching unit extractability. Unit-level view and balanced semantics make representation express the text in a fine-grained manner. Essential matching unit extractability makes passage representation sensitive to the given query to extract the pure matching information from the passage containing complex context. Experiments on BEIR show that our method can be effectively combined with different dense retrieval training methods (vanilla, hard negatives mining and knowledge distillation) to improve its generalization ability without any additional inference overhead and target domain data.

cs.IR↗

arXiv2022

Canonical nilpotent structure under bounded Ricci curvature and Reifenberg local covering geometry over regular limits

It is known that a closed collapsed Riemannian $n$-manifold $(M,g)$ of bounded Ricci curvature and Reifenberg local covering geometry admits a nilpotent structure in the sense of Cheeger-Fukaya-Gromov with respect to a smoothed metric $g(t)$. We prove that a canonical nilpotent structure over a regular limit space that describes the collapsing of original metric $g$ can be defined and uniquely determined up to a conjugation, and prove that the nilpotent structures arising from nearby metrics $g_ε$ with respect to $g_ε$'s sectional curvature bound are equivalent to the canonical one.

math.DG↗

arXiv2022

Quantitative rigidity of almost maximal volume entropy for both RCD spaces and integral Ricci curvature bound

The volume entropy of a compact metric measure space is known to be the exponential growth rate of the measure lifted to its universal cover at infinity. For a compact Riemannian $n$-manifold with a negative lower Ricci curvature bound and a upper diameter bound, it was known that it admits an almost maximal volume entropy if and only if it is diffeomorphic and Gromov-Hausdorff close to a hyperbolic space form. We prove the quantitative rigidity of almost maximal volume entropy for $\operatorname{RCD}$-spaces with a negative lower Ricci curvature bound and Riemannian manifolds with a negative $L^p$-integral Ricci curvature lower bound.

math.DG↗

arXiv2022

Match-Prompt: Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning

Text matching is a fundamental technique in both information retrieval and natural language processing. Text matching tasks share the same paradigm that determines the relationship between two given texts. The relationships vary from task to task, e.g.~relevance in document retrieval, semantic alignment in paraphrase identification and answerable judgment in question answering. However, the essential signals for text matching remain in a finite scope, i.e.~exact matching, semantic matching, and inference matching. Ideally, a good text matching model can learn to capture and aggregate these signals for different matching tasks to achieve competitive performance, while recent state-of-the-art text matching models, e.g.~Pre-trained Language Models (PLMs), are hard to generalize. It is because the end-to-end supervised learning on task-specific dataset makes model overemphasize the data sample bias and task-specific signals instead of the essential matching signals. To overcome this problem, we adopt a specialization-generalization training strategy and refer to it as Match-Prompt. In specialization stage, descriptions of different matching tasks are mapped to a few prompt tokens. In generalization stage, matching model explores the essential matching signals by being trained on diverse matching tasks. High diverse matching tasks avoid model fitting the data bias on a specific task, so that model can focus on learning the essential matching signals. Meanwhile, the prompt tokens obtained in the first step help the model distinguish different task-specific matching signals. Experimental results on public datasets show that Match-Prompt can improve multi-task generalization capability of PLMs in text matching and yield better in-domain multi-task, out-of-domain multi-task and new task adaptation performance than multi-task and task-specific models trained by previous fine-tuning paradigm.

cs.IR↗

arXiv2022

Convergence of Ricci-limit spaces under bounded Ricci curvature and local covering geometry I

We extend Cheeger-Gromov's and Anderson's convergence theorems to regular limit spaces of manifolds with bounded Ricci curvature and local covering geometry, by establishing the $C^{1,α}$-regularities that are the best one may expect on those Ricci-limit spaces. As an application we prove an optimal generalization of Fukaya's fibration theorem on collapsed manifolds with bounded Ricci curvature, which also improves the original version to $C^{1,α}$ limit spaces.

math.DG↗

arXiv2021

Total squared mean curvature of immersed submanifolds in a negatively curved space

Let $n\ge 2$ and $k\ge 1$ be two integers. Let $M$ be an isometrically immersed closed $n$-submanifold of co-dimension $k$ that is homotopic to a point in a complete manifold $N$, where the sectional curvature of $N$ is no more than $δ<0$. We prove that the total squared mean curvature of $M$ in $N$ and the first non-zero eigenvalue $λ_1(M)$ of $M$ satisfies $$λ_1(M)\le n\left(δ+\frac{1}{\operatorname{Vol} M}\int_M |H|^2 \operatorname{dvol}\right).$$ The equality implies that $M$ is minimally immersed in a metric sphere after lifted to the universal cover of $N$. This completely settles an open problem raised by E. Heintze in 1988.

math.DG↗

arXiv2019

Recognizing shape via 1st eigenvalue, mean curvature and upper curvature bound

Let $M^n$ be a closed immersed hypersurface lying in a contractible ball $B(p,R)$ of the ambient $(n+1)$-manifold $N^{n+1}$. We prove that, by pinching Heintze-Reilly's inequality via sectional curvature upper bound of $B(p,R)$, 1st eigenvalue and mean curvature of $M$, not only $M$ is Hausdorff close to a geodesic sphere $S(p_0,R_0)$ in $N$, but also the ``enclosed'' ball $B(p_0,R_0)$ is close to be of constant curvature, provided with a uniform control on the volume and mean curvature of $M$. We raise a conjecture for $M$ to be a diffeomorphic sphere, and give some positive partial answer.

math.DG↗

arXiv2019

1st eigenvalue pinching for convex hypersurfaces in a Riemannian manifold

Let $M^n$ be a closed convex hypersurface lying in a convex ball $B(p,R)$ of the ambient $(n+1)$-manifold $N^{n+1}$. We prove that, by pinching Heintze-Reilly's inequality via sectional curvature upper bound of $B(p,R)$, 1st eigenvalue and mean curvature of $M$, not only $M$ is Hausdorff close and almost isometric to a geodesic sphere $S(p_0,R_0)$ in $N$, but also its enclosed domain is $C^{1,α}$-close to a geodesic ball of constant curvature.

math.DG↗

arXiv2019

Margulis lemma and Hurewicz fibration Theorem on Alexandrov spaces

We prove the generalized Margulis lemma with a uniform index bound on an Alexandrov $n$-space $X$ with curvature bounded below, i.e., small loops at $p\in X$ generate a subgroup of the fundamental group of unit ball $B_1(p)$ that contains a nilpotent subgroup of index $\le w(n)$, where $w(n)$ is a constant depending only on the dimension $n$. The proof is based on the main ideas of V.~Kapovitch, A.~Petrunin, and W.~Tuschmann, and the following results: (1) We prove that any regular almost Lipschitz submersion constructed by Yamaguchi on a collapsed Alexandrov space with curvature bounded below is a Hurewicz fibration. We also prove that such fibration is uniquely determined up to a homotopy equivalence. (2) We give a detailed proof on the gradient push, improving the universal pushing time bound given by V.~Kapovitch, A.~Petrunin, and W.~Tuschmann, and justifying in a specific way that the gradient push between regular points can always keep away from extremal subsets.

math.DG↗

arXiv2018

Collapsed Manifolds With Ricci Bounded Covering Geometry

We study collapsed manifolds with Ricci bounded covering geometry i.e., Ricci curvature is bounded below and the Riemannian universal cover is non-collapsed or consists of uniform Reifenberg points. Via Ricci flows' techniques, we partially extend the nilpotent structural results of Cheeger-Fukaya-Gromov, on collapsed manifolds with (sectional curvature) local bounded covering geometry, to manifolds with (global) Ricci boundedcovering geometry.

math.DG↗