arXiv subjects

Yiqing Cai

Publications and source records attributed to Yiqing Cai.

4 recordsLinked to original sources

arXiv2024

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models

The deployment of large language models (LLMs) faces considerable challenges concerning resource constraints and inference efficiency. Recent research has increasingly focused on smaller, task-specific models enhanced by distilling knowledge from LLMs. However, prior studies have often overlooked the diversity and quality of knowledge, especially the untapped potential of negative knowledge. Constructing effective negative knowledge remains severely understudied. In this paper, we introduce a novel framework called quality-guided contrastive rationale distillation aimed at enhancing reasoning capabilities through contrastive knowledge learning. For positive knowledge, we enrich its diversity through temperature sampling and employ self-consistency for further denoising and refinement. For negative knowledge, we propose an innovative self-adversarial approach that generates low-quality rationales by sampling previous iterations of smaller language models, embracing the idea that one can learn from one's own weaknesses. A contrastive loss is developed to distill both positive and negative knowledge into smaller language models, where an online-updating discriminator is integrated to assess qualities of rationales and assign them appropriate weights, optimizing the training process. Through extensive experiments across multiple reasoning tasks, we demonstrate that our method consistently outperforms existing distillation techniques, yielding higher-quality rationales.

cs.CL↗

arXiv2024

GroundingGPT:Language Enhanced Multi-modal Grounding Model

Multi-modal large language models have demonstrated impressive performance across various tasks in different modalities. However, existing multi-modal models primarily emphasize capturing global information within each modality while neglecting the importance of perceiving local information across modalities. Consequently, these models lack the ability to effectively understand the fine-grained details of input data, limiting their performance in tasks that require a more nuanced understanding. To address this limitation, there is a compelling need to develop models that enable fine-grained understanding across multiple modalities, thereby enhancing their applicability to a wide range of tasks. In this paper, we propose GroundingGPT, a language enhanced multi-modal grounding model. Beyond capturing global information like other multi-modal models, our proposed model excels at tasks demanding a detailed understanding of local information within the input. It demonstrates precise identification and localization of specific regions in images or moments in videos. To achieve this objective, we design a diversified dataset construction pipeline, resulting in a multi-modal, multi-granularity dataset for model training. The code, dataset, and demo of our model can be found at https: //github.com/lzw-lzw/GroundingGPT.

cs.CV↗

arXiv2015

Two-Dimensional Pursuit-Evasion in a Compact Domain with Piecewise Analytic Boundary

In a pursuit-evasion game, a team of pursuers attempt to capture an evader. The players alternate turns, move with equal speed, and have full information about the state of the game. We consider the most restictive capture condition: a pursuer must become colocated with the evader to win the game. We prove two general results about pursuit-evasion games in topological spaces. First, we show that one pursuer has a winning strategy in any CAT(0) space under this restrictive capture criterion. This complements a result of Alexander, Bishop and Ghrist, who provide a winning strategy for a game with positive capture radius. Second, we consider the game played in a compact domain in Euclidean two-space with piecewise analytic boundary and arbitrary Euler characteristic. We show that three pursuers always have a winning strategy by extending recent work of Bhadauria, Klein, Isler and Suri from polygonal environments to our more general setting.

math.MG↗

arXiv2012

Cyclic Network Automata and Cohomological Waves

This paper considers a dynamic coverage problem for sensor networks that are sufficiently dense but not localized. Only a small fraction of sensors may be in an awake state at any given time. The goal is to find a decentralized protocol for establishing dynamic, sweeping barriers of awake-state sensors. Following Baryshnikov-Coffman-Kwak, we use network cyclic cellular automata to generate waves. This paper gives a rigorous analysis of network-based cyclic cellular automata in the context of a system of narrow hallways and shows that waves of awake-state nodes turn corners and automatically solve pusuit/evasion-type problems without centralized coordination. As a corollary of this work, we unearth some interesting topological interpretations of features previously observed in cyclic cellular automata (CCA). By considering CCA over networks and completing to simplicial complexes, we induce dynamics on the higher-dimensional complex. In this setting, waves are seen to be generated by topological defects with a nontrivial degree (or winding number). The simplicial complex has the topological type of the underlying map of the workspace (a subset of the plane), and the resulting waves can be classified cohomologically. This allows one to "program" pulses in the sensor network according to cohomology class. We give a realization theorem for such pulse waves.

math.AT↗