arXiv subjects

Qingyi Liu

Publications and source records attributed to Qingyi Liu.

4 recordsLinked to original sources

arXiv2026

Circle Pattern Theorem for Quasi-simplicial Triangulated Surfaces

The Circle Pattern Theorem characterizes the existence and rigidity of circle patterns with prescribed intersection angles on simplicial triangulations of closed surfaces. In this paper we extend the theorem to quasi-simplicial triangulations -- triangulations that may contain loops and multiple edges, but whose lifts to the universal cover are simplicial. Chow and Luo first considered such triangulations -- under the name \emph{generalized triangulations} (J.~Differential Geom.~\textbf{63}(1):97--129, 2003) -- but with the strong restriction that any three vertices determine at most one triangle; this condition keeps the combinatorics within the simplicial complex framework and consequently excludes most quasi-simplicial triangulations. We remove this restriction, work instead with the more flexible framework of Delta complexes, and use a finite covering technique to reduce the problem to the simplicial case. We prove that the curvature image is completely characterized by KAT inequalities imposed on all subsets of the lifted vertex set.

math.GT↗

arXiv2024

Autoregressive Pre-Training on Pixels and Texts

The integration of visual and textual information represents a promising direction in the advancement of language models. In this paper, we explore the dual modality of language--both visual and textual--within an autoregressive framework, pre-trained on both document images and texts. Our method employs a multimodal training strategy, utilizing visual data through next patch prediction with a regression head and/or textual data through next token prediction with a classification head. We focus on understanding the interaction between these two modalities and their combined impact on model performance. Our extensive evaluation across a wide range of benchmarks shows that incorporating both visual and textual data significantly improves the performance of pixel-based language models. Remarkably, we find that a unidirectional pixel-based model trained solely on visual data can achieve comparable results to state-of-the-art bidirectional models on several language understanding tasks. This work uncovers the untapped potential of integrating visual and textual modalities for more effective language modeling. We release our code, data, and model checkpoints at \url{https://github.com/ernie-research/pixelgpt}.

cs.CL↗

arXiv2024

On Training Data Influence of GPT Models

Amidst the rapid advancements in generative language models, the investigation of how training data shapes the performance of GPT models is still emerging. This paper presents GPTfluence, a novel approach that leverages a featurized simulation to assess the impact of training examples on the training dynamics of GPT models. Our approach not only traces the influence of individual training instances on performance trajectories, such as loss and other key metrics, on targeted test points but also enables a comprehensive comparison with existing methods across various training scenarios in GPT models, ranging from 14 million to 2.8 billion parameters, across a range of downstream tasks. Contrary to earlier methods that struggle with generalization to new data, GPTfluence introduces a parameterized simulation of training dynamics, demonstrating robust generalization capabilities to unseen training data. This adaptability is evident across both fine-tuning and instruction-tuning scenarios, spanning tasks in natural language understanding and generation. We make our code and data publicly available at https://github.com/ernie-research/gptfluence.

cs.CL↗

arXiv2022

UBARv2: Towards Mitigating Exposure Bias in Task-Oriented Dialogs

This paper studies the exposure bias problem in task-oriented dialog systems, where the model's generated content over multiple turns drives the dialog context away from the ground-truth distribution at training time, introducing error propagation and damaging the robustness of the TOD system. To bridge the gap between training and inference for multi-turn task-oriented dialogs, we propose session-level sampling which explicitly exposes the model to sampled generated content of dialog context during training. Additionally, we employ a dropout-based consistency regularization with the masking strategy R-Mask to further improve the robustness and performance of the model. The proposed UBARv2 achieves state-of-the-art performance on the standardized evaluation benchmark MultiWOZ and extensive experiments show the effectiveness of the proposed methods.

cs.CL↗