arXiv subjects

Feiyu Yao

Publications and source records attributed to Feiyu Yao.

13 recordsLinked to original sources

arXiv2026

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference

Long-context inference increasingly operates over CPU-resident KV caches, either because decoding-time KV states exceed GPU memory capacity or because disaggregated prefill-decode systems place KV data in host memory. Although block-sparse attention reduces attention cost in this setting, sparsity alone is insufficient for end-to-end efficiency. GPU-only designs remain constrained by PCIe bandwidth and metadata memory overhead, while CPU-GPU hybrid designs still suffer from substantial GPU idle time and bottlenecks in CPU-side top-k selection and sparse attention computation. Fluxion is built on three key insights: output-aware KV budgeting, head-specific and granularity-aware sparse configuration, and cross-device coordinated execution for sparse attention over CPU-resident KV caches. Guided by these insights, Fluxion combines a lightweight head-property predictor, a granularity-budget selector, and a priority-based scheduler to jointly optimize budget allocation, sparse configuration, and CPU-GPU execution overlap. This co-design enables hybrid sparse attention to achieve both accuracy and system efficiency in long-context inference. Across 2 models, 3 benchmarks, and 40 tasks, Fluxion preserves quality well -- the worst average degradation is only -0.26 relative to FULL, while delivering 1.5$\times$-3.7$\times$ speedup over the strongest fixed sparse hybrid baseline, whose KV budget is only 0.05.

cs.LG↗

arXiv2025

Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance

The emergence of text-to-image synthesis (TIS) models has significantly influenced digital image creation by producing high-quality visuals from written descriptions. Yet these models are sensitive on textual prompts, posing a challenge for novice users who may not be familiar with TIS prompt writing. Existing solutions relieve this via automatic prompt expansion or generation from a user query. However, this single-turn manner suffers from limited user-centricity in terms of result interpretability and user interactivity. Thus, we propose DialPrompt, a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. DialPrompt is designed to follow a multi-turn workflow, where in each round of dialogue the model guides user to express their preferences on possible optimization dimensions before generating the final TIS prompt. To achieve this, we mined 15 essential dimensions for high-quality prompts from advanced users and curated a multi-turn dataset. Through training on this dataset, DialPrompt improves user-centricity by allowing users to perceive and control the creation process of TIS prompts. Experiments indicate that DialPrompt improves significantly in user-centricity score compared with existing approaches while maintaining a competitive quality of synthesized images. In our user evaluation, DialPrompt is highly rated by 19 human reviewers (especially novices).

cs.AI↗

arXiv2025

Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge

Log analysis represents a critical sub-domain within AI applications that facilitates automatic approaches to fault and error management of large-scaled software systems, saving labors of traditional manual methods. While existing solutions using large language models (LLMs) show promise, they are limited by a significant domain gap between natural and log languages (the latter contains rich domain-specific tokens such as status codes, IP addresses, resource pathes), which restricts their effectiveness in real-world applications. However, directly adapting general-purpose LLMs to log analysis using raw logs may degrade their performance due to inconsistent token distribution. In this paper, we present a domain adaptation approach that addresses these limitations by integrating interpretable domain knowledge into open-source LLMs through continual pre-training (CPT), which bridges this domain gap by adapting LLMs on interpretable natural texts with log knowledge (instead of raw logs) to reduce distribution discrepancy. To achieve this, we developed NLPLog, a comprehensive dataset containing over 250,000 question-answer pairs on log-related knowledge. Our resulting model, SuperLog, achieves the best performance across four log analysis tasks, with an average accuracy improvement of 12.01% over the second-best model. Ablation study also suggests advantages of domain adaption using interpretable log knowledge over using raw logs.

cs.CL↗

arXiv2025

Learn from the Past: Fast Sparse Indexing for Large Language Model Decoding

As large language models (LLMs) continue to support increasingly longer contexts, the memory demand for key-value (KV) caches during decoding grows rapidly, becoming a critical bottleneck in both GPU memory capacity and PCIe bandwidth. Sparse attention mechanisms alleviate this issue by computing attention weights only for selected key-value pairs. However, their indexing computation typically requires traversing all key vectors, resulting in significant computational and data transfer overhead. To reduce the cost of index retrieval, existing methods often treat each decoding step as an independent process, failing to exploit the temporal correlations embedded in historical decoding information. To this end, we propose LFPS(Learn From the Past for Sparse Indexing), an acceleration method that dynamically constructs sparse indexing candidates based on historical attention patterns. LFPS captures two prevalent trends in decoder attention -vertical patterns (attending to fixed positions) and slash patterns (attending to relative positions) -and incorporates a positional expansion strategy to effectively predict the Top-k indices for the current step. We validate LFPS on challenging long-context benchmarks such as LongBench-RULER, using Llama-3.1-8B-Instruct as the base model. Experimental results show that LFPS achieves up to 22.8$\times$ speedup over full attention and 9.6$\times$ speedup over exact Top-k retrieval on an RTX 4090 GPU and a single CPU core of a Xeon Gold 6430, respectively, while preserving generation accuracy. These results demonstrate that LFPS offers a practical and efficient solution for decoding optimization in long-context LLM inference.

cs.LG↗

arXiv2024

Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective

Estimating 3D full-body pose from sparse sensor data is a pivotal technique employed for the reconstruction of realistic human motions in Augmented Reality and Virtual Reality. However, translating sparse sensor signals into comprehensive human motion remains a challenge since the sparsely distributed sensors in common VR systems fail to capture the motion of full human body. In this paper, we use well-designed Body Pose Graph (BPG) to represent the human body and translate the challenge into a prediction problem of graph missing nodes. Then, we propose a novel full-body motion reconstruction framework based on BPG. To establish BPG, nodes are initially endowed with features extracted from sparse sensor signals. Features from identifiable joint nodes across diverse sensors are amalgamated and processed from both temporal and spatial perspectives. Temporal dynamics are captured using the Temporal Pyramid Structure, while spatial relations in joint movements inform the spatial attributes. The resultant features serve as the foundational elements of the BPG nodes. To further refine the BPG, node features are updated through a graph neural network that incorporates edge reflecting varying joint relations. Our method's effectiveness is evidenced by the attained state-of-the-art performance, particularly in lower body motion, outperforming other baseline methods. Additionally, an ablation study validates the efficacy of each module in our proposed framework.

cs.CV↗

arXiv2023

Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environments

Vision-and-Language Navigation in Continuous Environments (VLN-CE) is a navigation task that requires an agent to follow a language instruction in a realistic environment. The understanding of environments is a crucial part of the VLN-CE task, but existing methods are relatively simple and direct in understanding the environment, without delving into the relationship between language instructions and visual environments. Therefore, we propose a new environment representation in order to solve the above problems. First, we propose an Environment Representation Graph (ERG) through object detection to express the environment in semantic level. This operation enhances the relationship between language and environment. Then, the relational representations of object-object, object-agent in ERG are learned through GCN, so as to obtain a continuous expression about ERG. Sequentially, we combine the ERG expression with object label embeddings to obtain the environment representation. Finally, a new cross-modal attention navigation framework is proposed, incorporating our environment representation and a special loss function dedicated to training ERG. Experimental result shows that our method achieves satisfactory performance in terms of success rate on VLN-CE tasks. Further analysis explains that our method attains better cross-modal matching and strong generalization ability.

cs.CV↗

arXiv2022

Few-shot Domain Adaptation for IMU Denoising

Different application scenarios will cause IMU to exhibit different error characteristics which will cause trouble to robot application. However, most data processing methods need to be designed for specific scenario. To solve this problem, we propose a few-shot domain adaptation method. In this work, a domain adaptation framework is considered for denoising the IMU, a reconstitution loss is designed to improve domain adaptability. In addition, in order to further improve the adaptability in the case of limited data, a few-shot training strategy is adopted. In the experiment, we quantify our method on two datasets (EuRoC and TUM-VI) and two real robots (car and quadruped robot) with three different precision IMUs. According to the experimental results, the adaptability of our framework is verified by t-SNE. In orientation results, our proposed method shows the great denoising performance.

cs.RO↗

arXiv2022

Thermodynamic Geometry of Black Holes Enclosed by a Cavity in Extended Phase Space

Recently, the phase space of black holes in a spherical cavity of radius $r_{B}$ has been extended by introducing a thermodynamic volume $V\equiv4πr_{B}^{3}/3$. In the extended phase space, we consider the thermodynamic geometry, which provides a powerful tool to understand the microscopic structure of black holes, of Reissner-Nordström (RN) black holes in a cavity, as well as that of Reissner-Nordström-AdS black holes. Although the phase structures of the cavity and AdS cases show striking resemblance, we find that there exist significant differences between the thermodynamic geometries of these two cases. In particular, a reentrant transition of the type of the microstructure interactions, i.e., repulsive $\rightarrow$ attractive $\rightarrow$ repulsive with increasing temperature in an isobaric process, is observed for RN black holes in a cavity.

gr-qc↗

arXiv2021

Scalarized Einstein-Maxwell-scalar Black Holes in a Cavity

In this paper, we study the spontaneous scalarization of Reissner-Nordstr{ö}% m (RN) black holes enclosed by a cavity in an Einstein-Maxwell-scalar (EMS) model with non-minimal couplings between the scalar and Maxwell fields. In this model, scalar-free RN black holes in a cavity may induce scalarized black holes due to the presence of a tachyonic instability of the scalar field near the event horizon. We calculate numerically the black hole solutions, and investigate the domain of existence, perturbative stability against spherical perturbations and phase structure. The scalarized solutions in a cavity are always thermodynamically preferred over scalar-free solutions. In addition, a reentrant phase transition, composed of a zeroth-order phase transition and a second-order one, occurs for large enough electric charge $Q$.

gr-qc↗

arXiv2021

Phase Structures and Transitions of Quintessence Surrounding RN Black Holes in a Grand Canonical Ensemble

Considering a grand canonical ensemble, we study the phase structures and transitions of RN black holes surrounded by quintessence dark energy on two different boundary conditions, namely AdS space and a Dirichlet wall. For AdS space, under the condition of fixed temperature and potential, as the temperature increases for lower potential, the black hole undergoes a first-order phase transition, while for higher potential, no phase transition occurs. There are two different regions in the parameter space. For the Dirichlet wall, on which the temperature and potential are fixed and the state parameter of quintessence $ω=-2/3$ is analyzed in detail. Then, three different physically allowed regions in the parameter space of the black hole are well studied. As the temperature rises, a first-order phase transition and a second-order phase transition may occur. In this case, there are nine regions in the parameter space, which is obviously distinct from the case of AdS space.

gr-qc↗

arXiv2020

Joule-Thomson Expansion of Born-Infeld AdS Black Holes

In this paper, the Joule-Thomson expansion of Born-Infeld AdS black holes is studied in the extended phase space, where the cosmological constant is identified with the pressure. The Joule-Thomson coefficient, the inversion curves and the isenthalpic curves are discussed in detail by 4-dimensional black hole. The critical point of Born-Infeld black hole is depicted with varying parameter $β$ and the charge $Q$. In $T-P$ plane, the inversion temperature curves and isenthalpic curves are obtained with different parameter $β$ and the charge $Q$. We find that the missing negative slope is still conserved in Born-Infeld black holes. We also extend our discussion to arbitrary dimension higher than 4. The critical temperature and the minimum of inversion temperature are compared, and the ratio is asymptotically $1/2$ as $Q$ increases or $β\to\infty$ in $D=4$, and reproduce the previous results in higher dimension.

gr-qc↗

arXiv2020

Extended Phase Space Thermodynamics for Dyonic Black Holes with a Power Maxwell Field

In this paper, we investigate the thermodynamics of dyonic black holes with the presence of power Maxwell electromagnetic field in the extended phase space, which includes the cosmological constant $Λ$ as a thermodynamic variable. For a generic power Maxwell black hole with the electric charge and magnetic charge, the equation of state is given as the function of rescaled temperature $\tilde{T}$ in terms of other rescaled variables $ \tilde{r}_{+}$, $\tilde{q}$ and $\tilde{h}$, where $r_{+}$ is the horizon radius, $q$ is the electric charge and $h$ is some magnetic parameter. For some values of $\tilde{q}$ and $\tilde{h}$, the phase structure of the black hole is uniquely determined. Moreover the peculiarity of multiple temperature with some fixed parameter configurations results in more rich phase structures. Focusing on the power Maxwell Lagrangian with $\mathcal{L} \left( s\right) =s^{2}$, we obtain the corresponding phase diagrams in the $ \tilde{q}$-$\tilde{h}$ plane, then analyse the black holes phase structure and critical behaviour. For this case, the critical line is semi-infinite and extends to $\tilde{h}=\infty$. We also examine thermal stabilities of these black holes.

gr-qc↗

arXiv2020

Extended Phase Space Thermodynamics for Black Holes in a Cavity

In this paper, we extend the phase space of black holes enclosed by a spherical cavity of radius $r_{B}$ to include $V\equiv4πr_{B}^{3}/3$ as a thermodynamic volume. The thermodynamic behavior of Schwarzschild and Reissner-Nordstrom (RN) black holes is then investigated in the extended phase space. In a canonical ensemble at constant pressure, we find that the aforementioned thermodynamic behavior is remarkably similar to that of the anti-de Sitter (AdS) counterparts with the cosmological constant being interpreted as a pressure. Specifically, a first-order Hawking-Page-like phase transition occurs for a Schwarzschild black hole in a cavity. The phase structure of a RN black hole in a cavity shows a strong resemblance to that of the van der Waals fluid. Our results may provide a new perspective for the extended thermodynamics of AdS black holes by analogy with black holes in a cavity.

gr-qc↗