arXiv subjects

Yichao Fu

Publications and source records attributed to Yichao Fu.

At least 19 recordsLinked to original sources

arXiv2026

FastKernels: Benchmarking GPU Kernel Generation in Production

LLM-based agents for GPU kernel generation are advancing rapidly, yet their progress is fundamentally constrained by the benchmarks they optimize against. Existing benchmarks are poorly aligned with production inference frameworks: they evaluate kernels on a single GPU with synthetic inputs, ignore the surrounding compilation stack, and reward replicating known optimizations rather than discovering new ones. The resulting reward signals are misleading: agents learn to generate kernels that score well in sandboxes but introduce interface incompatibilities, compilation-stack conflicts, and silent correctness degradation when integrated into real systems. We introduce FastKernels, a kernel benchmark built around a minimal set of 46 representative architectures spanning 8 categories, whose kernels collectively subsume those of 96.2% (409/425) of HuggingFace Transformers architectures. FastKernels doubles as a minimalistic, production-grade inference framework that runs at parity with hardened systems such as vLLM and SGLang on mainstream LLM serving and substantially exceeds upstream references on under-served architectures; each task's interface mirrors the corresponding module in the state-of-the-art library for its architecture family, enabling direct deployment of optimized kernels into production codebases. Evaluating state-of-the-art kernel agents on FastKernels, we find that even the strongest agent achieves only 0.94$\times$ aggregate speedup over production baselines, with weaker agents at $0.78\times$ and $0.53\times$ -- confirming that benchmark-production misalignment is a critical bottleneck for the field. We release FastKernels as a stepping stone toward kernel agents whose benchmark gains translate directly into production throughput improvements. Code is available at https://github.com/Snowflake-AI-Research/fastkernels

cs.LG↗

arXiv2026

Towards a Refinement of Krylov Complexity: Scrambling, Classical Operator Growth and Replicas

We propose and test logarithmic Krylov (logK) complexity, an operator growth measure akin to Krylov complexity defined through a replica approach, as a viable probe of early-time operator scrambling without false positives. In finite-dimensional quantum systems, such as the Lipkin--Meshkov--Glick (LMG) model and the mixed-field Ising model at the chaotic point, we provide numerical evidence that logK-complexity discriminates between genuine and saddle-dominated scrambling at early times, correctly avoiding the exponential contribution coming from the unstable saddle in the former case, and closely tracking the conventional Krylov complexity in the latter. In integrable quantum systems admitting infinite-dimensional Krylov subspaces, such as the SYK$_{2}$ model and the quantum inverted harmonic oscillator, we show that by modifying the Krylov spreading operator, obtained through generalizing the analytic continuation procedure in the replica trick, the logK complexity can be refined to capture the integrable properties of the theories. We supplement these analyses by extending the Krylov formalism in classical dynamical systems and defining classical versions of these operator growth measures, showing that the false positives arising from unstable saddles in classical phase space are non-existent.

hep-th↗

arXiv2026

Internalizing Agency from Reflective Experience

Large language models are increasingly deployed as autonomous agents that must plan, act, and recover from mistakes through long-horizon interaction with environments that provide rich feedback. However, prevailing outcome-driven post-training methods (e.g., RL with verifiable rewards) primarily optimize final success signals, leaving rich environment feedback underutilized. Consequently, they often lead to distribution sharpening: the policy becomes better at reproducing a narrow set of already-successful behaviors, while failing to improve the feedback-grounded agency needed to expand problem-solving capacity (e.g., Pass@k) in long-horizon settings. To address this, we propose LEAFE (Learning Feedback-Grounded Agency from Reflective Experience), a framework that internalizes recovery agency from reflective experience. Specifically, during exploration, the agent summarizes environment feedback into actionable experience, backtracks to earlier decision points, and explores alternative branches with revised actions. We then distill these experience-guided corrections into the model through supervised fine-tuning, enabling the policy to recover more effectively in future interactions. Across a diverse set of interactive coding and agentic tasks under fixed interaction budgets, LEAFE consistently improves Pass@1 over the base model and achieves higher Pass@k than outcome-driven baselines (GRPO) and experience-based methods such as Early Experience, with gains of up to 14% on Pass@128.

cs.AI↗

arXiv2026

When Drafts Evolve: Speculative Decoding Meets Online Learning

Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model rapidly generates candidate tokens that are then verified in parallel by a larger target model. However, due to limited model capacity, drafts often struggle to approximate the target distribution, resulting in shorter acceptance lengths and diminished speedup. A key yet under-explored observation is that speculative decoding inherently provides verification feedback that quantifies the deviation between the draft and target models at no additional cost. This process naturally forms an iterative "draft commits-feedback provides-draft adapts" evolving loop, which precisely matches the online learning paradigm. Motivated by this connection, we propose OnlineSPEC, a unified framework that systematically leverages interactive feedback to continuously evolve draft models. Grounded in dynamic regret minimization, we establish a formal link between online learning performance and speculative system's acceleration rate, and develop novel algorithms via modern online learning techniques, including optimistic online learning that adaptively reuses historical gradients as predictive update hints, and online ensemble learning that dynamically maintains multiple draft models. Our algorithms are equipped with theoretical justifications and improved acceleration rates, achieving up to 24% speedup over seven benchmarks and five foundation models.

cs.LG↗

arXiv2025

Fast and Accurate Causal Parallel Decoding using Jacobi Forcing

Multi-token generation has emerged as a promising paradigm for accelerating transformer-based large model inference. Recent efforts primarily explore diffusion Large Language Models (dLLMs) for parallel decoding to reduce inference latency. To achieve AR-level generation quality, many techniques adapt AR models into dLLMs to enable parallel decoding. However, they suffer from limited speedup compared to AR models due to a pretrain-to-posttrain mismatch. Specifically, the masked data distribution in post-training deviates significantly from the real-world data distribution seen during pretraining, and dLLMs rely on bidirectional attention, which conflicts with the causal prior learned during pretraining and hinders the integration of exact KV cache reuse. To address this, we introduce Jacobi Forcing, a progressive distillation paradigm where models are trained on their own generated parallel decoding trajectories, smoothly shifting AR models into efficient parallel decoders while preserving their pretrained causal inference property. The models trained under this paradigm, Jacobi Forcing Model, achieves 3.8x wall-clock speedup on coding and math benchmarks with minimal loss in performance. Based on Jacobi Forcing Models' trajectory characteristics, we introduce multi-block decoding with rejection recycling, which enables up to 4.5x higher token acceptance count per iteration and nearly 4.0x wall-clock speedup, effectively trading additional compute for lower inference latency. Our code is available at https://github.com/hao-ai-lab/JacobiForcing.

cs.CL↗

arXiv2025

Probabilistic Modeling of Multi-rater Medical Image Segmentation for Diversity and Personalization

Lesion segmentation is inherently influenced by imaging uncertainty, arising from ill-defined lesion boundaries and inter-observer variability in diagnosis. To address this challenge, previous works formulated the multi-rater medical image segmentation task, where multiple experts provide separate annotations for each image. However, existing models are typically constrained to either generate diverse segmentation that lacks expert specificity or to produce personalized outputs that merely replicate individual annotators. We propose \textbf{Pro}babilistic modeling of multi-rater lesion \textbf{Seg}mentation (\textbf{ProSeg}) that simultaneously enables both diversification and personalization. Specifically, we introduce two latent variables to model expert annotation preferences and lesion boundary ambiguity. Their conditional probabilistic distributions are then obtained through variational inference, allowing segmentation outputs to be generated by sampling from these distributions. Extensive experiments on both the nasopharyngeal carcinoma dataset (NPC) and the lung nodule dataset (LIDC-IDRI) demonstrate that our ProSeg achieves a new state-of-the-art performance, providing segmentation results that are both diverse and expert-personalized.

cs.CV↗

arXiv2025

Toward Krylov-based holography in double-scaled SYK

Building on the duality between Krylov complexity and geodesic length in Jackiw-Teitelboim and sine-dilaton gravity, we develop a precise holographic dictionary for quantities in the Krylov subspace of the double-scaled Sachdev-Ye-Kitaev model (DSSYK). First, we demonstrate that the growth rate of Krylov state complexity corresponds to the wormhole velocity, and show that its expectation value in coherent states serves as a boundary diagnostic of firewall-like structures via bulk reconstruction. We also delineate an alternative bulk description in terms of the proper momentum of an infalling particle at early times, establishing a threefold duality between the Krylov complexity growth rate, wormhole velocity, and proper momentum, with clear regimes of validity. Beyond the first moments, we argue that higher-order Krylov complexities capture connected bulk contributions encoded by replica wormholes, while the logarithmic variant probes the replica saddle structure. Finally, within a third-quantized setting incorporating baby universes, we show that the Krylov entropy equals the von Neumann entropy of the parent-geometry density matrix obtained after tracing out baby universes, thereby quantifying information flow into the baby universe sector. Together, these results elevate Krylov-space observables to sharp probes of bulk dynamics and topology in ensemble-averaged 2D gravity.

hep-th↗

arXiv2025

Deep Think with Confidence

Large Language Models (LLMs) have shown great potential in reasoning tasks through test-time scaling methods like self-consistency with majority voting. However, this approach often leads to diminishing returns in accuracy and high computational overhead. To address these challenges, we introduce Deep Think with Confidence (DeepConf), a simple yet powerful method that enhances both reasoning efficiency and performance at test time. DeepConf leverages model-internal confidence signals to dynamically filter out low-quality reasoning traces during or after generation. It requires no additional model training or hyperparameter tuning and can be seamlessly integrated into existing serving frameworks. We evaluate DeepConf across a variety of reasoning tasks and the latest open-source models, including Qwen 3 and GPT-OSS series. Notably, on challenging benchmarks such as AIME 2025, DeepConf@512 achieves up to 99.9% accuracy and reduces generated tokens by up to 84.7% compared to full parallel thinking.

cs.LG↗

arXiv2025

Scaling Speculative Decoding with Lookahead Reasoning

Reasoning models excel by generating long chain-of-thoughts, but decoding the resulting thousands of tokens is slow. Token-level speculative decoding (SD) helps, but its benefit is capped, because the chance that an entire $\gamma$-token guess is correct falls exponentially as $\gamma$ grows. This means allocating more compute for longer token drafts faces an algorithmic ceiling -- making the speedup modest and hardware-agnostic. We raise this ceiling with Lookahead Reasoning, which exploits a second, step-level layer of parallelism. Our key insight is that reasoning models generate step-by-step, and each step needs only to be semantically correct, not exact token matching. In Lookahead Reasoning, a lightweight draft model proposes several future steps; the target model expands each proposal in one batched pass, and a verifier keeps semantically correct steps while letting the target regenerate any that fail. Token-level SD still operates within each reasoning step, so the two layers of parallelism multiply. We show Lookahead Reasoning lifts the peak speedup of SD both theoretically and empirically. Across GSM8K, AIME, and other benchmarks, Lookahead Reasoning improves the speedup of SD from 1.4x to 2.1x while preserving answer quality, and its speedup scales better with additional GPU throughput. Our code is available at https://github.com/hao-ai-lab/LookaheadReasoning

cs.LG↗

arXiv2025

Quasinormal modes and complexity in saddle-dominated SU(N) spin systems

We study SU($N$) spin systems that mimic the behavior of particles in $N$-dimensional de Sitter space for $N=2,3$. Their Hamiltonians describe a dynamical system with hyperbolic fixed points, leading to emergent quasinormal modes at the quantum level. These manifest as quasiparticle peaks in the density of states. For a particle in 2-dimensional de Sitter, we find both principal and complementary series densities of states from a PT-symmetric version of the Lipkin-Meshkov-Glick model, having two hyperbolic fixed points in the classical phase space. We then study different spectral and dynamical properties of this class of models, including level spacing statistics, two-point functions, squared commutators, spectral form factor, Krylov operator and state complexity. We find that, even though the early-time properties of these quantities are governed by the saddle points -- thereby in some cases mimicking corresponding properties of chaotic systems, a close look at the late-time behavior reveals the integrable nature of the system.

hep-th↗

arXiv2025

Matrix Factorization with Dynamic Multi-view Clustering for Recommender System

Matrix factorization (MF), a cornerstone of recommender systems, decomposes user-item interaction matrices into latent representations. Traditional MF approaches, however, employ a two-stage, non-end-to-end paradigm, sequentially performing recommendation and clustering, resulting in prohibitive computational costs for large-scale applications like e-commerce and IoT, where billions of users interact with trillions of items. To address this, we propose Matrix Factorization with Dynamic Multi-view Clustering (MFDMC), a unified framework that balances efficient end-to-end training with comprehensive utilization of web-scale data and enhances interpretability. MFDMC leverages dynamic multi-view clustering to learn user and item representations, adaptively pruning poorly formed clusters. Each entity's representation is modeled as a weighted projection of robust clusters, capturing its diverse roles across views. This design maximizes representation space utilization, improves interpretability, and ensures resilience for downstream tasks. Extensive experiments demonstrate MFDMC's superior performance in recommender systems and other representation learning domains, such as computer vision, highlighting its scalability and versatility.

cs.IR↗

arXiv2025

Quantum Signatures of Chaos from Free Probability

A classical dynamical system can be viewed as a probability space equipped with a measure-preserving time evolution map, admitting a purely algebraic formulation in terms of the algebra of bounded functions on the phase space. Similarly, a quantum dynamical system can be formulated using an algebra of bounded operators in a non-commutative probability space equipped with a time evolution map. Chaos, in either setting, can be characterized by statistical independence between observables at $ t = 0 $ and $ t \to \infty $, leading to the vanishing of cumulants involving these observables. In the quantum case, the notion of independence is replaced by free independence, which only emerges in the thermodynamic limit (asymptotic freeness). In this work, we propose a definition of quantum chaos based on asymptotic freeness and investigate its emergence in quantum many-body systems including the mixed-field Ising model with a random magnetic field, a higher spin version of the same model, and the SYK model. The hallmark of asymptotic freeness is the emergence of the free convolution prediction for the spectrum of operators of the form $ A(0) + B(t) $, implying the vanishing of all free cumulants between $A(0)$ and $B(t)$ in the thermodynamic limit for an infinite-temperature thermal state. We systematically investigate the spectral properties of $ A(0) + B(t) $ in the above-mentioned models, show that fluctuations on top of the free convolution prediction follow universal Wigner-Dyson statistics, and discuss the connection with quantum chaos. Finally, we argue that free probability theory provides a rigorous framework for understanding quantum chaos, offering a unifying perspective that connects many different manifestations of it.

hep-th↗

arXiv2024

Efficiently Scaling LLM Reasoning with Certaindex

Test-time reasoning algorithms such as chain-of-thought, self-consistency, and MCTS enhance LLM problem-solving but can wastefully generate many tokens without improving accuracy. At the same time, we observe that these algorithms exhibit answer stabilization: their intermediate solutions often cease to change after a certain point, and further investment of compute does not change their final answer. To quantify this phenomenon, we introduce Certaindex, an algorithm-agnostic metric measuring this evolving stability, signaling when further computation is unlikely to alter the final result. Certaindex is lightweight, can accelerate reasoning program inference via early exit, and further enables dynamic token allocation, gang scheduling, and many opportunities when integrated with real-world LLM serving systems. To quantify real-world benefits, we built Certaindex as a scheduler into Dynasor, our reasoning-aware LLM serving system, and demonstrate up to 50% compute savings and 3.3x higher throughput in real workloads with no accuracy drop. Our code is available at https://github.com/hao-ai-lab/Dynasor.git

cs.LG↗

arXiv2024

Higher-Order Krylov State Complexity in Random Matrix Quenches

In quantum many-body systems, time-evolved states typically remain confined to a smaller region of the Hilbert space known as the $\textit{Krylov subspace}$. The time evolution can be mapped onto a one-dimensional problem of a particle moving on a chain, where the average position $\langle n \rangle$ defines Krylov state complexity or spread complexity. Generalized spread complexities, associated with higher-order moments $\langle n^p \rangle$ for $p>1$, provide finer insights into the dynamics. We investigate the time evolution of generalized spread complexities following a quantum quench in random matrix theory. The quench is implemented by transitioning from an initial random Hamiltonian to a post-quench Hamiltonian obtained by dividing it into four blocks and flipping the sign of the off-diagonal blocks. This setup captures universal features of chaotic quantum quenches. When the initial state is the thermofield double state of the post-quench Hamiltonian, a peak in spread complexity preceding equilibration signals level repulsion, a hallmark of quantum chaos. We examine the robustness of this peak for other initial states, such as the ground state or the thermofield double state of the pre-quench Hamiltonian. To quantify this behavior, we introduce a measure based on the peak height relative to the late-time saturation value. In the continuous limit, higher-order complexities show increased sensitivity to the peak, supported by numerical simulations for finite-size random matrices.

hep-th↗

arXiv2024

Wedge Holographic Complexity in Karch-Randall Braneworld

We investigate holographic complexities in the context of wedge holography, focusing specifically on black string geometry in AdS$_3$. The wedge spacetime is bounded by two end-of-the-world (EOW) branes with intrinsic Dvali-Gabadadze-Porrati (DGP) gravity. In line with this codimension-two holography, there are three equivalent perspectives: bulk perspective, brane perspective, and boundary perspective. Using both the ''Complexity=Volume'' (CV) and ''Complexity=Action'' (CA) proposals, we analyze the complexity in wedge black string geometry in the tensionless limit. By treating the branes as rigid, we find the late-time growth rates of CV and CA match exactly across bulk and brane perspectives. These results are consistent with those from JT gravity, with additional contributions from the intrinsic gravity of the branes. For fluctuating branes, we find that the late-time growth rates of CV and CA match between bulk and brane perspectives at the linear order of fluctuation. The CV results exhibit $\frac{\phi_h^2}{\phi_0}$ corrections from fluctuations, consistent with the findings in previous work. Moreover, the CA results reveal an additional constant term in the fluctuating branes case. We provide an interpretation of this in terms of gravitational edge mode effects. The distinct corrections arising from fluctuations in the CA and CV proposals suggest that the CV proposal is more sensitive to geometric details. Furthermore, we discuss these results in relation to Lloyd's bound on complexity, their general time dependence, and the effects of fluctuations.

hep-th↗

arXiv2024

Statistics and Complexity of Wavefunction Spreading in Quantum Dynamical Systems

We consider the statistics of the results of a measurement of the spreading operator in the Krylov basis generated by the Hamiltonian of a quantum system starting from a specified initial pure state. We first obtain the probability distribution of the results of measurements of this spreading operator at a certain instant of time, and compute the characteristic function of this distribution. We show that the moments of this characteristic function are related to the so-called generalised spread complexities, and obtain expressions for them in several cases when the Hamiltonian is an element of a Lie algebra. Furthermore, by considering a continuum limit of the Krylov basis, we show that the generalised spread complexities of higher orders have a peak in the time evolution for a random matrix Hamiltonian belonging to the Gaussian unitary ensemble. We also obtain an upper bound on the change in generalised spread complexity at an arbitrary time in terms of the operator norm of the Hamiltonian and discuss the significance of these results.

quant-ph↗

arXiv2024

KA$^2$ER: Knowledge Adaptive Amalgamation of ExpeRts for Medical Images Segmentation

Recently, many foundation models for medical image analysis such as MedSAM, SwinUNETR have been released and proven to be useful in multiple tasks. However, considering the inherent heterogeneity and inhomogeneity of real-world medical data, directly applying these models to specific medical image segmentation tasks often leads to negative domain shift effects, which can severely weaken the model's segmentation capabilities. To this end, we propose an adaptive amalgamation knowledge framework that aims to train a versatile foundation model to handle the joint goals of multiple expert models, each specialized for a distinct task. Specifically, we first train an nnUNet-based expert model for each task, and reuse the pre-trained SwinUNTER as the target foundation model. Then, the input data for all challenging tasks are encoded in the foundation model and the expert models, respectively, and their backbone features are jointly projected into the adaptive amalgamation layer. Within the hidden layer, the hierarchical attention mechanisms are designed to achieve adaptive merging of the target model to the hidden layer feature knowledge of all experts, which significantly reduces the domain shift arising from the inter-task differences. Finally, the gold amalgamated features and the prompt features are fed into the mask decoder to obtain the segmentation results. Extensive experiments conducted in these challenging tasks demonstrate the effectiveness and adaptability of our foundation model for real-world medical image segmentation.

cs.CV↗

arXiv2024

Efficient LLM Scheduling by Learning to Rank

In Large Language Model (LLM) inference, the output length of an LLM request is typically regarded as not known a priori. Consequently, most LLM serving systems employ a simple First-come-first-serve (FCFS) scheduling strategy, leading to Head-Of-Line (HOL) blocking and reduced throughput and service quality. In this paper, we reexamine this assumption -- we show that, although predicting the exact generation length of each request is infeasible, it is possible to predict the relative ranks of output lengths in a batch of requests, using learning to rank. The ranking information offers valuable guidance for scheduling requests. Building on this insight, we develop a novel scheduler for LLM inference and serving that can approximate the shortest-job-first (SJF) schedule better than existing approaches. We integrate this scheduler with the state-of-the-art LLM serving system and show significant performance improvement in several important applications: 2.8x lower latency in chatbot serving and 6.5x higher throughput in synthetic data generation. Our code is available at https://github.com/hao-ai-lab/vllm-ltr.git

cs.LG↗