arXiv subjects

Yueqi Wang

Publications and source records attributed to Yueqi Wang.

At least 19 recordsLinked to original sources

arXiv2026

Geometry-aware LegONet for PDE Learning on Arbitrary Domains

Learned PDE solvers often entangle governing operators with the geometry, boundary conditions, and discretization used for training. This limits reuse when the same physics is posed on new domains, and it also makes physical-law discovery geometry-dependent. We introduce Geometry-aware LegONet (gLegONet), a boundary-manifold extension of Lego-like operator learning. Physical mechanisms are pretrained once as modular variational blocks on an ambient spectral domain. For a target geometry, sampled boundary constraints define an affine admissible manifold. Its mass-orthonormal tangent coordinates are used to evolve the dynamics and evaluate candidate law-discovery features directly. Changing the domain therefore changes only an algebraic coordinate interface, not the learned operator blocks. This converts arbitrary-domain PDE learning from geometry-specific retraining or soft penalty enforcement into boundary-guaranteed assembly of reusable mechanisms. In forward simulations and sparse identification tests on unseen domains, the method maintains boundary residuals near the algebraic tolerance and yields predictive governing laws from short-time observations.

math.NA↗

arXiv2026

Vectorizing the Trie: Efficient Constrained Decoding for LLM-based Generative Retrieval on Accelerators

Generative retrieval has emerged as a powerful paradigm for LLM-based recommendation. However, industrial recommender systems often benefit from restricting the output space to a constrained subset of items based on business logic (e.g. enforcing content freshness or product category), which standard autoregressive decoding cannot natively support. Moreover, existing constrained decoding methods that make use of prefix trees (Tries) incur severe latency penalties on hardware accelerators (TPUs/GPUs). In this work, we introduce STATIC (Sparse Transition Matrix-Accelerated Trie Index for Constrained Decoding), an efficient and scalable constrained decoding technique designed specifically for high-throughput LLM-based generative retrieval on TPUs/GPUs. By flattening the prefix tree into a static Compressed Sparse Row (CSR) matrix, we transform irregular tree traversals into fully vectorized sparse matrix operations, unlocking massive efficiency gains on hardware accelerators. We deploy STATIC on a large-scale industrial video recommendation platform serving billions of users. STATIC produces significant product metric impact with minimal latency overhead (0.033 ms per step and 0.25% of inference time), achieving a 948x speedup over a CPU trie implementation and a 47-1033x speedup over a hardware-accelerated binary-search baseline. Furthermore, the runtime overhead of STATIC remains extremely low across a wide range of practical configurations. To the best of our knowledge, STATIC enables the first production-scale deployment of strictly constrained generative retrieval. In addition, evaluation on academic benchmarks demonstrates that STATIC can considerably improve cold-start performance for generative retrieval. Our code is available at https://github.com/youtube/static-constraint-decoding.

cs.IR↗

arXiv2026

Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMs

Video Large Language Models (Video LLMs) incur high inference latency due to a large number of visual tokens provided to LLMs. To address this, training-free visual token pruning has emerged as a solution to reduce computational costs; however, existing methods are primarily validated on Multiple-Choice Question Answering (MCQA) benchmarks, where coarse-grained cues often suffice. In this work, we reveal that these methods suffer a sharp performance collapse on fine-grained understanding tasks requiring precise visual grounding, such as hallucination evaluation. To explore this gap, we conduct a systematic analysis and identify sink tokens--semantically uninformative tokens that attract excessive attention--as a key obstacle to fine-grained video understanding. When these sink tokens survive pruning, they distort the model's visual evidence and hinder fine-grained understanding. Motivated by these insights, we propose Sink-Token-aware Pruning (SToP), a simple yet effective plug-and-play method that introduces a sink score to quantify each token's tendency to behave as a sink and applies this score to existing spatial and temporal pruning methods to suppress them, thereby enhancing video understanding. To validate the effectiveness of SToP, we apply it to state-of-the-art pruning methods (VisionZip, FastVid, and Holitom) and evaluate it across diverse benchmarks covering hallucination, open-ended generation, compositional reasoning, and MCQA. Our results demonstrate that SToP significantly boosts performance, even when pruning up to 90% of visual tokens.

cs.LG↗

arXiv2026

Numerical homogenization for indefinite time-harmonic Maxwell equations

We propose a novel numerical homogenization method based on the edge multiscale approach for solving indefinite time-harmonic Maxwell equations in heterogeneous media with large wavenumber. Numerical methods for these equations in homogeneous media with high wavenumber are particularly challenging due to the so-called pollution effect: the mesh size must be significantly smaller than the reciprocal of the wavenumber to achieve a desired accuracy. This challenge is amplified in heterogeneous media, which frequently occur in practical applications such as metamaterial simulations, since resolving the heterogeneity is necessary for obtaining reliable solutions. Our approach overcomes this difficulty by avoiding explicit resolution of the heterogeneity, while employing a mesh size that depends almost linearly on the reciprocal of the wavenumber. The approximation properties and stability of the method rely critically on the development and rigorous analysis of a novel, nonstandard variational formulation, which constitutes the main innovation of this work. Extensive numerical experiments are provided to validate our theoretical findings.

math.NA↗

arXiv2026

True random number generation through stochastic magnonic bistability

True random number generators (TRNGs) underpin modern cryptography, yet existing implementations face fundamental trade-offs between speed, scalability, and entropy quality. Here, we demonstrate that stochastic switching in the bistable regime of spin-wave dynamics provides a physical entropy source for high-quality random number generation. Our magnonic random number generator (mRNG), based on a lithography-patterned microstrip on yttrium iron garnet (YIG), exploits thermal fluctuations near the nonlinear bistable regime to generate random bitstreams that pass all 15 NIST SP 800-22 statistical tests at rates with 20 Mb/s. We implement a random-bit multiplier using synchronized mRNG units and demonstrate scalability to 200-nm-wide nanoscale waveguides, establishing spin-wave bistability as a viable physical entropy source for integrated random number generation.

cond-mat.mtrl-sci↗

arXiv2026

LegONet: Plug-and-Play Structure-Preserving Neural Operator Blocks for Compositional PDE Learning

Learned PDE solvers are often trained as monolithic surrogates for a specific equation, boundary condition and discretization. This makes them difficult to reuse when mechanisms change and it can limit stability under long-horizon rollout. We introduce Lego-like Operator Network (LegONet), a compositional framework that builds PDE solvers from plug-and-play, structure-preserving operator blocks defined on shared boundary-adapted spectral representations. LegONet separates boundary handling from mechanism learning, satisfying boundary conditions by construction. It also separates mechanism learning from time integration, enabling pretrained blocks to be assembled into new solvers without retraining. We also derive a finite-horizon error decomposition that separates block mismatch from splitting error and provides mechanism-level diagnostics for long-horizon predictions. Across ten time-dependent PDEs, LegONet delivers accurate closed-loop rollouts with improved stability under cross-PDE recombination and boundary reconfiguration. More broadly, this modular formulation suggests a path from task-specific neural solvers towards plug-and-play operator libraries for scientific computing.

math.NA↗

arXiv2026

FASA: Frequency-aware Sparse Attention

The deployment of Large Language Models (LLMs) faces a critical bottleneck when handling lengthy inputs: the prohibitive memory footprint of the Key Value (KV) cache. To address this bottleneck, the token pruning paradigm leverages attention sparsity to selectively retain a small, critical subset of tokens. However, existing approaches fall short, with static methods risking irreversible information loss and dynamic strategies employing heuristics that insufficiently capture the query-dependent nature of token importance. We propose FASA, a novel framework that achieves query-aware token eviction by dynamically predicting token importance. FASA stems from a novel insight into RoPE: the discovery of functional sparsity at the frequency-chunk (FC) level. Our key finding is that a small, identifiable subset of "dominant" FCs consistently exhibits high contextual agreement with the full attention head. This provides a robust and computationally free proxy for identifying salient tokens. Building on this insight, FASA first identifies a critical set of tokens using dominant FCs, and then performs focused attention computation solely on this pruned subset. Across a spectrum of long-context tasks, from sequence modeling to complex CoT reasoning, FASA consistently outperforms all token-eviction baselines and achieves near-oracle accuracy, demonstrating remarkable robustness even under constraint budgets. Notably, on LongBench-V1, FASA reaches nearly 100\% of full-KV performance when only keeping 256 tokens, and achieves 2.56$\times$ speedup using just 18.9\% of the cache on AIME24.

cs.CL↗

arXiv2026

Ultra Fast PDE Solving via Physics Guided Few-step Diffusion

Diffusion-based models have demonstrated impressive accuracy and generalization in solving partial differential equations (PDEs). However, they still face significant limitations, such as high sampling costs and insufficient physical consistency, stemming from their many-step iterative sampling mechanism and lack of explicit physics constraints. To address these issues, we propose Phys-Instruct, a novel physics-guided distillation framework which not only (1) compresses a pre-trained diffusion PDE solver into a few-step generator via matching generator and prior diffusion distributions to enable rapid sampling, but also (2) enhances the physics consistency by explicitly injecting PDE knowledge through a PDE distillation guidance. Physic-Instruct is built upon a solid theoretical foundation, leading to a practical physics-constrained training objective that admits tractable gradients. Across five PDE benchmarks, Phys-Instruct achieves orders-of-magnitude faster inference while reducing PDE error by more than 8 times compared to state-of-the-art diffusion baselines. Moreover, the resulting unconditional student model functions as a compact prior, enabling efficient and physically consistent inference for various downstream conditional tasks. Our results indicate that Phys-Instruct is a novel, effective, and efficient framework for ultra-fast PDE solving powered by deep generative models.

cs.LG↗

arXiv2026

A POD-DeepONet Framework for Forward and Inverse Design of 2D Photonic Crystals

We develop a reduced-order operator-learning framework for forward and inverse band-structure design of two-dimensional photonic crystals with binary, pixel-based $p4m$-symmetric unit cells. We construct a POD--DeepONet surrogate for the discrete band map along the standard high-symmetry path by coupling a POD trunk extracted from high-fidelity finite-element band snapshots with a neural branch network that predicts reduced coefficients. This architecture yields a compact and differentiable forward model that is tailored to the underlying Bloch eigenvalue discretization. We establish continuity of the discrete band map on the relaxed design space and prove a uniform approximation property of the POD--DeepONet surrogate, leading to a natural decomposition of the total surrogate error into POD truncation and network approximation contributions. Building on this forward surrogate, we formulate two end-to-end neural inverse design procedures, namely dispersion-to-structure and band-gap inverse design, with training objectives that combine data misfit, binarity promotion, and supervised regularization to address the intrinsic non-uniqueness of the inverse mapping and to enable stable gradient-based optimization in the relaxed space. Our numerical results show that the proposed framework achieves accurate forward predictions and produces effective inverse designs on practical high-contrast, pixel-based photonic layouts.

physics.optics↗

arXiv2025

Reduced-Basis Deep Operator Learning for Parametric PDEs with Independently Varying Boundary and Source Data

Parametric PDEs power modern simulation, design, and digital-twin systems, yet their many-query workloads still hinge on repeatedly solving large finite-element systems. Existing operator-learning approaches accelerate this process but often rely on opaque learned trunks, require extensive labeled data, or break down when boundary and source data vary independently from physical parameters. We introduce RB-DeepONet, a hybrid operator-learning framework that fuses reduced-basis (RB) numerical structure with the branch-trunk architecture of DeepONet. The trunk is fixed to a rigorously constructed RB space generated offline via Greedy selection, granting physical interpretability, stability, and certified error control. The branch network predicts only RB coefficients and is trained label-free using a projected variational residual that targets the RB-Galerkin solution. For problems with independently varying loads or boundary conditions, we develop boundary and source modal encodings that compress exogenous data into low-dimensional coordinates while preserving accuracy. Combined with affine or empirical interpolation decompositions, RB-DeepONet achieves a strict offline-online split: all heavy lifting occurs offline, and online evaluation scales only with the RB dimension rather than the full mesh. We provide convergence guarantees separating RB approximation error from statistical learning error, and numerical experiments show that RB-DeepONet attains accuracy competitive with intrusive RB-Galerkin, POD-DeepONet, and FEONet while using dramatically fewer trainable parameters and achieving significant speedups. This establishes RB-DeepONet as an efficient, stable, and interpretable operator learner for large-scale parametric PDEs.

cs.LG↗

arXiv2025

PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations

Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world knowledge inherent in these large models. In this paper, we introduce PLUM, a framework designed to adapt pre-trained LLMs for industry-scale recommendation tasks. PLUM consists of item tokenization using Semantic IDs, continued pre-training (CPT) on domain-specific data, and task-specific fine-tuning for recommendation objectives. For fine-tuning, we focus particularly on generative retrieval, where the model is directly trained to generate Semantic IDs of recommended items based on user context. We conduct comprehensive experiments on large-scale internal video recommendation datasets. Our results demonstrate that PLUM achieves substantial improvements for retrieval compared to a heavily-optimized production model built with large embedding tables. We also present a scaling study for the model's retrieval performance, our learnings about CPT, a few enhancements to Semantic IDs, along with an overview of the training and inference methods that enable launching this framework to billions of users in YouTube.

cs.IR↗

arXiv2025

LLM-Powered Nuanced Video Attribute Annotation for Enhanced Recommendations

This paper presents a case study on deploying Large Language Models (LLMs) as an advanced "annotation" mechanism to achieve nuanced content understanding (e.g., discerning content "vibe") at scale within a large-scale industrial short-form video recommendation system. Traditional machine learning classifiers for content understanding face protracted development cycles and a lack of deep, nuanced comprehension. The "LLM-as-annotators" approach addresses these by significantly shortening development times and enabling the annotation of subtle attributes. This work details an end-to-end workflow encompassing: (1) iterative definition and robust evaluation of target attributes, refined by offline metrics and online A/B testing; (2) scalable offline bulk annotation of video corpora using LLMs with multimodal features, optimized inference, and knowledge distillation for broad application; and (3) integration of these rich annotations into the online recommendation serving system, for example, through personalized restrict retrieval. Experimental results demonstrate the efficacy of this approach, with LLMs outperforming human raters in offline annotation quality for nuanced attributes and yielding significant improvements of user participation and satisfied consumption in online A/B tests. The study provides insights into designing and scaling production-level LLM pipelines for rich content evaluation, highlighting the adaptability and benefits of LLM-generated nuanced understanding for enhancing content discovery, user satisfaction, and the overall effectiveness of modern recommendation systems.

cs.IR↗

arXiv2025

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization

The emergence of large Vision Language Models (VLMs) has broadened the scope and capabilities of single-modal Large Language Models (LLMs) by integrating visual modalities, thereby unlocking transformative cross-modal applications in a variety of real-world scenarios. Despite their impressive performance, VLMs are prone to significant hallucinations, particularly in the form of cross-modal inconsistencies. Building on the success of Reinforcement Learning from Human Feedback (RLHF) in aligning LLMs, recent advancements have focused on applying direct preference optimization (DPO) on carefully curated datasets to mitigate these issues. Yet, such approaches typically introduce preference signals in a brute-force manner, neglecting the crucial role of visual information in the alignment process. In this paper, we introduce Re-Align, a novel alignment framework that leverages image retrieval to construct a dual-preference dataset, effectively incorporating both textual and visual preference signals. We further introduce rDPO, an extension of the standard direct preference optimization that incorporates an additional visual preference objective during fine-tuning. Our experimental results demonstrate that Re-Align not only mitigates hallucinations more effectively than previous methods but also yields significant performance gains in general visual question-answering (VQA) tasks. Moreover, we show that Re-Align maintains robustness and scalability across a wide range of VLM sizes and architectures. This work represents a significant step forward in aligning multimodal LLMs, paving the way for more reliable and effective cross-modal applications. We release all the code in https://github.com/taco-group/Re-Align.

cs.CV↗

arXiv2025

mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generation

Large Vision-Language Models (LVLMs) have made remarkable strides in multimodal tasks such as visual question answering, visual grounding, and complex reasoning. However, they remain limited by static training data, susceptibility to hallucinations, and inability to verify claims against up-to-date, external evidence, compromising their performance in dynamic real-world applications. Retrieval-Augmented Generation (RAG) offers a practical solution to mitigate these challenges by allowing the LVLMs to access large-scale knowledge databases via retrieval mechanisms, thereby grounding model outputs in factual, contextually relevant information. Here in this paper, we conduct the first systematic dissection of the multimodal RAG pipeline for LVLMs, explicitly investigating (1) the retrieval phase: on the modality configurations and retrieval strategies, (2) the re-ranking stage: on strategies to mitigate positional biases and improve the relevance of retrieved evidence, and (3) the generation phase: we further investigate how to best integrate retrieved candidates into the final generation process. Finally, we extend to explore a unified agentic framework that integrates re-ranking and generation through self-reflection, enabling LVLMs to select relevant evidence and suppress irrelevant context dynamically. Our full-stack exploration of RAG for LVLMs yields substantial insights, resulting in an average performance boost of 5% without any fine-tuning.

cs.AI↗

arXiv2025

Adaptive Crouzeix-Raviart finite elements for the first eigenpair of $p$-Laplacian

In this paper, we propose and analyze an adaptive Crouzeix-Raviart finite element method for computing the first Dirichlet eigenpair of the $p$-Laplacian problem. We prove that the sequence of error estimators produced by the adaptive algorithm has a vanishing limit and that, starting from a fine initial mesh, the relevant sequence of approximate eigenvalues converges to the first eigenvalue and the distance in a mesh-dependent broken norm between discrete eigenfunctions and the set composed of relevant continuous eigenfunctions also tends to zero. The analysis hinges on establishing a compactness property for Crouzeix-Raviart finite elements over a sequence of adaptively generated meshes, which represents key theoretical challenges and novelties. We present numerical results to illustrate the advantage of the proposed algorithm.

math.NA↗

arXiv2025

User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems

Exploration, the act of broadening user experiences beyond their established preferences, is challenging in large-scale recommendation systems due to feedback loops and limited signals on user exploration patterns. Large Language Models (LLMs) offer potential solutions by leveraging their world knowledge to recommend novel content outside these loops. A key challenge is aligning LLMs with user preferences while preserving their knowledge and reasoning. To enhance planning for new user interests using LLMs, this paper introduces a novel approach that combines hierarchical planning with LLM inference-time scaling. This method aims to improve recommendation relevancy without compromising novelty. We decouple novelty and user-alignment, training separate LLMs for each objective. We then scale up the novelty-focused LLM's inference and select the best-of-n predictions using the user-aligned LLM. Live experiments demonstrate efficacy, showing significant gains in both user satisfaction (measured by watch activity and active user counts) and exploration diversity.

cs.IR↗

arXiv2025

Your Causal Self-Attentive Recommender Hosts a Lonely Neighborhood

In the context of sequential recommendation, a pivotal issue pertains to the comparative analysis between bi-directional/auto-encoding (AE) and uni-directional/auto-regressive (AR) attention mechanisms, where the conclusions regarding architectural and performance superiority remain inconclusive. Previous efforts in such comparisons primarily involve summarizing existing works to identify a consensus or conducting ablation studies on peripheral modeling techniques, such as choices of loss functions. However, far fewer efforts have been made in (1) theoretical and (2) extensive empirical analysis of the self-attention module, the very pivotal structure on which performance and designing insights should be anchored. In this work, we first provide a comprehensive theoretical analysis of AE/AR attention matrix in the aspect of (1) sparse local inductive bias, a.k.a neighborhood effects, and (2) low rank approximation. Analytical metrics reveal that the AR attention exhibits sparse neighborhood effects suitable for generally sparse recommendation scenarios. Secondly, to support our theoretical analysis, we conduct extensive empirical experiments on comparing AE/AR attention on five popular benchmarks with AR performing better overall. Empirical results reported are based on our experimental pipeline named Modularized Design Space for Self-Attentive Recommender (ModSAR), supporting adaptive hyperparameter tuning, modularized design space and HuggingFace plug-ins. We invite the recommendation community to utilize/contribute to ModSAR to (1) conduct more module/model-level examining beyond AE/AR comparison and (2) accelerate state-of-the-art model design. Lastly, we shed light on future design choices for performant self-attentive recommenders. We make our pipeline implementation and data available at https://github.com/yueqirex/SAR-Check.

cs.IR↗

arXiv2024

Predicting band structures for 2D Photonic Crystals via Deep Learning

Photonic crystals (PhCs) are periodic dielectric structures that exhibit unique electromagnetic properties, such as the creation of band gaps where electromagnetic wave propagation is inhibited. Accurately predicting dispersion relations, which describe the frequency and direction of wave propagation, is vital for designing innovative photonic devices. However, traditional numerical methods, like the Finite Element Method (FEM), can encounter significant computational challenges due to the multiple scales present in photonic crystals, especially when calculating band structures across the entire Brillouin zone. To address this, we propose a supervised learning approach utilizing U-Net, along with transfer learning and Super-Resolution techniques, to forecast dispersion relations for 2D PhCs. Our model reduces computational expenses by producing high-resolution band structures from low-resolution data, eliminating the necessity for fine meshes throughout the Brillouin zone. The U-Net architecture enables the simultaneous prediction of multiple band functions, enhancing efficiency and accuracy compared to existing methods that handle each band function independently. Our findings demonstrate that the proposed model achieves high accuracy in predicting the initial band functions of 2D PhCs, while also significantly enhancing computational efficiency. This amalgamation of data-driven and traditional numerical techniques provides a robust framework for expediting the design and optimization of photonic crystals. The approach underscores the potential of integrating deep learning with established computational physics methods to tackle intricate multiscale problems, establishing a new benchmark for future PhC research and applications.

math.NA↗