arXiv subjects

Weiguo Gao

Publications and source records attributed to Weiguo Gao.

At least 19 recordsLinked to original sources

arXiv2026

PACO: A Fully Cache-Oblivious Parallel FFT with One Global Redistribution

Fast Fourier transforms on parallel machines incur two forms of data movement: processor-local transfers through the memory hierarchy and global redistribution between processors. Four-step FFT organizations switch the active transform dimension with one global transpose-like exchange, but this alone does not yield cache-efficient local computation. Cache-oblivious FFTs achieve asymptotically optimal local memory traffic through recursive layout transformations, whose direct materialization can require extra data-rearrangement passes and global exchanges. We present PACO, a fully cache-oblivious parallel FFT framework that reconciles these objectives. PACO executes LocalFFT -> OneGlobalPermutation -> LocalFFT. Its local stages recursively partition both transform and batch dimensions without knowledge of the cache parameters. Rather than materializing the transpose-like layouts induced by this recursion, PACO defers them, showing that they compose into a base-b digit-reversal permutation that is fused with the redistribution already required to change the local transform dimension. The resulting middle stage is a perfectly balanced parallel cache-oblivious digit-reversal permutation in which each source-destination processor pair exchanges exactly $N/p^2$ elements. Under an exact base-b slab decomposition in a hybrid ideal-cache/BSP model, PACO computes an N-point DFT exactly with maximum per-processor work $\Theta((N/p) \log N)$ and maximum per-processor cache complexity $\Theta((N/(pB))(1 + \log_M N))$, using exactly one global redistribution round. PACO returns canonical logical DFT coefficients under a factor-swapped target slab ownership. The single redistribution is necessary under the stated no-replication ownership model, while its communication volume is optimal for the prescribed fused permutation and source-target slab distributions.

cs.DS↗

arXiv2026

On the representation for stochastic graph delay propagation

In this work, we utilize the Hermite expansion to approximate the distributions of the sum and maximum of independent random variables. We model distributions with a three-segment representation, where the left and right tails are respectively modeled as combinations of Hermite functions, and the intermediate segment is approximated by piecewise polynomials. This approximation admits rigorous $L^2$- and pointwise convergence properties supported by classical results. We develop an algorithmic framework for applying our model to the graph delay propagation problem, where sum and max operations are performed on the proposed model structure. Numerical experiments demonstrate that our model can capture the quantile values with high accuracy compared to Monte Carlo simulation results, significantly outperforming classical Gaussian-based models.

math.NA↗

arXiv2026

A Quantitative Approximation Framework for Flow Distillation in Diffusion Models

We develop a quantitative framework for diffusion distillation by viewing few step sampling as approximation through compositions of learned flow maps. For trajectory distillation of the probability flow ODE, we show that low noise multimodal regimes separate score approximability from dynamical stability: the score remains efficiently approximable, while small local errors may be strongly amplified by stiff flow dynamics. In a Gaussian mixture Ornstein--Uhlenbeck model, we prove time uniform $L^p(p_t)$ score approximation by ReLU and ReQU networks with explicit polylogarithmic complexity, and derive a computable Lipschitz bound $L(t)$ for the flow velocity. The stability factor $\exp\bigl(\int_s^t L(u)\mathrm du\bigr)$ can grow exponentially as noise decreases and mixture separation increases. Comparing this certificate with a certified local Lipschitz budget for one step students identifies regimes of direct distillation difficulty, without implying an approximation lower bound. We also show that deep residual compositions control global transport error through propagated local errors, and that equalizing cumulative stability yields an optimal nonuniform segmentation. With eight segments, this grid reduces final mean relative MSE by up to $51.9\%$ versus uniform grids.

stat.ML↗

arXiv2026

Toward Theoretical Insights into Diffusion Trajectory Distillation via Operator Merging

Diffusion trajectory distillation accelerates sampling by training a student model to approximate the multi-step denoising trajectories of a pretrained teacher model using far fewer steps. Despite strong empirical results, the trade-off between distillation strategy and generative quality remains poorly understood. We provide a theoretical characterization by reinterpreting trajectory distillation as an operator merging problem, differentiating our analysis between two distinct regimes. In the linear Gaussian regime, where approximation error is zero, we isolate optimization error, specifically signal shrinkage driven by finite training time, as the primary bottleneck. This characterization allows us to derive the theoretically optimal merging strategy, which exhibits a variance-driven phase transition and is computable via a Pareto dynamic programming algorithm. In the nonlinear Gaussian mixture regime, we prove that distilling composite steps incurs unavoidable approximation error due to the exponential growth of mixture components, and we quantify how these errors amplify across merges. Together, these results clarify the distinct theoretical mechanisms governing each regime and provide principled guidance for method selection.

cs.LG↗

arXiv2026

Structure-Preserving Dynamic Mode Decomposition for Highly Oscillatory Dynamics of Semiclassical Schrödinger Equations

We propose two novel data-driven dynamic mode decomposition (DMD)-type methods, the Crank--Nicolson DMD and the semi-implicit DMD, to predict the highly oscillatory dynamics of the semiclassical Schrödinger equations efficiently and accurately. Unlike many existing DMD-type methods which directly models the dynamics of the wave function, our approach is based on learning the Schrödinger operator while explicitly incorporating mass and energy conservation laws. This approach ensures physical fidelity and endows the resulting methods with built-in model order reduction capabilities, without the necessity for additional dimensionality-reduction preprocessing. An analysis of training and prediction errors are given for theoretical guarantees. Extensive numerical experiments demonstrate the noise robustness, computational efficiency, and transferability to other equations of the proposed methods.

math.NA↗

arXiv2026

ProFlow: Zero-Shot Physics-Consistent Sampling via Proximal Flow Guidance

Inferring physical fields from sparse observations while strictly satisfying partial differential equations (PDEs) is a fundamental challenge in computational physics. Recently, deep generative models offer powerful data-driven priors for such inverse problems, yet existing methods struggle to enforce hard physical constraints without costly retraining or disrupting the learned generative prior. Consequently, there is a critical need for a sampling mechanism that can reconcile strict physical consistency and observational fidelity with the statistical structure of the pre-trained prior. To this end, we present ProFlow, a proximal guidance framework for zero-shot physics-consistent sampling, defined as inferring solutions from sparse observations using a fixed generative prior without task-specific retraining. The algorithm employs a rigorous two-step scheme that alternates between: (\romannumeral1) a terminal optimization step, which projects the flow prediction onto the intersection of the physically and observationally consistent sets via proximal minimization; and (\romannumeral2) an interpolation step, which maps the refined state back to the generative trajectory to maintain consistency with the learned flow probability path. This procedure admits a Bayesian interpretation as a sequence of local maximum a posteriori (MAP) updates. Comprehensive benchmarks on Poisson, Helmholtz, Darcy, and viscous Burgers' equations demonstrate that ProFlow achieves superior physical and observational consistency, as well as more accurate distributional statistics, compared to state-of-the-art diffusion- and flow-based baselines.

cs.LG↗

arXiv2026

A sixth-order compact time-splitting Fourier pseudospectral method

In this paper, we propose a novel sixth-order compact time-splitting scheme, denoted as $ S_{6\text{c}}$, for solving the Dirac equation in the absence of external magnetic potentials. This method is easy to implement, and it provides a substantial reduction in computational complexity compared to the existing sixth-order splitting schemes. By incorporating a time-ordering technique, we also extend $S_{6\text{c}}$ to address problems with time-dependent potentials. Comprehensive comparisons with various time-splitting methods show that $S_{6\text{c}}$ exhibits significant advantages in terms of both precision and efficiency. Moreover, numerical results indicate that $S_{6\text{c}}$ maintains the super-resolution property for the Dirac equation in the nonrelativistic regime in the absence of external magnetic potentials.

math.NA↗

arXiv2026

Decentralized Nonsmooth Nonconvex Optimization with Client Sampling

This paper considers decentralized nonsmooth nonconvex optimization problem with Lipschitz continuous local functions. We propose an efficient stochastic first-order method with client sampling, achieving the $(δ,ε)$-Goldstein stationary point with the overall sample complexity of ${\mathcal O}(δ^{-1}ε^{-3})$, the computation rounds of ${\mathcal O}(δ^{-1}ε^{-3})$, and the communication rounds of ${\tilde{\mathcal O}}(γ^{-1/2}δ^{-1}ε^{-3})$, where $γ$ is the spectral gap of the mixing matrix for the network. Our results achieve the optimal sample complexity and the sharper communication complexity than existing methods. We also extend our ideas to zeroth-order optimization. Moreover, the numerical experiments show the empirical advantage of our methods.

math.OC↗

arXiv2026

Terminally constrained flow-based generative models from an optimal control perspective

We address the problem of sampling from terminally constrained distributions with pre-trained flow-based generative models through an optimal control formulation. Theoretically, we characterize the value function by a Hamilton-Jacobi-Bellman equation and derive the optimal feedback control as the minimizer of the associated Hamiltonian. We show that as the control penalty increases, the controlled process recovers the reference distribution, while as the penalty vanishes, the terminal law converges to a generalized Wasserstein projection onto the constraint manifold. Algorithmically, we introduce Terminal Optimal Control with Flow-based models (TOCFlow), a geometry-aware sampling-time guidance method for pre-trained flows. Solving the control problem in a terminal co-moving frame that tracks reference trajectories yields a closed-form scalar damping factor along the Riemannian gradient, capturing second-order curvature effects without matrix inversions. TOCFlow therefore matches the geometric consistency of Gauss-Newton updates at the computational cost of standard gradient guidance. We evaluate TOCFlow on three high-dimensional scientific tasks spanning equality, inequality, and global statistical constraints, namely Darcy flow, constrained trajectory planning, and turbulence snapshot generation with Kolmogorov spectral scaling. Across all settings, TOCFlow improves constraint satisfaction over Euclidean guidance and projection baselines while preserving the reference model's generative quality.

cs.LG↗

arXiv2025

Tree Reward-Aligned Search for TReASURe in Masked Diffusion Language Models

Tree search has recently emerged as a powerful framework for aligning generative models with task-specific rewards at test time. Applying tree search to Masked Diffusion Language Models, however, introduces two key challenges: (i) parallel unmasking yields highly correlated branches, limiting exploration, and (ii) reward evaluation via sampled completions produces high-variance estimates, making pruning unstable. We propose TReASURe, a tree-search test-time alignment method that addresses these issues. It introduces (i) UnmaskBranch, a branching strategy based on first-hitting unmasking that diversifies both token content and reveal order with a single model call per parent node, and (ii) ResubstituteScore, a pruning rule that uses deterministic resubstitution to score partially masked sequences with low-variance proxy completions. Theoretically, we quantify branching efficiency gains in NFEs (number of function evaluations), show that the scoring rule approximates the true reward with error bounded by predictive uncertainty, and prove improvements with larger tree widths. Empirically, TReASURe achieves state-of-the-art results on perplexity, linguistic acceptability, and control of sentiment and toxicity, outperforming prior methods under matched compute budgets, with especially strong gains in low-NFE regimes.

cs.CL↗

arXiv2025

Explicit Global Convergence Rates of BFGS without Line Search

This paper studies the convergence rates of the Broyden--Fletcher--Goldfarb--Shanno~(BFGS) method without line search. We show that the BFGS method with an adaptive step size [Gao and Goldfarb, Optimization Methods and Software, 34(1):194-217, 2019] exhibits a two-phase non-asymptotic global convergence behavior when minimizing a strongly convex function, i.e., a linear convergence rate of $\mathcal{O}((1 - 1 / \varkappa)^{k})$ in the first phase and a superlinear convergence rate of $\mathcal{O}((\varkappa / k)^{k})$ in the second phase, where $k$ is the iteration counter and $\varkappa$ is the condition number. In contrast, the existing analysis only establishes asymptotic results. Furthermore, we propose a novel adaptive BFGS method without line search, which allows a larger step size by taking the gradient Lipschitz continuity into the algorithm design. We prove that our method achieves faster convergence when the initial point is far away from the optimal solution.

math.OC↗

arXiv2025

Convergence Dynamics and Stabilization Strategies of Co-Evolving Generative Models

The increasing prevalence of synthetic data in training loops has raised concerns about model collapse, where generative models degrade when trained on their own outputs. While prior work focuses on this self-consuming process, we study an underexplored yet prevalent phenomenon: co-evolving generative models that shape each other's training through iterative feedback. This is common in multimodal AI ecosystems, such as social media platforms, where text models generate captions that guide image models, and the resulting images influence the future adaptation of the text model. We take a first step by analyzing such a system, modeling the text model as a multinomial distribution and the image model as a conditional multi-dimensional Gaussian distribution. Our analysis uncovers three key results. First, when one model remains fixed, the other collapses: a frozen image model causes the text model to lose diversity, while a frozen text model leads to an exponential contraction of image diversity, though fidelity remains bounded. Second, in fully interactive systems, mutual reinforcement accelerates collapse, with image contraction amplifying text homogenization and vice versa, leading to a Matthew effect where dominant texts sustain higher image diversity while rarer texts collapse faster. Third, we analyze stabilization strategies implicitly introduced by real-world external influences. Random corpus injections for text models and user-content injections for image models prevent collapse while preserving both diversity and fidelity. Our theoretical findings are further validated through experiments.

cs.LG↗

arXiv2025

Parallel Multi-Coordinate Descent Methods for Full Configuration Interaction

We develop a multi-threaded parallel coordinate descent full configuration interaction algorithm (mCDFCI), for the electronic structure ground-state calculation in the configuration interaction framework. The FCI problem is reformulated as an unconstrained minimization problem, and tackled by a modified block coordinate descent method with a deterministic compression strategy. mCDFCI is designed to prioritize determinants based on their importance, with block updates enabling efficient parallelization on shared-memory, multi-core computing infrastructure. We demonstrate the efficiency of the algorithm by computing an accurate benchmark energy for the chromium dimer in the Ahlrichs SV basis (48e, 42o), which explicitly includes $2.07 \times 10^9$ variational determinants. We also provide the binding curve of the nitrogen dimer under the cc-pVQZ basis set (14e, 110o). Benchmarks show up to $79.3\%$ parallel efficiency on 128 cores.

physics.chem-ph↗

arXiv2024

A mixed precision Jacobi SVD algorithm

We propose a mixed precision Jacobi algorithm for computing the singular value decomposition (SVD) of a dense matrix. After appropriate preconditioning, the proposed algorithm computes the SVD in a lower precision as an initial guess, and then performs one-sided Jacobi rotations in the working precision as iterative refinement. By carefully transforming a lower precision solution to a higher precision one, our algorithm achieves about 2 times speedup on the x86-64 architecture compared to the usual one-sided Jacobi SVD algorithm in LAPACK, without sacrificing the accuracy.

math.NA↗

arXiv2024

How Do Flow Matching Models Memorize and Generalize in Sample Data Subspaces?

Real-world data is often assumed to lie within a low-dimensional structure embedded in high-dimensional space. In practical settings, we observe only a finite set of samples, forming what we refer to as the sample data subspace. It serves an essential approximation supporting tasks such as dimensionality reduction and generation. A major challenge lies in whether generative models can reliably synthesize samples that stay within this subspace rather than drifting away from the underlying structure. In this work, we provide theoretical insights into this challenge by leveraging Flow Matching models, which transform a simple prior into a complex target distribution via a learned velocity field. By treating the real data distribution as discrete, we derive analytical expressions for the optimal velocity field under a Gaussian prior, showing that generated samples memorize real data points and represent the sample data subspace exactly. To generalize to suboptimal scenarios, we introduce the Orthogonal Subspace Decomposition Network (OSDNet), which systematically decomposes the velocity field into subspace and off-subspace components. Our analysis shows that the off-subspace component decays, while the subspace component generalizes within the sample data subspace, ensuring generated samples preserve both proximity and diversity.

cs.LG↗

arXiv2024

Resource Efficient Boolean Function Solver on Quantum Computer

Nonlinear boolean equation systems play an important role in a wide range of applications. Grover's algorithm is one of the best-known quantum search algorithms in solving the nonlinear boolean equation system on quantum computers. In this paper, we propose three novel techniques to improve the efficiency under Grover's algorithm framework. A W-cycle circuit construction introduces a recursive idea to increase the solvable number of boolean equations given a fixed number of qubits. Then, a greedy compression technique is proposed to reduce the oracle circuit depth. Finally, a randomized Grover's algorithm randomly chooses a subset of equations to form a random oracle every iteration, which further reduces the circuit depth and the number of ancilla qubits. Numerical results on boolean quadratic equations demonstrate the efficiency of the proposed techniques.

quant-ph↗

arXiv2024

Fast randomized algorithms for low-rank matrix approximations with applications in global comparative analysis of a class of data sets

Generalized singular values (GSVs) play an essential role in the comparative analysis. In the real world data for comparative analysis, both data matrices are usually numerically low-rank. This paper proposes a randomized algorithm to first approximately extract bases and then calculate GSVs efficiently. The accuracy of both basis extration and comparative analysis quantities, angular distances, generalized fractions of the eigenexpression, and generalized normalized Shannon entropy, are rigursly analyzed. The proposed algorithm is applied to both synthetic data sets and the genome-scale expression data sets. Comparing to other GSVs algorithms, the proposed algorithm achieves the fastest runtime while preserving sufficient accuracy in comparative analysis.

math.NA↗

arXiv2024

MEP: Multiple Kernel Learning Enhancing Relative Positional Encoding Length Extrapolation

When the predicted sequence length exceeds the length seen during training, the transformer's inference accuracy diminishes. Existing relative position encoding methods, such as those based on the ALiBi technique, address the length extrapolation challenge exclusively through the implementation of a single kernel function, which introduces a constant bias to every post-softmax attention scores according to their distance. These approaches do not investigate or employ multiple kernel functions to address the extrapolation challenge. Drawing on the ALiBi approach, this study proposes a novel relative positional encoding method, called MEP, which employs a weighted average to combine distinct kernel functions(such as the exponential kernel and the Gaussian kernel) to generate a bias that is applied to post-softmax attention scores. Initially, the framework utilizes various kernel functions to construct multiple kernel functions. Each kernel function adheres to a consistent mean weight coefficient, harnessing the synergistic advantages of different kernels to formulate an innovative bias function. Subsequently, specific slopes are tailored for each kernel function, applying penalties at varying rates, to enhance the model's extrapolation capabilities. Finally, this bias is seamlessly incorporated as a penalty to the post-softmax scores. We present two distinct versions of our method: a parameter-free variant that requires no new learnable parameters, which enhances length extrapolation capabilities without compromising training efficiency, and a parameterized variant capable of integrating state-of-the-art techniques. Empirical evaluations across diverse datasets have demonstrated that both variants of our method achieve state-of-the-art performance, outperforming traditional parameter-free and parameterized approaches.

cs.LG↗