arXiv subjects

Sanghyun Lee

Publications and source records attributed to Sanghyun Lee.

At least 19 recordsLinked to original sources

arXiv2026

A Conjugate Gradient Formulation of the EnKF Algorithm

Ensemble Kalman Filter (EnKF) based data assimilation algorithms synthesize predictive numerical forecast models with accumulated data as time evolves and account for model uncertainty and noisy measurements. The computational cost of these algorithms can be expensive, in particular for highly dimensional dynamical systems. Often, EnKF based algorithms have traded accuracy for reduced computational cost. In this paper, we present a novel parallelizable Conjugate Gradient-based Ensemble Kalman Filter (CGD-EnKF) algorithm that maintains comparable computational cost to efficient algorithms while realizing better state estimation accuracy in select cases. Here, we established the new approach by reformulating a matrix inverse calculation with a classical Conjugate Gradient (CGD) method. In addition, we discuss the upper error bound under CGD, error convergence to the classical EnKF result, and the computational complexity of the algorithm. We also showcase the CGD-EnKF-Reduced algorithm that is shown to be further computationally efficient for highly dimensional dynamical systems under small ensemble formulation. Numerical examples demonstrate the performance of our proposed algorithms and analytical properties, highlighting their comparability and advantages with respect to some benchmark EnKF algorithms.

math.NA↗

arXiv2026

PATTON: Enabling Commodity PIM for Production LLM Serving

Processing-in-Memory (PIM) is promising for accelerating memory-bound decode attention, but attention acceleration alone is insufficient for production LLM serving, where engines dynamically allocate, populate, share, cache, and reclaim logical KV cache blocks. Supporting this lifecycle on commodity PIM requires efficient physical memory allocation, block-to-address mapping, and command generation. For the Value cache, these requirements create a fundamental conflict among GEMV efficiency, single-token write efficiency, and memory capacity: GEMV-optimized layouts scatter newly generated Value vectors across rows, making writes costly, while finer-grained memory sharing improves capacity utilization but fragments GEMV reductions. We present PATTON, a PIM runtime that integrates production LLM serving engines with commodity PIM. PATTON introduces hierarchical granule allocation: block-sized Key and Value granules map one-to-one to logical token blocks, fixing their physical placements and commands, while coarser granules group blocks for efficient GEMV execution and memory utilization. A Commit Zone stages partial Value blocks for efficient single-token writes before committing them to GEMV-optimized locations. PATTON tracks these placements to generate KV cache writes and QK-transpose/SV commands. Across attention execution and runtime-induced prefill recomputation, PATTON achieves an average 1.95x speedup and 4.83x higher energy efficiency over evaluated baselines, requires no PIM processing-unit modifications, and maintains a KV cache hit rate comparable to the native GPU KV cache in vLLM.

cs.AR↗

arXiv2026

Understanding and Accelerating the Training of Masked Diffusion Language Models

Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more slowly than ARMs, which may become problematic when scaling MDMs to larger models. Therefore, we ask the following question: how can we accelerate standard MDM training while maintaining its final performance? To this end, we first provide a detailed analysis of why MDM training is slow. We find that the main factor is the locality bias of language: the predictive information for a token is concentrated in nearby positions. We further investigate how this bias slows learning and suggest a simple yet effective remedy: bell-shaped time sampling as a training strategy. Notably, MDMs trained with our training recipe reach the same validation negative log-likelihood (NLL) up to $\sim4\times$ faster than standard training on One Billion Word Benchmark (LM1B). We also show faster improvements in generative perplexity, zero-shot perplexity, and downstream task performance on various benchmarks.

cs.LG↗

arXiv2026

Optimal mean-time path planning for unmanned underwater vehicles: a Hamilton-Jacobi approach

Unmanned underwater vehicles (UUV) integrate ocean forecasts with path planning algorithms in order to identify energy- or time-minimizing paths that enable mission completion. Typically, a well-defined deterministic ocean forecast is assumed to be available for path planning; however, in practice, different ocean forecasts can disagree. In this paper, we extend previous work on deterministic optimal path planning to identify optimal mean-time paths when presented with an ensemble of possible ocean forecasts. In particular, we formulate a system of time-independent Hamilton-Jacobi partial differential equations that incorporates forecast uncertainty and yields the optimal mean reachability travel time and the necessary controls to find the associated optimal path. An efficient numerical solution of this system of PDEs is obtained through an extension of the Fast Sweeping Method; verification and benchmarking results are provided. Additional numerical examples illustrate the impact uncertainty can have on the optimal path; in particular, these results demonstrate that the vehicle's optimal path can deviate significantly from the deterministic optimal paths associated with the individual ensemble members.

math.OC↗

arXiv2026

A Posteriori Error Estimation for Parabolic Equations with Enriched Galerkin Finite Element Methods

This paper introduces a novel a posteriori error estimation framework for the enriched Galerkin (EG) finite element method applied to linear parabolic equations. While the EG method has been recognized for its local conservation property and computational efficiency compared to discontinuous Galerkin methods, its mathematical analysis in the context of a posteriori error estimation for parabolic problems remains unexplored. In this work, we prove reliability and efficiency using the residual-based approach. Furthermore, we integrate these error estimators into an adaptive mesh refinement strategy, demonstrating their effectiveness in achieving efficient and reliable error control through several numerical examples. The proposed approach provides a significant advancement in the mathematical foundation and practical applicability of the EG method for time-dependent problems.

math.NA↗

arXiv2026

Online Spectral Deflation for State Constrained Optimal Control Problems

Parametric PDE-constrained optimal control with pointwise state constraints requires repeated solution of restricted Schur-complement systems on parameter-dependent inactive sets. In a primal active-set method, each inactive-set system is symmetric positive definite, but the active set can change nonsmoothly with the parameter. The resulting operator may vary in dimension, sparsity pattern, and spectrum, limiting reuse of sparse factorizations, multigrid hierarchies, and Krylov information. We propose a reusable spectral-deflation strategy anchored to one full-domain reference Schur complement. Low reference eigenmodes are computed once, restricted online to each inactive set, and used as an A-DEF2 deflation basis for Jacobi-preconditioned CG. The framework also supports POD enrichment, Rayleigh-Ritz reselection, coarse-grid or analytical reference modes, and conditioning safeguards. Given the active set, the method preserves the high-fidelity inactive-set system and solves it to the prescribed CG tolerance; it accelerates the linear algebra rather than replacing the optimal-control solve with a surrogate. We explain the method through a spectral-coherence view, motivated by interlacing and perturbation arguments and assessed with principal-angle diagnostics. Across diffusion, convection-diffusion, nonlinear thermal, and conjugate-heat-transfer benchmarks, deflation reduces CG iterations by about 55 to 98 percent. GPU deployments also show wall-time gains over CPU sparse-direct and algebraic-multigrid baselines, because the reference basis is built once whereas competing solver structures are rebuilt per instance. Coarse-grid or analytical modes amortize the offline cost within a single parameter sweep; fine-grid eigensolves remain more precompute-limited. Timings isolate the inactive-set linear-solve kernel; reducing the active-set outer loop is outside the present scope.

cs.CE↗

arXiv2026

Enhancing neural network extrapolation in thermo-fluid systems using steady-state solutions

Time-dependent partial differential equations (PDEs) arise in many engineering systems, including thermo-fluid applications. Classical numerical simulations of such systems can become computationally expensive for long-time dynamics because they typically require sequential time integration with time steps constrained by stability, accuracy, or nonlinear solvers. Although scientific machine learning provides an alternative for approximating PDE solutions, standard neural network approximations often degrade when extrapolated beyond the training time interval. In this work, we propose a steady-state-informed neural network representation for dissipative PDE systems whose solutions relax toward a stationary equilibrium. The proposed ansatz decomposes the solution into a steady-state component and a transient correction modulated by a time-dependent decay profile. When the decay profile vanishes at long time and the transient correction remains bounded, the representation embeds convergence to the prescribed steady state directly into the architecture, rather than enforcing it through an additional penalty term. This allows the network to learn the transient dynamics while preserving the correct asymptotic behavior. We implement the approach within a physics-informed neural network (PINN) framework and train the resulting model using the SOAP optimizer. The method is evaluated on a sequence of problems of increasing physical and geometric complexity, ranging from the one-dimensional heat equation to incompressible Navier-Stokes flow in a lid-driven cavity, natural convection in a square cavity, and a full three-dimensional conjugate heat transfer problem. The numerical results show that the steady-state-informed architecture substantially improves temporal extrapolation beyond the training interval compared with architectures that do not explicitly enforce the asymptotic condition.

cs.CE↗

arXiv2026

A Parallel and Adaptive Mesh-Free Method for Discontinuous Coefficient Fields in Heterogeneous Porous Media

Discontinuous coefficient fields arise in many computational physics problems and are often represented as cellwise constant data tied to a given spatial discretization. Such representations are inherently mesh-dependent, requiring interpolation or projection whenever they are transferred to a different discretization. In this work, we develop \emph{Parallel and Adaptive Mesh-Free Approximation (PAM)}, a mesh-independent framework that approximates discontinuous data by a continuous, closed-form function. The resulting approximation can be evaluated consistently across different geometries and numerical discretizations, while preserving sharp interface features. The proposed PAM framework employs radial basis functions (RBFs) to construct continuous approximations of discontinuous data. To accurately capture discontinuities, we incorporate Shepard-normalization, which stabilizes the approximation near sharp interfaces. The coefficients of the RBF expansion are determined via sparse regression, enabling automatic selection of the most relevant basis functions and promoting robust representations. In addition, we develop a novel adaptive refinement approach which further enriches the approximation in regions of rapid spatial variation. We provide a theoretical analysis showing that the proposed normalized RBF framework achieves arbitrarily small $L^1$ error in approximating discontinuous step functions. To enhance computational efficiency, the domain is partitioned into subdomains, and the reconstruction problem is solved independently on each subdomain in parallel. Numerical experiments demonstrate the accuracy, adaptivity, scalability, and downstream impact of the proposed method on Darcy flow simulations, including tests on heterogeneous permeability fields, mesh-transfer settings, and the SPE10 benchmark.

math.NA↗

arXiv2026

Looped Diffusion Language Models

Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models for language modeling, yet the effective design of transformer architectures for MDMs remains underexplored. In this paper, we show that selectively looping the early-middle transformer layers significantly improves both training efficiency and model performance in MDMs. We call this approach LoopMDM(Looped Masked Diffusion Model), which brings two key benefits: looping layers at training-time yields a depth-scaling effect without adding parameters, while varying the number of loops at inference-time enables flexible compute scaling. Despite the simplicity, the results are striking: across multiple pre-training corpora, LoopMDM matches the performance of same-size MDMs with up to 3.3 fewer training FLOPs, while its final performance outperforms them on various reasoning benchmarks, including up to 8.5 points on GSM8K. It even surpasses deeper non-looped MDMs trained with comparable per-step compute, indicating that selective looping is more effective than naive depth scaling. Furthermore, LoopMDM can scale inference-time compute by increasing the number of loops. Adaptively adjusting the number of loops throughout the sampling process further yields additional gains in compute efficiency while maintaining performance. Lastly, with attention analysis, we provide evidence that looping is effective in MDMs by promoting interactions among masked positions. Our code and weights will be publicly released.

cs.LG↗

arXiv2026

Unifying Masked Diffusion Models with Various Generation Orders and Beyond

Masked diffusion models (MDMs) are a potential alternative to autoregressive models (ARMs) for language generation, but generation quality depends critically on the generation order. Prior work either hard-codes an ordering (e.g., blockwise left-to-right) or learns an ordering policy for a pretrained MDM, which incurs extra cost and can yield suboptimal solutions due to the two-stage optimization. Motivated by this, we propose order-expressive masked diffusion model (OeMDM) for a broad class of diffusion generative processes with various generation orders, enabling the interpretation of MDM, ARM, and block diffusion in a single framework. Furthermore, building on OeMDM, we introduce learnable-order masked diffusion model (LoMDM), which jointly learns the generation ordering and diffusion backbone through a single objective from scratch, enabling the diffusion model to generate text in context-dependent ordering. Empirically, we confirm that LoMDM outperforms various discrete diffusion models across multiple language modeling benchmarks.

cs.LG↗

arXiv2026

Heat Transfer Modeling in Enhanced Geothermal Energy: A Three-Temperature Approach for Solid, Injected, and Residing Fluids

Enhanced geothermal systems (EGS) involve strongly coupled, advection-dominated flow and heat transfer in fractured porous media. Conventional models typically assume local thermal equilibrium with a single effective fluid temperature or, at best, an averaged pore-fluid temperature, so the thermal evolution of injected cold fluid is only inferred indirectly. In this work, we develop a local thermal non-equilibrium (LTNE) model that explicitly resolves the temperature of injected fluid as it moves through the reservoir and exchanges heat with the hot rock and resident fluid. The key ingredient is a concentration variable that tracks the injected fluid and induces a three-way LTNE coupling among rock, resident-fluid, and injected-fluid temperatures. This framework distinguishes, at the continuum scale, how newly injected fluid parcels are heated by conductive and convective exchange, and predicts production-well temperatures without relying on bulk averages. To discretize the resulting nonlinear, advection-dominated system, we employ an enriched Galerkin (EG) finite element method for Darcy flow, temperature, and concentration, providing local mass conservation with relatively few degrees of freedom. We further design a flux-corrected transport (FCT) strategy for the EG concentration and temperature equations to enforce a discrete maximum principle and suppress nonphysical oscillations while preserving local conservation. Time integration uses an IMPES-type splitting combined with a strong-stability-preserving Runge--Kutta scheme. Numerical experiments for fractured EGS problems show that the proposed LTNE--EG--FCT framework captures injected-fluid heating paths and thermal breakthrough behavior not resolved by standard single-temperature or averaged LTNE models.

math.NA↗

arXiv2026

Discontinuous Galerkin finite element operator network for solving non-smooth PDEs

We introduce Discontinuous Galerkin Finite Element Operator Network (DG--FEONet), a data-free operator learning framework that combines the strengths of the discontinuous Galerkin (DG) method with neural networks to solve parametric partial differential equations (PDEs) with discontinuous coefficients and non-smooth solutions. Unlike traditional operator learning models such as DeepONet and Fourier Neural Operator, which require large paired datasets and often struggle near sharp features, our approach minimizes the residual of a DG-based weak formulation using the Symmetric Interior Penalty Galerkin (SIPG) scheme. DG-FEONet predicts element-wise solution coefficients via a neural network, enabling data-free training without the need for precomputed input-output pairs. We provide theoretical justification through convergence analysis and validate the model's performance on a series of one- and two-dimensional PDE problems, demonstrating accurate recovery of discontinuities, strong generalization across parameter space, and reliable convergence rates. Our results highlight the potential of combining local discretization schemes with machine learning to achieve robust, singularity-aware operator approximation in challenging PDE settings.

math.NA↗

arXiv2025

Pressure-robust enriched Galerkin finite element methods for coupled Navier-Stokes and heat equations

We propose a pressure-robust enriched Galerkin (EG) finite element method for the incompressible Navier-Stokes and heat equations in the Boussinesq regime. For the Navier-Stokes equations, the EG formulation combines continuous Lagrange elements with a discontinuous enrichment vector per element in the velocity space and a piecewise constant pressure space, and it can be implemented efficiently within standard finite element frameworks. To enforce pressure robustness, we construct velocity reconstruction operators that map the discrete EG velocity field into exactly divergence-free, H(div)-conforming fields. In particular, we develop reconstructions based on Arbogast-Correa (AC) mixed finite element spaces on quadrilateral meshes and demonstrate that the resulting schemes remain stable and accurate even on highly distorted grids. The nonlinearity of the coupled Navier-Stokes-Boussinesq system is treated with several iterative strategies, including Picard iterations and Anderson-accelerated iterations; our numerical study shows that Anderson acceleration yields robust and efficient convergence for high Rayleigh number flows within the proposed framework. The performance of the method is assessed on a set of benchmark problems and application-driven test cases. These numerical experiments highlight the potential of pressure-robust EG methods as flexible and accurate tools for coupled flow and heat transport in complex geometries.

cs.CE↗

arXiv2025

A Machine Learning and Finite Element Framework for Inverse Elliptic PDEs via Dirichlet-to-Neumann Mapping

Inverse problems for Partial Differential Equations (PDEs) are crucial in numerous applications such as geophysics, biomedical imaging, and material science, where unknown physical properties must be inferred from indirect measurements. In this work, we present a new approach to solving the inverse problem for elliptic PDEs, using only boundary data. Our method leverages the Dirichlet-to-Neumann (DtN) map, which captures the relationship between boundary inputs and flux responses. This enables the reconstruction of the unknown physical properties within the domain from boundary measurements alone. Our framework employs a self-supervised machine learning algorithm that integrates a Finite Element Method (FEM) in the inner loop for the forward problem, ensuring high accuracy. Moreover, our approach illustrates its effectiveness in challenging scenarios with only partial boundary observations, which is often the case in real-world scenarios. In addition, the proposed algorithm effectively handles discontinuities by incorporating carefully designed loss functions. This combined FEM and machine learning approach offers a robust, accurate solution strategy for a broad range of inverse problems, enabling improved estimation of critical parameters in applications from medical diagnostics to subsurface exploration.

math.NA↗

arXiv2025

Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement

Test-time scaling through reward-guided generation remains largely unexplored for discrete diffusion models despite its potential as a promising alternative. In this work, we introduce Iterative Reward-Guided Refinement (IterRef), a novel test-time scaling method tailored to discrete diffusion that leverages reward-guided noising-denoising transitions to progressively refine misaligned intermediate states. We formalize this process within a Multiple-Try Metropolis (MTM) framework, proving convergence to the reward-aligned distribution. Unlike prior methods that assume the current state is already aligned with the reward distribution and only guide the subsequent transition, our approach explicitly refines each state in situ, progressively steering it toward the optimal intermediate distribution. Across both text and image domains, we evaluate IterRef on diverse discrete diffusion models and observe consistent improvements in reward-guided generation quality. In particular, IterRef achieves striking gains under low compute budgets, far surpassing prior state-of-the-art baselines.

cs.LG↗

arXiv2025

Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models

Masked Diffusion Models (MDMs) as language models generate by iteratively unmasking tokens, yet their performance crucially depends on the inference time order of unmasking. Prevailing heuristics, such as confidence based sampling, are myopic: they optimize locally, fail to leverage extra test-time compute, and let early decoding mistakes cascade. We propose Lookahead Unmasking (LookUM), which addresses these concerns by reformulating sampling as path selection over all possible unmasking orders without the need for an external reward model. Our framework couples (i) a path generator that proposes paths by sampling from pools of unmasking sets with (ii) a verifier that computes the uncertainty of the proposed paths and performs importance sampling to subsequently select the final paths. Empirically, erroneous unmasking measurably inflates sequence level uncertainty, and our method exploits this to avoid error-prone trajectories. We validate our framework across six benchmarks, such as mathematics, planning, and coding, and demonstrate consistent performance improvements. LookUM requires only two to three paths to achieve peak performance, demonstrating remarkably efficient path selection. The consistent improvements on both LLaDA and post-trained LLaDA 1.5 are particularly striking: base LLaDA with LookUM rivals the performance of RL-tuned LLaDA 1.5, while LookUM further enhances LLaDA 1.5 itself showing that uncertainty based verification provides orthogonal benefits to reinforcement learning and underscoring the versatility of our framework. Code will be publicly released.

cs.LG↗

arXiv2025

Where and How to Perturb: On the Design of Perturbation Guidance in Diffusion and Flow Models

Recent guidance methods in diffusion models steer reverse sampling by perturbing the model to construct an implicit weak model and guide generation away from it. Among these approaches, attention perturbation has demonstrated strong empirical performance in unconditional scenarios where classifier-free guidance is not applicable. However, existing attention perturbation methods lack principled approaches for determining where perturbations should be applied, particularly in Diffusion Transformer (DiT) architectures where quality-relevant computations are distributed across layers. In this paper, we investigate the granularity of attention perturbations, ranging from the layer level down to individual attention heads, and discover that specific heads govern distinct visual concepts such as structure, style, and texture quality. Building on this insight, we propose "HeadHunter", a systematic framework for iteratively selecting attention heads that align with user-centric objectives, enabling fine-grained control over generation quality and visual attributes. In addition, we introduce SoftPAG, which linearly interpolates each selected head's attention map toward an identity matrix, providing a continuous knob to tune perturbation strength and suppress artifacts. Our approach not only mitigates the oversmoothing issues of existing layer-level perturbation but also enables targeted manipulation of specific visual styles through compositional head selection. We validate our method on modern large-scale DiT-based text-to-image models including Stable Diffusion 3 and FLUX.1, demonstrating superior performance in both general quality enhancement and style-specific guidance. Our work provides the first head-level analysis of attention perturbation in diffusion models, uncovering interpretable specialization within attention layers and enabling practical design of effective perturbation strategies.

cs.CV↗

arXiv2025

Observation of the crossover from quantum fluxoid to half-quantum fluxoid in a chiral superconducting device

Topological superconductors are one of the intriguing material groups from the viewpoint of not only condensed matter physics but also industrial application such as quantum computers based on Majorana fermion. For the real application, developments of the thin-film topological superconductors are highly desirable. Bi/Ni bilayer is a possible candidate for thin-film chiral superconductors where the time-reversal symmetry is broken. Here we report the phase shift of resistance oscillations by half flux quantum in a ring-shaped device of epitaxial Bi/Ni bilayer induced by a small magnetic field through the ring. The half quantum fluxoid can be a decisive evidence for unconventional superconductors where the superconducting order parameter has an internal degree of freedom. The present result provides a functional operating principle for quantum devices where the phase of the supercurrent can be shifted by πwith a small magnetic field, based on the internal degree of freedom possessed by topological superconductivity.

cond-mat.supr-con↗